Real-time Indexing with RSS Feeds or API

AddSearch provides an RSS-feed feature and Search API for adding your content immediately to the search index.

The RSS feed feature allows you to define RSS feed containing new content on your site. Once defined, we poll the feed frequently, and as soon as new content appears in the feed, we add it to the search.

When AddSearch polls the RSS-feed, it looks for content that was published or updated since the last poll. Thus you only need to have updated content on your RSS-feed. For RSS crawling to work correctly, please, make sure that items in the feed contain either publication or updated date! See the sample feeds below for an example.

Tom make sure that AddSearch crawls all content from your website, we recommend using sitemaps.

RSS feeds feature is useful on websites with rapid content changes. For instance, the frequent changes in content on news sites and large online stores need immediate adding to the search so that the user can find the content using the search.

The support for the RSS feeds is available to Professional, Premium and Enterprise plans.

Instructions

To enable the RSS feed on your account, follow these instructions:

1. Create the RSS-feed on your site (max 2 MB, preferably smaller)
2. Login to your AddSearch Account
3. Go to Domains and crawling under Setup
4. Under RSS-feeds click Add new feed +
5. Add your RSS Feed URL in the input field
6. Click Save to submit changes

The RSS-feed will now be polled every 5 minutes.

We recommend using formats Atom 1.0 or RSS 2.0 in that order. Below you can find examples of both feeds.

Atom-feed example:

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">

    <title>Example Feed</title>
    <link href="https://www.addsearch.com/feed"/>
    <id>https://www.addsearch.com/feed</id>
    <updated>2019-05-29T14:09:42Z</updated>

    <entry>
        <title>Product Updates</title>
        <id>https://www.addsearch.com/product/updates/</id>
        <link href="https://www.addsearch.com/product/updates/"/>
        <published>2017-05-22T09:00:00Z</published>
        <updated>2019-05-29T11:12:39Z</updated>
    </entry>
</feed>

RSS-feed example:

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
    <channel>
        <title>A simple RSS feed</title>
        <link>https://www.addsearch.com/feed/</link>
        <description>AddSearch example feed</description>
        <lastBuildDate>Mon, 17 Jun 2019 01:45:45 GMT</lastBuildDate>
        
        <item>
            <title>Product Updates</title>
            <link>https://www.addsearch.com/product/updates/</link>
            <pubDate>Sun, 16 Jun 2019 13:00:00 GMT</pubDate>
        </item>
    </channel>
</rss>

Crawl API endpoint

You can also use our API’s crawl endpoint to add, remove or modify content. We will re-crawl the URL immediately and update your index in real-time.