The Crawler Status prompt displays the following information in the Dashboard page of the AddSearch dashboard.

• Scheduled for recrawl: the number of URLs scheduled for crawling or recrawling
• Status: the status of the crawlers

Queue

Some of the URLs scheduled for recrawl will not be indexed. For instance, robots.txt rules, server-side permissions, and redirects may prevent our crawlers’ from accessing the URLs. Also, some pages might be detected as duplicates and excluded from the index.

If the number of items scheduled for recrawl is 0, we haven’t found any other pages to index and your index is up-to-date. However, if the number of URLs scheduled for recrawl increases and doesn’t decrease within a couple of days, contact our support.

Status

• Crawling: the crawlers are crawling URLs
• Starting soon: the crawlers are not crawling URLs

AddSearch runs crawling in batches. Our crawlers check for pages queued for recrawling and update them regularly. As this process runs continuously in the background, usually the “Starting soon” status is displayed in the Dashboard.

When you start a full recrawl the “Starting soon” status is displayed until the next crawl batch starts.