You can exclude a single document from the search index with robots.txt and robots tag. AddSearch also allows you to define exclusions using Site areas from the AddSearch dashboard.

If you have deleted a web page from your website AddSearch will automatically exclude the document from the search index. This may take some time depending on the frequency of updates.

If you wish to update the index manually after you have deleted the page follow these instructions.

Instructions

Follow the instructions to exclude single documents from the search index.

Robots.txt

To exclude obsolete-page.php page from all search engine’s search indices use the following snippet:

User-agent: *
Disallow: /obsolete-page.php

To exclude obsolete-page.php from the AddSearch search index, use the following snippet:

User-agent: AddSearchBot
Disallow: /obsolete-page.php

Robots metatag

To exclude a page from all search engine’s indices with robots meta tag, place the following snippet into HEAD section of your page:

<meta name="robots" content="noindex" />

To exclude a page from AddSearch’s index with robots meta tag, replace the robots name value with addsearchbot and place the following snippet into HEAD section of your page:

<meta name="AddSearchBot" content="noindex" />

To include a page in AddSearch’s index and exclude it from all other search engines’ indices, place the following snippet into the HEAD section of your page:

<meta name="robots" content="noindex" />
<meta name="AddSearchBot" content="index" />

Inversely, if you would like to exclude a page from the AddSearch index while still allowing other search engines to add it to their indices, this can be done by adding the following snippet to the HEAD section of your page:

<meta name="robots" content="index" />
<meta name="AddSearchBot" content="noindex" />

Site areas

You can exclude a single page using 1) Site areas to set exclusion rule and 2) updating the status of the indexed to excluded with the Index tools re-crawl with the following instructions:

1. Login to your AddSearch Account
2. Go to Site areas under Manage results
3. Click Add new url +
4. Insert the page URL you want to exclude to the input field
5. From the drop-down menu choose Excluded
6. Click Save to submit changes

To update the index after setting the rules, follow these instructions:

1. Go to the Index tools
2. Under Page status and actions insert the page URL to the input field
3. Click Check status

The contents of the indexed page appear below Page status and actions.

4. Click Recrawl URL to re-crawl the page url

The indexed page will be excluded in a few seconds time. You can check the updated status of the page with clicking the Check status button.

For more information visit our documentation on exclusions and updating single pages.