Indexing Structured Data

When you make a search with a search engine you see a preview of the contents of the web page in the search result. Depending on the search engine you will see the title of the page followed by a short summary of the contents. Some search engines, such as AddSearch, also provide a thumbnail to accommodate the textual summary of the web page.

In this article, we will discuss how to use meta tags to optimize search results. We will first look at the most common meta tags and continue to the custom meta tags that AddSearch supports.

Meta tags

Meta tags hold information on how the browser should render the page.  They also let you control what the crawler of a search engine should index and thus also return in the search results.

Meta tags are located within the head section of a web page.

<html>
  <head>
    <meta name="description" content="Summary of the web page.">
    <meta property="og:image" content="https://example.com/image.jpg">
  </head>
<body>
</body>
</html>

Meta tags are void elements, which means that they only have a start tag. What this means is that their contents are stored in the attributes within the tag. Meta tags start with metatag and have at least name and content attributes.

In the context of optimizing the search results also Open Graph protocol (og) meta tags are useful. Open Graph meta tags require replacing the name attribute with the property attribute.

For the purposes of this article, we have grouped the meta tags into three categories that allow you to

  • Decide what to index
  • Choose what to in the search results
  • Use metadata for filtering the search results

Search engines, such as Google and Bing, commonly utilize the two first meta tag categories. The last category, however, focuses on the meta tags that AddSearch provides for optimizing the search results filtering.

Decide what to index

The robots meta tag allows you to control the way in which your web page should be indexed. The default setting for robots meta tags is to allow all crawling. Thus if you wish to have the page crawled don’t add the robots meta tag on the page.

The first example disallows the crawling of the page from all crawlers. This approach can be used when you don’t want the page indexed at all. The meta tag has the following syntax:

<meta name="robots" content="noindex">

The second example allows the crawling of the page but disallows the crawler from following the links. This approach can be used to help prevent indexing spammy links on pages, where content is created by the visitors.

<meta name="robots" content="nofollow">

The third example disallows the crawling of the page specifically from crawler using ‘googlebot’ as its user agent. With this approach, you can choose which search engines should index and thus return the page in the search results.

<meta name="googlebot" content="noindex">

The robots meta tag is the most rudimentary of the meta tags as it lets you control whether to index a page. This also determines whether the page will be shown in the search results.

Choose what to show in the search results

You can summarize the contents of your web page in a meta tag to make sure that the search result will turn out exactly the way you wanted. A meta tag can also link directly to a picture which returns along with the search results provided that the search engine supports images. You can achieve these results using the description and the image meta tags.

Description

The purpose of description and open graph description (og:description) meta tags is to provide a summary of the contents of a webpage. The summary in the description and og:description is shown in the search results of a search engine.

The description and og:description meta tags have the following syntax:

<meta name="description" content="A summary of your web page.">
<meta property="og:description" content="A summary of your web page.">

Both of the meta tags, descriptions, and og:description are self-explanatory. However, instead of the name attribute, used with the description meta tag, the og:description uses the property attribute. The content in both cases should include a short summary of the web page.

You can set up to show the meta description from your Dashboards Setup, Look & Feel under Show meta descriptions in search results.

Image

Open graph image (og:graph) meta tag allows you to control what image tp use in the search results with a visual search engine.

<meta property="og:image" content="http://example.com/page-image.jpg">

The meta tag consists of the og:image property attribute followed by the URL of the image that should be shown in the search results.

You can set up the image for the search results from your Dashboards Setup, Documents and crawling under Featured images. When the Featured image is enabled you need to initiate a full re-crawl. Re-crawl can be started from the Index tools section.

Use metadata for filtering search results

The custom fields meta tag is more versatile than the category meta tag. It allows you to define multiple fields with multiple values and use them for filtering the searches.

A use case for the custom field meta tag could be a website selling books. On the website, the custom field metadata is defined throughout all of the product pages. The custom fields have definitions for the genre (fiction, non-fiction, thriller, mystery, self-help), the format (e-book, paperback, hardcover). The product page includes the author’s name as well.

You can separate values with a semicolon or a separate line. The custom meta tag has the following syntax:

<meta name="addsearch-custom-field" content="author=Sigmund Freud">
<meta name="addsearch-custom-field" content="genre=non-fiction;genre=psychology">
<meta name="addsearch-custom-field" content="format=hardcover">
<meta name="addsearch-custom-field" content="format=e-book">

The custom fields, spanning across the product pages, make it possible to search based on genre, format and the author. To find out how to set up the filtering visit these instructions.

Additional information

For additional information on indexing meta keywords please visit here.

Was this helpful?

Need more help?

We’re always happy to help with code or other questions you might have. Search our documentation, contact support, or connect with our sales team.