navSEO

Here is a little SEO bit that some of the even more experienced SEOs may not have heard of. The order of the meta robots tag is incredibly important. Did you know that the effects are cumulative when it comes to Google understanding the restrictions set in the tag?

Google’s JohnMu said in a Google Webmaster Help thread:

For the robots meta tag, the effects are cumulative with regards to the restrictions, eg:

would result in Googlebot treating it as a noindex, nofollow, noodp, noimageindex. This is different than the robots.txt file. You cannot provide more restrictive directives for the generic “robots” setting than for individual crawlers.

So if you need more granular control over controlling robots, use the robots.txt file.

Forum discussion at Google Webmaster Help and hat tip to Colin McDermott.

Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!

SEO Google Webmaster Help, little seo bit, robots, Robots META tag, txt file

Here is a little SEO bit that some of the even more experienced SEOs may not have heard of. The order of the meta robots tag is incredibly important. Did you know that the effects are cumulative when it comes to Google understanding the restrictions set in the tag?

Google’s JohnMu said in a Google Webmaster Help thread:

For the robots meta tag, the effects are cumulative with regards to the restrictions, eg:

would result in Googlebot treating it as a noindex, nofollow, noodp, noimageindex. This is different than the robots.txt file. You cannot provide more restrictive directives for the generic “robots” setting than for individual crawlers.

So if you need more granular control over controlling robots, use the robots.txt file.

Forum discussion at Google Webmaster Help and hat tip to Colin McDermott.

Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!

SEO Google Webmaster Help, little seo bit, robots, Robots META tag, txt file

Robots plays important role in the field of SEO. We have two ways to control pages and folders one is using Robots META tag and other is through robots.txt

A web page creator can specify which page should be indexed and which page should not be indexed by search engines by placing Robots META tag in the HTML section

Here are some Robots tags that are common

- Ignore content and follow links
- Include content and do not follow links
- Ignore content and do not follow links
- Include content and follow links
- Cache link should not show Search results pages
- The Open Directory Project (ODP) title and description for the page should not be displayed in Search results
- The Yahoo Directory title and description for the page should not be displayed in Search results
- Titles are only displayed in Search results page and not description or text context for this page

In addition to manage folder level user agent control robots.txt file can be used. This file can be placed in root of each server and the format is plain text not HTML

Through this file website owner or webmaster can allow access to web page content and disallow access to admin, cgi and any secured files that you dont want search engines to index

A typical robots.txt file will look like

User-agent: *

Allow: /

Disallow: /admin*

Allow: *content*

Disallow: /test/

Disallow: /paypal/

Disallow: /credit/

Disallow: /cgi/

Explains, all robots can crawl except the admin files, and crawl files named content folder, and should not crawl test, paypal, credit and cgi folder.

Hope this article helps to know more about robots and its act in SEO. You can also analyze further on checking Google’s robots.txt file http://www.google.com/robots.txt

Source: community.zdnet.co.uk

SEO disallow access, robots, Robots META tag, txt file, typical robots