How to Add Custom Robots.txt File to Blogger

Allowing the Search engines to Crawl and Index your blog's post's and pages is very important thing in the terms of Search Engine Optimization(SEO).
However sometimes you may not want to index all pages and post in search engine.
Then you have to use robots.txt file.

READ - What is a robots.txt file ?

Today you will know how to change robots.txt file in blogspot,

9 Steps:

1. First of all, go to blogger.com and login using your login credentials.
2. Select the blog of whose “Robot.txt” file you wish to edit.
3. Click  the settings tab.
4. From the Settings tab, click on the “Search Preferences” sub-menu.
5. Now, from the “Search Preferences” sub-menu look out for the “Custom Robots.txt” option.
6. Click on the “Edit” button beside the “Custom Robot.txt file” option.
7. A box will appear below the option.
8. There you can add your custom robots.txt text.
9. Now if you have some knowledge about robots.txt file you can edit it yourself or try out the example below,

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://example.com/feeds/posts/default?orderby=UPDATED

Replace example.com with your blog's domain name.

Understanding This Example:

• User-agent: Mediapartners-Google: The default “Robot.txt” file won’t allow any other bots other than Search Engine bots, to crawl your blog. And if you’re using Google Adsense on your blog, then Adsense bots need to crawl your blog for ad optimization. Hence, this line tells your blog to allow Adsense bots also, to crawl your blog.
• User-agent:* : This code allows all Search Engine bots to crawl and index your blog.
• Disallow: /search : This prevents Search Engine bots from crawling labels and other pages with search after your domain on your blog.
• Allow: / : This code allows Search Engine bots to crawl the Index/Homepage of your blog.

Things To Note:

1. Don't customize the robots.txt file if you don't know much about it.
2. Wrong text may stop Search Engines from crawling and indexing your site .
3. After every customization check your Google Webmaster Tools account .

Any comments and suggestions are welcome.