Website Sitemap Design
Components Needed for this Project:
Now, Create a 'robots.txt' file in your "Custom robots.txt" directory. Copy the above text and paste it into the text file. concerned about your online content post which doesn't want to be indexed on the search engine (Google, Bing) being indexed on the search engines? well, the Robots.Txt generator is a handy tool. But, in Some cases, some wrong data is indexed on the search engine which you don't want people to see.
Most of search engines do not read all (all posts, pages) of the meta tags, so to be double sure you have to use the robots.txt file.
if any mistake in the format is made the search robots won't perform any task. format for a robots.txt file:
Blogger Simple:
User-agent: *
Disallow: /search/
Disallow: /category/
Disallow: /tag/
Allow: /Sitemap: https://www.yourwebsite.com/sitemap.xml
WordPress Simple:
User-Agent: *
Allow: /wp-content/uploads/
Allow: /wp-admin/admin-ajax.php
Disallow: /wp-content/plugins/
Disallow: /wp-admin/
Disallow: /blog/
Disallow: /page/
Disallow: /search/
Sitemap: https://www.yourwebsite.com/sitemap.xml
01. 1st robots.txt file: The robot's file should be the Generator of the website you want to verify. for your website robots.txt file.
02. Validate the Syntax: You can use online robis.txt validators to check the syntax of your robots.txt file. The website will analyze the file & identify any potential issues or errors. Some popular validators include Google's Robots.Txt Tester, Bing webmaster tools, and various third-party websites.
01. Firstly Visit your Blogger side.
02.After that go to the settings and click "Crawlers and indexing" option.
03. From this go to the "Enable custom robots.txt" click to enble.
03.After that paste your Robots.Txt file there to add Blogger side.
04.Then save the setting and you are done.
01.Firstly Visit your C-Panel, Select "file manager" From the click.
02. After the return to folder "Public_html". open.
03. Create a new text file and name it "robots.txt".
04.Then save the setting and you are done.
If you have any doubts then you can Contact us here.
