How to Fix Indexed, though blocked by robots.txt [ Permanently ]

How to Fix Indexed, though blocked by robots.txt [ Permanently ] 

Hello to everyone, I am Sumit Pratap Yadav, 

In this blog, we will see A very common problem Google Search Console Related It is called Indexed Through Blocked by Robots.txt. First of all, we will see What is this Indexed Through Blocked by Robots.txt. This is such a file that tells the search engine that they can be called on your website and cannot go So this file lists all such contents. Which you want to lock with a search engine like Google With the help of this file you other than Google Can tell search engines that can tell search engines that How to crawl allowed content. 

For the latest updates install my app — Smartest Blogger 

So basically the Robots.txt file Instructs these search engine crawlers or bots How to browse your website For the purpose of indexing They do this with the help of user agents Shown on screen The next part is This robots.txt file Why is needed You on screen Can see many reasons for example Duplicate content Keeping some sections of the website private Stop crawling Keeping some sections of the website private Prevent internal search result page from crawling Prevent server overload Images, videos and research files Preventing Google search results Google does not typically index those webpages. That is blocked in the Robots.txt file. What is this Indexed Through Blocked by Robots.txt error! it means Some Robots.txt file blocked content Still indexed in google. 

In simple words, This error means This indicates that Google has found your page But he has also got an instruction that Ignores it in your specific Robots.txt file. Which means that it will not appear on the results page. Sometimes this error can be intentional Or accidental may be due to different reasons But it can be fixed Now we will see how we can fix this error Now we will see how we can fix this error First you open google search console Click on the 'Start now' option. 

This is your Google Search Console window Click on the website for which you want to check the error Click on the website for which you want to check the error Click on the website for which you want to check the error Click on 'Coverage' Now here you can see an issue Click there Now here you can see a special error Indexed through blocked By Robots.txt. 

Now go back to your blogger website Login with the same username and password from google search console Click on 'Settings' Now click on "search preferences" Now click on "search inclinations" tendencies" Here you can see the Custom Robot header marks. Click on the 'Edit' link Click on the Yes option Click on Yes option. Now select the options shown on the screen from each category here -

Choose 'all' and 'noodp' from the 'Home page'. 

Choose 'noindex' and 'noodp' from 'Archive and Search pages'. 

Choose 'All' and 'noodp' from 'Default for Pages and Posts'. 

Click on 'Save changes' Click on 'Edit' in the 'Custom robots.txt' section. Click on the Yes option Now here you have to add Blogger Sitemap And this is your custom robots.txt content And this is your custom robots.txt content. 

For the latest updates install my app — Smartest Blogger 

Just goes to Google and type Blogger Sitemap in the search box Click on the first link Enter your full blogger URL here And click on 'Generate Sitemap' This is the XML sitemap for your blog Just select and copy it Just select and copy it Paste it here in the "Customs robots.txt" section of the blog And make some changes to the content as shown here Now add 'category' and 'tag' in front of two more 'Disallow' sections. And click on 'Save changes'. Now here you can see that Customs robots.txt and Custom robot header tags have been enabled. 

They were previously disabled When you have this type of error in Google search console When you have this type of error in Google search console When you have this type of error in Google search console Google sends you an email notifying you of this type of error Google sends you an email notifying you of this type of error Here you can see the name of the error. 

Now we have fixed this error in our Blogger section So we can click on 'Fix Coverage issues' So we can click on 'Fix Coverage issues' This opens your Google search console window Here it shows you the related URL and error 

For the latest updates install my app — Smartest Blogger 

Here it shows you the related URL and error Click on the link 'VALIDATE FIX' Now click on 'SEE DETAILS' Here you can see the pending request sent by you on Google search console for correction Here you can search Google for correction. 

Best Custom robots.txt Code - 

At the place of my website URL, you can place your URL. 

User-agent: Mediapartners-Google 

Disallow: User-agent: * Disallow: /search/labels/ 

Allow: / 

Sitemap: https://sumitpratapyadav.blogspot.com//atom.xml?redirect=false&start-index=1&max-results=500 

🙏🙏🙏