mikaylastp Posted March 11, 2022 Share Posted March 11, 2022 Hey guys, Just need some help troubleshooting. These URL's keep appearing in SEMRush as being blocked from crawling. I've tried several fixes but can't for the life of me work it out. Hoping someone can see something I can't! Links below. Cheers, Mikayla https://www.steffantp.com.au/cart https://www.steffantp.com.au/council-updates?author=5d76de1369eed51bfad3f250 https://www.steffantp.com.au/news?author=5d76de1369eed51bfad3f250 https://www.steffantp.com.au/news?offset=1557285960746&reversePaginate=true https://www.steffantp.com.au/projects?author=5d76de1369eed51bfad3f250 1 10 Link to comment
edharris Posted March 11, 2022 Share Posted March 11, 2022 Hi Mikayla - those URLs are blocked from crawling deliberately by Squarespace. You can see the rules that block them in the robots.txt file generated by Squarespace for your site: https://steffantp.com.au/robots.txt It looks like most of these are just preventing Google's crawlers from wasting time on the archive pages associated with authors and collection items. Unless you really want those pages to be crawled (and then potentially indexed) by Google then you can safely ignore these issues in Semrush 🙂 Hope that helps! - Ed -- Blue Hills DigitalSupport with website optimization, SEO and finding a digital marketing strategy that works. Latest resource: Squarespace SEO Services Link to comment
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment