Jump to content

Can't seem to rectify these crawling issues

Recommended Posts

Hey guys, 

Just need some help troubleshooting. These URL's keep appearing in SEMRush as being blocked from crawling. I've tried several fixes but can't for the life of me work it out.

Hoping someone can see something I can't! Links below.

Cheers, 

Mikayla

 

https://www.steffantp.com.au/cart

https://www.steffantp.com.au/council-updates?author=5d76de1369eed51bfad3f250

 
 

https://www.steffantp.com.au/news?author=5d76de1369eed51bfad3f250


 
 

https://www.steffantp.com.au/news?offset=1557285960746&reversePaginate=true

 
 
https://www.steffantp.com.au/projects?author=5d76de1369eed51bfad3f250
   
 
     
  • 1
Link to comment
  • Replies 1
  • Views 717
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Hi Mikayla - those URLs are blocked from crawling deliberately by Squarespace. You can see the rules that block them in the robots.txt file generated by Squarespace for your site: https://steffantp.com.au/robots.txt

It looks like most of these are just preventing Google's crawlers from wasting time on the archive pages associated with authors and collection items.

Unless you really want those pages to be crawled (and then potentially indexed) by Google then you can safely ignore these issues in Semrush 🙂

Hope that helps!

- Ed

--

Blue Hills Digital
Support with website optimization, SEO and finding a digital marketing strategy that works.

Latest resource: Squarespace SEO Services

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment


×
×
  • Create New...

Squarespace Webinars

Free online sessions where you’ll learn the basics and refine your Squarespace skills.

Hire a Designer

Stand out online with the help of an experienced designer or developer.