Jump to content

Help pls - Google Indexing new SQSP Site: Failed: Blocked by robots.txt

Go to solution Solved by Collaborada,

Recommended Posts

Hi,

https://www.ashleymichaelbriggs.com/

Site's been up for a month. Have tried Google Indexing twice, both failed. This is frying my mind - I feel like I'm back at high school physics class. Please can some one offer some tips. Thanks in advance. (Domain is hosted off squarespace if that has anything to do with it. And I have another URL pointing to this one, ashleybriggs.co.uk)

When I type in https://www.ashleymichaelbriggs.com/robots.txt or use a robots.txt Validator and Testing Tool I get this, which seems ok, right?;

# Squarespace Robots Txt

User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: CCBot
User-agent: anthropic-ai
User-agent: Google-Extended
User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
User-agent: AdsBot-Google-Mobile-Apps
User-agent: *
Disallow: /

Sitemap: https://www.ashleymichaelbriggs.com/sitemap.xml

 

I get a green Valid / Status '200 OK' on this testing tool.

 

Using Google URL Inspection it results in either:

Blocked due to unauthorized request (401)

 

Or;

URL is not available to Google

Page cannot be indexed: Blocked by robots.txt
Discovery
Not checked in live tests
Crawl
Time
Feb 21, 2024, 12:11:41 PM
Crawled as
Google Inspection Tool smartphone
Crawl allowed?
error
No: blocked by robots.txt
Page fetch
error
Failed: Blocked by robots.txt
Indexing allowed?
N/A
Indexing
User-declared canonical
N/A
Google-selected canonical
 
Only determined after indexing
Link to comment
  • Replies 2
  • Views 1.5k
  • Created
  • Last Reply

Top Posters In This Topic

  • Solution

Hi @Ash_B based on above, you disallowed crawling. Visit Settings > Crawlers and toggle back on. 

Have SEO questions? Chances are we've written about it! Try a quick Google search to find our advice. Or for personalized support, see our consulting and training sessions.  Official Squarespace Experts since 2014 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

×
×
  • Create New...

Squarespace Webinars

Free online sessions where you’ll learn the basics and refine your Squarespace skills.

Hire a Designer

Stand out online with the help of an experienced designer or developer.