Jump to content

How to block nefarious web spiders

Recommended Posts

Site URL: https://www.simonmetzwoodworking.com

I have an issue where a scam site from China has a page that appears to link to my site but in actuality links to their home page. I need to block this as it is some kind of scan financial services site and don't want to get dinged by Google. Normally I would block spiders in robots.txt file but that's not available in SS. Anyone have any ideas on how to block spiders. I used to bloc Baidu when I coded my own site as their bot spent hours indexing my site from what my logs told me.


The URL I'm trying to block is https://m.pushade.com/by/simonmetzwoodworking/



Screen Shot 2021-09-19 at 3.04.18 PM.png

Edited by prjctfish
Link to comment

I don't think you can block a url on someone else's page. In reality however the url you posted doesn't actually exist on the site, its an empty URL redirect. You could substitute any website domain for yours and it will still go to the sites home page, and then your broswer cache remembers that redirect as if it were a real url. For example you can substitute facebook for simonwetz.... and the site behavior is the same. 

Edited by derricksrandomviews
Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

  • Create New...

Squarespace Webinars

Free online sessions where you’ll learn the basics and refine your Squarespace skills.

Hire a Designer

Stand out online with the help of an experienced designer or developer.