Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Advanced Search
  • 0

How to block nefarious web spiders


prjctfish

Question

Site URL: https://www.simonmetzwoodworking.com

I have an issue where a scam site from China has a page that appears to link to my site but in actuality links to their home page. I need to block this as it is some kind of scan financial services site and don't want to get dinged by Google. Normally I would block spiders in robots.txt file but that's not available in SS. Anyone have any ideas on how to block spiders. I used to bloc Baidu when I coded my own site as their bot spent hours indexing my site from what my logs told me.

 

The URL I'm trying to block is https://m.pushade.com/by/simonmetzwoodworking/

 

Thanks!

Screen Shot 2021-09-19 at 3.04.18 PM.png

Edited by prjctfish
Link to comment

2 answers to this question

Recommended Posts

  • 0

I don't think you can block a url on someone else's page. In reality however the url you posted doesn't actually exist on the site, its an empty URL redirect. You could substitute any website domain for yours and it will still go to the sites home page, and then your broswer cache remembers that redirect as if it were a real url. For example you can substitute facebook for simonwetz.... and the site behavior is the same. 

Edited by derricksrandomviews
Link to comment
  • 0

Thanks, didn't see that. I was trying to block the chinese spiders so that cannot even see my site. When I coded my own site my logs showed that their spiders would spend hours on my sites which keeps the servers working for no good reason, so I just blocked them wholesale 🙂

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

×
×
  • Create New...