- Joined
- Aug 3, 2003
- Messages
- 2,909
- Reaction score
- 91
The 'junk crawlers' are getting out of hand -- about 80% of my server bandwidth is wasted by crawlers trying to steal content, email addresses, or information about my sites. I'm paying for the cpu load and bandwidth, and they get the data free.
I've been looking over my logs at the biggest offenders, and one of those is WHOIS.SC, AKA Name Intelligence. I went to their site, I see this warning
"Automated Queries are strictly forbidden.
Anonymous queries for whois reports limited to 25 records.
Free membership is required for unlimited access as this helps control automated bots."
In other words, these people are complete hypocrits!
Any reason NOT to ban these people? Any others ban-worthy? Next on my list are those "trademark check" systems. I may have a long ban list
I've been looking over my logs at the biggest offenders, and one of those is WHOIS.SC, AKA Name Intelligence. I went to their site, I see this warning
"Automated Queries are strictly forbidden.
Anonymous queries for whois reports limited to 25 records.
Free membership is required for unlimited access as this helps control automated bots."
In other words, these people are complete hypocrits!
Any reason NOT to ban these people? Any others ban-worthy? Next on my list are those "trademark check" systems. I may have a long ban list
