Comments on: Keeping the bots out

By: jidanni

jidanni — Fri, 20 Feb 2009 05:18:59 +0000

I use WWWOFFLE, and libwww-perl’s HEAD and GET, etc. to fetch and then
browse later offline all the time, even though I now have ADSL. So if
I’m blocked from somebody’s site, well, it’s their loss.

By: David Carrero Fdez-Baillo

David Carrero Fdez-Baillo — Thu, 18 Oct 2007 07:02:56 +0000

I use modsecurity for all sites, and block many attacks :))

By: Dirk Haun

Dirk Haun — Sat, 13 Oct 2007 08:37:26 +0000

As others have noted, a lot of the script kiddies do use the unmodified libwww-perl user agent string, so it does help a lot to simply block that.

Also, they often try these include tricks with each and every PHP script that takes a parameter. Just because they try doesn’t mean an actual exploit exists for that script.

By: Nick

Nick — Fri, 12 Oct 2007 18:40:37 +0000

Abel, David,
My guess would be, that unless there’s a top-secret 0day out there ;) It’s more likely that some script-kiddy didn’t get what the vulnerability is, and therefore is pointless trying to include.

By: David Kierznowski

David Kierznowski — Fri, 12 Oct 2007 11:03:03 +0000

Abel, yes I know of the wp-pass redirect vulnerability that Adrian released on BlogSec a couple months back; however, the vulnerability they are trying to exploit here is a remote file include bug.

By: Abel Cheung

Abel Cheung — Fri, 12 Oct 2007 10:37:47 +0000

You're asking what's that wp-pass thing? Interesting to see that you forgot your own post 3 months ago. :-D About user-agent, I think Donncha actually got a point. From statistics of my own site, the user-agent from spam messages surprisingly concentrates on just a few choices (though most of them have null user-agent string). Not the ultimate solution, but can still block a noticable portion of the spam messages.

By: James McKay

James McKay — Thu, 11 Oct 2007 21:16:36 +0000

I saw Donncha’s post and I think you’ve missed the point of it somewhat. The point wasn’t to eliminate all the nasties altogether — you will need to rely on other solutions for that — but reduce the load on your server. The traffic that it blocks is all coming from bots, which are churning out requests by the millions to try and cause as widespread damage as they can.

This technique may not stop all bot attacks, but it is not entirely useless.

Your point about RSS aggregation agents may be valid however, though do any really legitimate ones actually use the libwww-perl user agent, or is that more the domain of splogs and things, as I would have thought to be the case? In any case, if there are some legitimate LWP aggregators out there, I’m sure it must be possible to come up with some .htaccess rules that allow them access to your RSS feeds while denying them everything else.

By: Gareth Heyes

Gareth Heyes — Thu, 11 Oct 2007 21:01:26 +0000

Our new WPIDS will protect against these attacks :)

They are generic scans against PHP code and not based on any Wordpress code, I get these a lot.

By: Donncha O Caoimh

Donncha O Caoimh — Thu, 11 Oct 2007 21:00:23 +0000

I beg to differ. Even though it’s trivial to change the UA they obviously haven’t. I’m certainly not the first to urge for blocking libwww which means the exploit authors either don’t use Google, or realise only a small number of sites actually block the UA.

Looking forward to the modsecurity post!