[clug] googlebot doing funny things in logs

[Scott Ferguson]

On Fri, 17 Jun 2011 22:29:34 +1000 Paul Wayper wrote:

> 
> On 06/15/2011 11:55 AM, Hal Ashburner wrote:
>> > I changed my server on the weekend and after restoring the mythtv database
>> > forgot to put the usual
>> > User-agent: *
>> > Disallow: /
>> > in a robots.txt file.
>> > I was just glancing through some logs and in amongst one or two seemingly
>> > fairly unsophisticated attempts at entry, google bot made an appearance.
> Are you sure it was from Google?  Check the IP addresses it's reading from.

First thing I checked when I read his post. It was genuine.

> One of the script kiddie pranks is to disguise their bots as Google and other
> search engines so that people let them through.  It might be a probe for a
> vulnerability in Mythweb.
> 
> HTH,
> 
> Paul