[clug] googlebot doing funny things in logs
scott.ferguson.clug at gmail.com
Fri Jun 17 19:59:27 MDT 2011
On Fri, 17 Jun 2011 22:29:34 +1000 Paul Wayper wrote:
> On 06/15/2011 11:55 AM, Hal Ashburner wrote:
>> > I changed my server on the weekend and after restoring the mythtv database
>> > forgot to put the usual
>> > User-agent: *
>> > Disallow: /
>> > in a robots.txt file.
>> > I was just glancing through some logs and in amongst one or two seemingly
>> > fairly unsophisticated attempts at entry, google bot made an appearance.
> Are you sure it was from Google? Check the IP addresses it's reading from.
First thing I checked when I read his post. It was genuine.
> One of the script kiddie pranks is to disguise their bots as Google and other
> search engines so that people let them through. It might be a probe for a
> vulnerability in Mythweb.
More information about the linux