Eric Hodel (drbrain) wrote,
Eric Hodel

MSNBot sucks

Favorite checkin of the week!

Tell msnbot not to be a dick

Revision:	4847
Author:		eric
Date:		2005-08-24T13:41:56-0700

Total Changed:	+ 3	- 0
WWW Changed:	+ 3	- 0
Test Changed:	+ 0	- 0

Files Modified: 1

Modified: hugster/trunk/public/robots.txt
--- hugster/trunk/public/robots.txt	2005-08-24 20:07:57 UTC (rev 4846)
+++ hugster/trunk/public/robots.txt	2005-08-24 20:41:56 UTC (rev 4847)
@@ -1,6 +1,9 @@
 # robots.txt for

+User-agent: msnbot
+Crawl-delay: 20
 User-agent: *
 Disallow: /admin
 Disallow: /auth

It dropped 10 queries in 1 second and 10 seconds later proceeded with 35 queries in 3 seconds.

  • (no subject)

    I dreamed I was in a grocery store with my mom reading Seattle Weekly from sometime late at night until 11AM. The Seattle Weekly had an article about…

  • Free Microwave! (and stuff)

    I have a convection oven microwave (this means you could bake a cake in a metal pan in it) I'd like to get rid of. If you or anyone you know would…

  • Tools!

    Today I used several of my tools. My new Collector Vehicle plate arrived, so I retired the 948-MPC plates to my closet and attached the new plate.…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.