Changes for version 0.02 - 2011-12-24

  • improved documentation near by exported functions if a robots.txt line is only "Disallow:", don't consider it as "Disallow: /"

Modules

database of robots.txt-derived permissions. This is a fork of WWW::RobotRules

Provides

in lib/WWW/RobotRules/Extended.pm