NAME

Robots::Validate - Validate that IP addresses are associated with known robots

SYNOPSIS

use Robots::Validate;

my $rv = Robots::Validate->new;

...

if ( my $res = $rs->validate( $ip, $user_agent ) ) {
   ...
}

DESCRIPTION

This module allows one to validate a robot user-agent string against the IP addresses.

RECENT CHANGES

Changes for version v0.3.8 (2026-08-27)

• Enhancements
  • Reverted back to using XS modules. There is no point using pure-Perl alternatives if this already uses Net::Patricia.
  • Updated robot rules.
• Bug Fixes
  • Fixed _check_dns so that it correctly returns false instead of undef if there is no match.
• Documentation
  • Updated KNOWN ISSUES.
• Tests
  • Add a "live" test with Google DNS.

See the Changes file for more details.

REQUIREMENTS

This module lists the following modules as runtime dependencies:

• Algorithm::AhoCorasick::XS
• File::ShareDir
• File::Slurper
• List::Util
• Moo version 1 or later
• Net::DNS::Resolver
• Net::IP
• Net::Patricia
• PerlX::Maybe
• Ref::Util
• Sub::Util version 1.40 or later
• TOML::XS
• Try::Tiny
• Types::Common
• constant
• experimental
• namespace::autoclean
• perl version v5.24.0 or later

See the cpanfile file for the full list of prerequisites.

CHI is required to use the caching features.

INSTALLATION

The latest version of this module (along with any dependencies) can be installed from CPAN with the cpan tool that is included with Perl:

cpan Robots::Validate

You can also extract the distribution archive and install this module (along with any dependencies):

cpan .

You can also install this module manually using the following commands:

perl Makefile.PL
make
make test
make install

If you are working with the source repository, then it may not have a Makefile.PL file. But you can use the Dist::Zilla tool in anger to build and install this module:

dzil build
dzil test
dzil install --install-command="cpan ."

For more information, see How to install CPAN modules.

SECURITY CONSIDERATIONS

When using the "cache", ensure that it is configured to expire the data by setting "expires_in" in CHI and digest the keys by setting "max_key_length" in CHI to 0. This is to keep the cache from growing too large, and to reduce the likelihood of cache backend vulnerabilities being exploited through user-agent strings.

When setting the cache_failure option, be aware that cached failures may need a shorter expiration time.

When specifying a domain for verification rules, ensure that there is an initial dot in the suffix, or that the regular expression matches the entire domain name. Otherwise imposter domains with the same suffix will be validated.

When the network list contains cloud addresses, it is important to regularly update the addresses from the documented information, as an imposter can use abandoned cloud IP addresses.

SUPPORT

Only the latest release of this module will be supported.

This module requires Perl v5.24 or later. Future releases may only support Perl versions released in the last ten (10) years.

Reporting Bugs and Submitting Feature Requests

Please report any bugs or feature requests on the bugtracker website https://github.com/robrwo/Robots-Validate/issues

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

If the bug you are reporting has security implications which make it inappropriate to send to a public issue tracker, then see SECURITY.md for instructions how to report security vulnerabilities.

SOURCE

The development version is on github at https://github.com/robrwo/Robots-Validate and may be cloned from https://github.com/robrwo/Robots-Validate.git

AUTHOR

Robert Rothenberg perl@rhizomnic.com

Some of the development of this module was sponsored by Science Photo Library https://www.sciencephoto.com.

COPYRIGHT AND LICENSE

This software is Copyright (c) 2018-2026 by Robert Rothenberg.

This is free software, licensed under:

The Artistic License 2.0 (GPL Compatible)

SEE ALSO

The file robots.toml included with this distribution contains links to documented rules.

The TOML specification can be found at https://toml.io.