- QUICK START
- AVAILABLE OPTIONS
- DATABASE STRUCTURE
- COPYRIGHT AND LICENSE
proxyhunter - free proxy searcher and checker
proxyhunter may be used to find free proxies, it performs searching using available adapters for search engines (like Google and so on). All found proxies may be checked for availability, proxy type and speed.
proxyhunter uses database as storage for found proxies, so you can easily perfrom any SELECT statement to find what you need.
First of all you should install one of the available database schema. The easiest for deploy is SQLite (but not very fast). See CPAN for other alternatives.
$ cpan App::ProxyHunter::Model::Schema::SQLite
Then you should generate configuration file and edit if needed
$ proxyhunter --create-config proxyhunter.jconf
Now it is time to create database schema
$ proxyhunter --config proxyhunter.jconf --create-schema
And finally you can start the process
$ proxyhunter --config proxyhunter.jconf
Generates default configuration file
Creates table structures inside database. This should be used with
Which config file to use for creating schema or to run
Run proxyhunter as daemon.
/path/to/pidfile may be omitted, otherwise pid of the daemon will be writed to this file. Proc::Daemon should be additionally installed for this to work.
--create-config option to create default configuration file. Configuration file format is JCONF (see Parse::JCONF).
Database settings. Available options listed below
DBI driver to use. You should have schema installed for this driver, which implemented as
App::ProxyHunter::Model::Schema subclass. See CPAN for available implementations. You can implement schema for preferred database yourself, see App::ProxyHunter::Model::Schema.
DBD driver specific options, like
mysql_auto_reconnect and so on
Host of the database, if needed
Login for database, if needed
Password for database, if needed
Proxy sercher settings. Searcher uses specified search engines to find out proxies. You can implement new search engine, see App::ProxyHunter::SearchEngine. See CPAN for available implementations.
true to enable proxy searching ,
false to disable
A list of queries which will be passed to search engine (like "free proxy", "proxy list", ...)
Which search engines to use. Search engine implementation should be in
Checker settings. Checker checks proxy for availability and type. Checking performed right after proxy was inserted into database. If proxy will not pass check (e.g. bad proxy) it will be deleted. Available options listed below
true to enable proxy checking ,
false to disable
How many workers should perform checking in parallel. Bigger value causes higher check speed and higher resources usage
Should checker perform speed checking right after successful checking for availability. If
true will perform checking even if speed checking disabled in
Rechecker settings. Rechecker checks proxy for availability and type, only proxies which passed first check by checker. Rechecker used to check proxies that passed at least one check and was not deleted from database yet (
fails_before_delete limit not exceeded). Available options listed below
Enable or disable rechecking
How many workers to use for rechecking
How often proxy may be rechecked. For example
180 means no more than each 180 seconds
Perform speed check right after recheck
How much in a row proxy check may failed before it will be deleted from database. For example
3 means that proxy will be deleted on fourth check fail
Speed checker settings. Each proxy which passed initial check by checker may be checked for a speed. Available options listed below
Enable or disable speed checking
How many workers to use for speed checking
How often to recheck proxy speed
HTTP url to some file which will be downloaded to check proxy speed. For measurment accuracy file should be at least 1 mb. Not whole file will be downloaded, but only part needed for measurment accuracy, so file may be much bigger. HTTP url used for those proxies that may be used for HTTP protocol
HTTPS url to some file which will be downloaded to check proxy speed. For measurment accuracy file should be at least 1 mb. Not whole file will be downloaded, but only part needed for measurment accuracy, so file may be much bigger. HTTPS url used for those proxies that may be used for HTTPS protocol
Database has one table called
proxy. This table has columns listed below
Proxy host as ipv4 address
Is proxy checked at least one time after inserting in the database
How many times proxy check successfully passed
How many times proxy check failed
Date and time when this proxy was inserted in the database
Date and time when last check was performed
Date and time when last speed check was performed
How much failed checks in a row was performed. Each successful check resets this counter
Type of the proxy. One of
DEAD_PROXY. Not checked proxy has
DEAD_PROXY type. See Net::Proxy::Type for proxy types description.
Is proxy in the queue for check
Connection time to this proxy host
Proxy speed in bytes per second
Oleg G, <email@example.com>
COPYRIGHT AND LICENSE
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself