-
-
28 Jul 2008 10:37:52 UTC
- Browse (raw)
- Changes
- How to Contribute
- Issues (10)
- Testers (18 / 1 / 1)
- Kwalitee
Bus factor: 1- License: perl_5
- Perl: v5.8.0
- Activity
24 month- Tools
- Download (93.32KB)
- MetaCPAN Explorer
- Permissions
- Subscribe to distribution
- Permalinks
- This version
- Latest version
and 1 contributors-
Copyright (c) 2007 Daisuke Maki
- Dependencies
- Best
- Class::Accessor::Fast
- Class::C3::Componentised
- Class::Data::Inheritable
- Class::Inspector
- Config::Any
- Data::Dumper
- Event::Notify
- Exception::Class
- FindBin
- Getopt::Long
- HTTP::Request
- HTTP::Response
- HTTP::Status
- Log::Dispatch
- POE
- POE::Component::Client::DNS
- POE::Component::Client::HTTP
- POE::Component::Client::Keepalive
- Path::Class
- Pod::Usage
- Regexp::Common
- Storable
- UNIVERSAL::isa
- UNIVERSAL::require
- URI
- and possibly others
- Reverse dependencies
- CPAN Testers List
- Dependency graph
Changes for version 0.09008
- Component::Throttle
- Replace Data::Throttler with Data::Valve
Documentation
高性能WebクローラーフレームワークGunghoコンポーネント親クラスGungho認証親クラスGunghoでBasic認証を行う内部IPアドレスに解決するリクエストを拒否するキャッシュ機能を組み込むrobots.txtの処理を行うrobots.txtルールrobots.txtストレージrobots.txt情報をDB_Fileに格納するページ内のRobotsMETAをパースするスロットリング用ベースクラスドメイン/ホスト毎にスロットリングリクエスト総数でスロットルGungho用POEエンジンGunghoログモジュールGunghoの基本Gungho FAQインストール高性能クローラーフレームワークプロセスが停止する時間を指定リクエスト履歴をログするGunghoリクエストオブジェクトGunghoレスポンス オブジェクトGungho FAQTODO ItemsAn Extensible, High-Performance Web Crawler FrameworkModules
Yet Another High Performance Web Crawler FrameworkBase Class For Various Gungho ObjectsBase For Classes That Won't Be InstantiatedComponent Base Class For GunghoBase Class For WWW AuthenticationAdd Basic Auth To GunghoBlock Requests With Private IP AddressUse Cache In Your AppGungho Core MethodsRespect robots.txtA Rule ObjectRobotRules Storage Base ClassCache Storage For RobotRulesDB_File Storage For RobotRulesAutomatically Parse Robots METAWeb::Scraper From Within GunghoRoutines To Setup GunghoBase Class To Throttle RequestsThrottle By DomainThrottle Calls To The ProviderThrottle By Number Of RequestsData::Throttler Based ThrottlingBase Class For Gungho EngineGungho Engine Using Danga::SocketIO::Async EnginePOE Engine For GunghoGungho ExceptionsBase Class For Gungho HandlersWrite Out Fetched Contents To FileInline HandlerA Handler That Does NothingInline Your Providers And Handlers (Deprecated)Log Base Class For GunghoLog::Dispatch-Based Log For GunghoSimple Gungho Log ClassGungho Plugin Base ClassStop Execution In Long-Running ProcessesLog RequestsKeep Track Of Time To Finish RequestGather Crawler StatisticsFormat Statistics As XMLBase Class For Gungho PrividersProvide Requests From A Simple FileInline ProviderAn In-Memory, Simple ProviderSpecify requests in YAML formatA Gungho Request ObjectHTTP specific utilitiesGungho HTTP Response ObjectGungho General UtilitiesProvides
in lib/Gungho/Engine/IO/Async.pmin lib/Gungho/Plugin/Apoptosis.pmExamples
- examples/robotrules/simple.yml
- examples/simple-file/simple-file.yml
- examples/simple-file/url.txt
- examples/simple-log-dispatch/simple.yml
- examples/simple-write-to-file/simple-write-to-file.yml
- examples/simple/simple.yml
- examples/site-crawler/site-crawler.pl
- examples/throttle-simple/throttle-simple.yml
- examples/throttle-simple/url.txt
- examples/yaml/config.yml
- examples/yaml/url.yml
Other files
Module Install Instructions
To install Gungho, copy and paste the appropriate command in to your terminal.
cpanm Gungho
perl -MCPAN -e shell install Gungho
For more information on module installation, please visit the detailed CPAN module installation guide.