NAME

App::Wubot::Plugin::WebFetch - fetch content from a URL

VERSION

version 0.3.6

SYNOPSIS

  ~/wubot/config/plugins/WebFetch/mypage.yaml

  ---
  delay: 24h
  url: http://myweb.com/somepage.html
  field: content

DESCRIPTION

Fetch content from a web page at regular intervals.

The retrieved content will be stored in the 'field' specified in the config. If no field is specified, the default target field will be 'content'.

GITHUB TRAFFIC

I like to keep track of how many hits some of my projects have received on github. I haven't found any sort of a feed that will provide that data. The only place I have seen that is on the 'traffic' graph page. So I use the following config to pull that page once per day, grab the number of hits and send me a notification.

  ---
  delay: 24h
  url: https://github.com/wu/wubot/graphs/traffic

  react:

    - name: content
      condition: contains content
      rules:

        - name: get page views
          plugin: CaptureData
          config:
            source_field: content
            regexp: '^.*?Page Views \((\d+) over last 90 days\)'
            target_field: hits

        - name: build subject
          plugin: Template
          config:
            template: wubot: hits last 90 days: {$hits}
            target_field: subject

        - name: color
          plugin: SetField
          config:
            set:
              color: purple
              sticky: 1

GRAPHING ROUTER TRAFFIC

My DSL router does not provide any mechanism to graph the traffic sent or received. However it does provide a web page where it lists the the total number of packets sent and received. So I use the following config which captures the sent and received packets every 5 minutes.

  ---
  delay: 5m
  url: http://192.168.1.98/cgi-bin/webcm?getpage=../html/lan_status.html

  react:

    - name: content
      condition: contains content
      rules:

        - name: get packets sent
          plugin: CaptureData
          config:
            source_field: content
            regexp: '^.*?Packets Sent:</td>.*?<td\>(\d+)'
            target_field: sent

        - name: get packets received
          plugin: CaptureData
          config:
            source_field: content
            regexp: '^.*?Packets Received:</td>.*?<td>(\d+)'
            target_field: recv

Then I use the following rule in the reactor to graph the data using RRD:

  - name: WebFetch RRD
    condition: key matches ^WebFetch-router
    plugin: RRD
    config:
      base_dir: /usr/home/wu/wubot/rrd
      fields:
        sent: COUNTER
        recv: COUNTER
      period:
        - day
        - week
        - month
      graph_options:
        lower_limit: 0
        right-axis: 1:0
        width: 375

SEE ALSO

This plugin uses App::Wubot::Util::WebFetcher.

SUBROUTINES/METHODS

check( $inputs )

The standard monitor check() method.