The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
Virani-1.1.0
River stage zero No dependents
++
/ Virani

NAME

Virani - PCAP retrieval for a FPC setup writing to PCAP files.

VERSION

Version 1.1.0

SYNOPSIS

    use Virani;

    my $virani = Virani->new();
    ...

METHODS

new_from_conf

Initiates the Virani object from the specified file.

    - conf :: The config TOML to use.
        - Default :: /usr/local/etc/virani.toml

new

Initiates the object.

    - allowed_subnets :: The allowed subnets for fetching PCAPs for mojo-varini.
        Defaults :: [ '192.168.0.0/', '127.0.0.1/8', '::1/127', '172.16.0.0/12' ]

    - apikey :: Optional API key for mojo-varini.
        Defaults :: undef

    - auth_by_IP_only :: Auth by IP only and don't use a API key.
        Default :: 1

    - default_set :: The default set to use.
        Default :: default

    - cache :: Cache directory to write to.
        Default :: /var/cache/virani

    - default_regex :: The regex to use for getting the timestamp. The regex to pass to
                       File::Find::IncludesTimeRange for finding PCAP files with timestamps
                       that include the range in question.
        Default :: (?<timestamp>\\d\\d\\d\\d\\d\\d+)(\\.pcap|(?<subsec>\\.\\d+)\\.pcap)$

    - verbose_to_syslog :: Send verbose items to syslog. This is used by mojo-virani.
        Default :: 0

    - pcap_glob :: The glob to use for matching files.
        Default :: *.pcap*

    - ts_is_unixtime :: The timestamp is unixtime and does not requires additional processing.
        Default :: 1

    - verbose :: Print verbose info.
        Default :: 1

    - type :: Either tcpdump, tshark, or bpf2tshark, which to use for filtering PCAP files in the
              specified time slot. tcpdump is faster, but in general will not nicely handles
              some VLAN types. For that tshark is needed, but it is signfigantly slower. bpf2tshark
              is handled via Virani->bpf2tshark and that should be seen for more info on that.
        Default :: tcpdump

    - padding :: How many seconds to add to the start and end time stamps to ensure the specified
                 time slot is definitely included.
        Default :: 5

    - sets :: A hash of hashes of available sets.
        Default :: { default => { path => '/var/log/daemonlogger' } }

For sets, the following keys are usable, of which only path is required.

    - path :: The base path of which the PCAPs are located.

    - padding :: Padding value for this set.

    - regex :: The timestamp regex to use with this set.

    - type :: The default filter type to use with this set.

    - ts_is_unixtime :: The timestamp is unixtime and does not requires additional processing.

bpf2tshark

Does a quick and dumb conversion of a BPF filter to tshark.

    my $tshark=$virani->bpf2tshark($bpf);



    ()  ->  ()
    not ()  ->  !()

    icmp -> icmp
    tcp -> tcp
    udp -> udp

    port $port -> ( tcp.port == $port or udp.port == $port )
    not port $port -> ( tcp.port != $port or udp.port != $port )

    dst port $port -> ( tcp.dstport == $port or udp.dstport == $port )
    not dst port $port -> ( tcp.dstport != $port or udp.dstport != $port )

    src port $port -> ( tcp.srcport == $port or udp.srcport == $port )
    not src port $port -> ( tcp.srcport != $port or udp.srcport != $port )

    host $host -> ip.addr == $host
    not host $host -> ip.addr != $host

    dst host $host -> ip.dst == $host
    not dst host $host -> ip.dst != $host

    src host $host -> ip.src == $host
    not src host $host -> ip.src != $host

    dst $host -> ip.dst == $host
    not host $host -> ip.dst != $host

    src src $host -> ip.src == $host
    not src $host -> ip.src != $host

filter_clean

Removes starting and trailing whitespace as well as collapsing consecutive whitespace to a single space.

The purpose for this is to make sure that tshark/BPF filters passed are consistent for cacheing, even if their white space differs.

A undef passed to it will return ''.

Will die if the filter matches /^\w*\-/ as it starts with a '-', which tcpdump will interpret as a switch.

    my $cleaned_bpf=$virani->filter_clean($bpf);

check_apikey

Checks the API key.

If auth_via_IP_only is 1, this will always return true.

        my $apikey=$c->param('apikey');
        if (!$virani->check_apikey($apikey)) {
                $c->render( text => "Invalid API key\n", status=>403, );
                return;
        }

check_remote_ip

Checks if the remote IP is allowed or not.

    if ( ! $virani->check_remote_ip( $c->{tx}{original_remote_address} )){
                $c->render( text => "IP or subnet not allowed\n", status=>403, );
                return;
    }

check_type

Verify if the check is valid or note

Returns 0/1 based on if it a known type or not.

    if ( ! $virani->check_type( $type )){
        print $type." is not known\n";
    }

get_default_set

Returns the deefault set to use.

    my $set=$virani->get_default_set;

get_cache_file

Takes the same args as get_pcap_lcal.

Returns the path to the file.

    my $cache_file=$virani->get_cache_file(%opts);
    if (! -f $cache_file.'json'){
        echo "Cache file metadata does not exist, so either get_pcap_local died or it has not been ran\n";
    }

get_pcap_local

Generates a PCAP locally and returns the path to it.

    - start :: A L<Time::Piece> object of when to start looking.
        - Default :: undef

    - end :: A L<Time::Piece> object of when to stop looking.
        - Default :: undef

    - padding :: Number of seconds to pad the start and end with.
        - Default :: 5

    - filter :: The BPF or tshark filter to use.
        - Default :: ''

    - set :: The PCAP set to use. Will use what ever the default is set to if undef or blank.
        - Default :: $virani->get_default_set

    - file :: The file to output to. If undef it just returns the path to
              the cache file.
        - Default :: undef

    - no_cache :: If cached, don't return that, but regen and if applicable re-cache.
        - Default :: 0

    - auto_no_cache :: If the cache dir is being used and not writeable and a file
                       as been specified, don't die, but use the output file name
                       as the basis of for the tmp file.
        - Default :: 1

    - type :: 'tcpdump' or 'tshark', depending on what one wants the filter todo.
        - Default :: tcpdump

The return is a hash reference that includes the following keys.

    - pcaps :: A array of PCAPs used.

    - pcap_count :: A count of used PCAPs.

    - pcap_glob :: The value of pcap_glob used.

    - ts_is_unixtime :: The value of ts_is_unixtime used.

    - failed :: A hash of PCAPs that failed. PCAP path as key and value being the reason.

    - failed_count :: A count of failed PCAPs.

    - path :: The path to the results file. If undef, unable it was unable
              to process any of them.

    - success_found :: A count of successfully processed PCAPs.

    - filter :: The used filter.

    - total_size :: The size of all PCAP files checked.

    - failed_size :: The size of the PCAP files that failed.

    - success_size :: the size of the PCAP files that successfully processed

    - type :: The value of $opts{type}

    - padding :: The value of padding.

    - start_s :: Start time in seconds since epoch, not including pading.

    - end :: Send time in the format '%Y-%m-%dT%H:%M:%S%z'.

    - end_s :: End time in seconds since epoch, not including pading.

    - end :: End time in the format '%Y-%m-%dT%H:%M:%S%z'.

    - using_cache :: If the cache was used or not.

    - req_start :: Timestamp of when the it started. In the format
                   %Y-%m-%dT%H:%M:%S%z

    - req_start_s :: Same as req_start, but unixtime.

    - req_end :: Timestamp of when the it finished. In the format
                 %Y-%m-%dT%H:%M:%S%z

    - req_end_s :: Same as req_end, but unixtime.

    - req_time :: Number of seconds it took.

get_set_path

Returns the path to a set.

If no set is given, the default is used.

Will return undef if the set does not exist or if the set does not have a path defined.

    my $path=$virani->get_set_path($set);

set_verbose

Set if it should be verbose or not.

    # be verbose
    $virani->verbose(1);

    # do not be verbose
    $virani->verbose(0);

set_verbose_to_syslog

Set if it should be verbose or not.

    # send verbose messages to syslog
    $virani->set_verbose_to_syslog(1);

    # do not send verbose messages to syslog
    $virani->set_verbose_to_syslog(0);

verbose

Prints out error messages. This is inteded to be internal.

Only sends the string if verbose is enabled.

There is no need to add a "\n" as it will automatically if not sending to syslog.

Two variables are taken. The first is level the second is the message. Level is only used for syslog. Default level is info.

    - Levels :: emerg, alert, crit, err, warning, notice, info, debug

    $self->verbose('info', 'some string');

CONFIG

The config format used toml, processed via TOML.

'new_from_conf' will initiate virani by reading it in and feeding it to 'new'.

DAEMONLOGGER ON FREEBSD

With daemonlogger setup along the lines of like below...

    daemonlogger_enable="YES"
    daemonlogger_flags="-f /usr/local/etc/daemonlogger.bpf -d -l /var/log/daemonlogger -t 120"

The following can be made available via mojo-varini or locally via varini with the set name of default as below.

    allowed_subnets=["192.168.14.0/23", "127.0.0.1/8"]
    [sets.default]
    path='/var/log/daemonlogger'

If you want to use 'init/freebsd' to start mojo-virani, you just need to copy it it to '/usr/local/etc/rc.d/virani' and add the following or the like to '/etc/rc.conf'.

    virani_enable="YES"
    virani_flags="daemon -m production -l http://127.0.0.1:8080 -l http://192.168.14.1:8080"

See the script for information on the various possible config args for it.

AUTHOR

Zane C. Bowers-Hadley, <vvelox at vvelox.net>

BUGS

Please report any bugs or feature requests to bug-virani at rt.cpan.org, or through the web interface at https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Virani. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Virani

You can also look for information at:

ACKNOWLEDGEMENTS

LICENSE AND COPYRIGHT

This software is Copyright (c) 2023 by Zane C. Bowers-Hadley.

This is free software, licensed under:

  The GNU Lesser General Public License, Version 2.1, February 1999