App::JsonLogUtils - Command line utilities for dealing with JSON-formatted log files


version 0.03


  # From the command line
  tail -f /path/to/log/file.log \
    | jgrep -m message="some pattern" \
    | jcut -f "timestamp priority message" \
    | cols -c "timestamp priority message" -s '|' \
    | column -t -s '|'

  # From code
  use App::JsonLogUtils qw(tail json_log);

  my $log = json_log tail '/path/to/file.log';

  while (my $entry = <$log>) {
    my ($json, $line) = @$entry;

  # Grepping JSON logs
  use App::JsonLogUtils qw(lines json_log);
  use Iterator::Simple qw(igrep imap);

  my $entries = igrep{ $_->{foo} =~ /bar/ } # filter objects
                imap{ $_->[0] }             # select the object
                json_log                    # parse
                lines '/path/to/file.log';  # read


Writing logs in JSON, one object per line, makes them very easily machine readable. Wonderful. Unfortunately, it also makes it unfuriating to deal with them using the standard unix command line tools. This package provides a few tools to salve the burn.



Greps patterns in individual object fields.


Filter the fields included in objects.


Display fields in a format suitable for column.


An interactive shell for monitoring JSON log files.


If desired, the iterators used to implement the tools above are optionally exported by the main module.


Accepts a file path or opened file handle and returns an iterator which yields the chomped lines from the file.

  my $log = lines '/path/to/file.log';

  while (my $line = <$log>) {


Accepts a file path or opened file handle and returns an iterator while yields chomped lines from the file as they are appended, starting from the end of the file. Lines already written to the file when this routine is first called are ignored (that is, there is no equivalent to tail -c 10 at this time).

  my $log = tail '/path/to/file.log';

  while (my $line = <$log>) { # sleeps until lines appended to file


Accepts a file iterator (see "tail" and "lines") and returns an iterator yielding an array ref holding two items, a hash ref of the parsed JSON log entry, and the original log entry string. Empty lines are skipped with a warning. JSON decoding errors are ignored with a warning.

  my $lines = json_log tail '/path/to/file.log';

  while (my $entry = <$lines>) {
    my ($object, $line) = @_;


Accepts a list of fields (as a space-separared string or array ref of strings), a string separator, and an iterator over JSON object strings, and returns a new iterator. The returned iterator will first yield a string of column names joined by the separator string. Subsequent calls will iterate over the JSON object strings, return the value of each of the selected fields joined by the separator string.

  # File $input
  {"a": 1, "b": 2, "c": 3}
  {"a": 4, "b": 5, "c": 6}
  {"a": 7, "b": 8, "c": 9}

  # Select columns a and c, separated by a pipe
  my $cols = json_cols "a c", "|" , lines $input;

  # ...yields the following strings:


Accepts a space-separated string or array ref of $fields, boolean $inverse, and an iterator of JSON log lines. Returns an iterator yielding objects containing only the fields selected in $fields. If $inverse is true, instead yields objects containing only the fields not contained in $fields.

Give the same input as the previous example:

  my $cut = json_cut "a c", 0, lines $input;

  # ...yields the following hash refs:
  {a => 1, c => 3}
  {a => 4, c => 6}
  {a => 7, c => 9}

  # Inverted
  my $cut = json_cut "a c", 1, lines $input;

  # ...yields:
  {b => 2}
  {b => 5}
  {b => 8}


Accepts a hash ref where keys are field names and values are arrays of regular expressions, a boolean $inverse, and an iterator of JSON object strings. Returns an iterator yielding array refs of the parsed JSON object hash and the original string (just like "json_log"). Only those entries for which all fields' patterns match are returned. If $inverse is set, the logic is negated and only those entries for which all patterns test false are returned.

  # File $input
  {"foo": "bar"}
  {"foo": "baz"}
  {"foo": "bat"}
  {"foo": "BAR"}

  # Code
  my $entries = json_grep { foo => [qw/bar/i, qr/baz/] }, 0, lines $input;

  # ...yields the following:
  [ {foo => "bar"}, '{"foo": "bar"}' ]
  [ {foo => "baz"}, '{"foo": "baz"}' ]
  [ {foo => "BAR"}, '{"foo": "BAR"}' ]


None, but will happily consider requests and patches.


Jeff Ober <>


This software is copyright (c) 2018 by Jeff Ober.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.