The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.

HDR EXAMPLES

This document will help you understand how to use the hdr command which is a part of the Data::HexDump::Range module. Some advanced examples use the module directly.

The latest version is at http://search.cpan.org/search?query=Data%3A%3AHexDump%3A%3ARange.

This document contains many embeded examples in HTML format. It is mean to be processed by pod2html or equivalent.

When in the hexdump land, do as hexdump does

Without range definition hdr acts like hexdump.

 >$ hdr file_name

00000000 63 6f 6d 6d 69 74 20 38 38 30 39 63 38 65 38 34  commit 8809c8e84 
00000010 65 62 33 32 33 62 62 61 61 33 36 36 39 65 39 34  eb323bbaa3669e94 
00000020 38 62 36 38 35 30 32 36 63 33 35 66 61 35 39 20  8b685026c35fa59  
00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A 
00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe 
00000050 6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir  
00000060 67 3e 0a 44 61 74 65 3a 20 20 20 53 75 6e 20 41  g>.Date:   Sun A 
00000070 70 72 20 31 38 20 30 33 3a 33 32 3a 30 33 20 32  pr 18 03:32:03 2 
00000080 30 31 30 20 2b 30 32 30 30 0a 0a 20 20 20 20 41  010 +0200..    A 

Output format

The default output format is ANSI. ASCII is like ANSI without the colors.

ASCII

  00000000 63 6f 6d 6d 69 74 20 38 38 30 39 63 38 65 38 34  commit 8809c8e84
  00000010 65 62 33 32 33 62 62 61 61 33 36 36 39 65 39 34  eb323bbaa3669e94
  00000020 38 62 36 38 35 30 32 36 63 33 35 66 61 35 39 20  8b685026c35fa59
  00000030 28 48 45 41 44 2c 20 6d 61 73 74 65 72 29 0a 41  (HEAD, master).A
  00000040 75 74 68 6f 72 3a 20 6e 61 64 69 6d 20 6b 68 65  uthor: nadim khe
  00000050 6d 69 72 20 3c 6e 6b 68 40 63 70 61 6e 2e 6f 72  mir <nkh@cpan.or
  00000060 67 3e 0a 44 61 74 65 3a 20 20 20 53 75 6e 20 41  g>.Date:   Sun A
  00000070 70 72 20 31 38 20 30 33                          pr 18 03

ANSI and HTML

Both format will generate colorized output.

Range descritption

                  format                          range example
                  
  normal range => integer                         header, 4, bright_blue
  comment      => #                               data section start, # 
  extra header => @                               header, @, red 
  bitfield     => [XInteger][xInteger]bInteger    bitfield, X2x4b4         # X is byte offset, x is bit offset
  skip range   => XInteger                        boring, X256,, comment

Note that the integer part can be a hexadecimal value starting with 0x.

Direct command line defintion

You can define your range description on the command line by passing a string to the --range_description (short -r) option.

  hdr -r 'magic, 5,   :other,37  :bf,b8   :skip,X32,, I skipped :more, 20'

Definition from a file

The --range_description option also accept a file name for a range description written in Perl format.

  hdr -r my_range_description_file

simple file

        [
          ['data header', 5, 'blue on_cyan'],
          ['data', 20, 'blue on_bright_yellow'],
        ] ;

more structured file

 use strict ;
 use warnings ;
 
 # any perl code you may need to generate your range definition
 
 my $data_range = # definition to re-use
        [
          ['data header', 5, 'blue on_cyan'],
          ['data', 20, 'blue on_bright_yellow'],
        ] ;
 
 my $structured_range = 
        [
          ['zero size', 0],
          [
            ['magic cookie', 12, 'red'],
            ['padding', 32, 'yellow'],
            $data_range, 
          ],
          ['A comment', '#'],           
          [
            ['extra data', 18, undef],
              [
              $data_range, 
              ['footer', 4, 'bright_yellow on_red'],
            ]
          ],
        ] ; 
        
 $structured_range ; # return the range description

See Also "Advanced usage"

Default dump format

   hdr  -r 'header,20 :DATA,4  :c,b24  :e,x4b5 :uninterresting, x16  :more,20' -rul -col -o ver
RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP        
                                   0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789abcdef  
header           00000000          23 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d  #--------------- 
header           00000010 00000010 2d 2d 2d 23                                      ---#             
DATA             00000014                      0a 23 20 54                              .# T         
.c               00 .. 23          -- 23 20 54    --------001000110010000001010100  .bitfield: -# T  
.e               04 .. 08          -- -- -- 05    -----------------------00101----  .bitfield: ---.  
>>uninterresting 00000018 00000027 Skipped 00 00 00 10 bytes                                         
more             00000028                                  23 0a 23 2d 2d 2d 2d 2d          #.#----- 
more             00000030 00000008 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d 2d              ------------     

Combined hexadecimal and ASCII dump

It is possible to get a combined hex/ASCII dump by giving the following options to hdr, -display_hexascii_dump 1 -display_hex_dump 0 -display_ascii_dump 0.

   hdr  -r 'header,20 :DATA,4  :c,b24  :e,x4b5 :uninterresting, x16  :more,20' -display_hexascii_dump 1 -display_hex_dump 0 -display_ascii_dump 0 -rul -col -o ver
RANGE_NAME       OFFSET   CUMULATI HEXASCII_DUMP                                                                     
                                   0    1    2    3    4    5    6    7    8    9    a    b    c    d    e    f      
header           00000000          23/# 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/-  
header           00000010 00000010 2d/- 2d/- 2d/- 23/#                                                              
DATA             00000014                              0a/. 23/# 20/  54/T                                          
.c               00 .. 23          -- 23 20 54    --------001000110010000001010100   .bitfield: -# T                
.e               04 .. 08          -- -- -- 05    -----------------------00101----   .bitfield: ---.                
>>uninterresting 00000018 00000027 Skipped 00 00 00 10 bytes                        
more             00000028                                                  23/# 0a/. 23/# 2d/- 2d/- 2d/- 2d/- 2d/-  
more             00000030 00000008 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/- 2d/-                      

Decimal dump

You can get a decimal dump instead for an hexidecimal dump.

 hdr  -r 'header,20 :DATA,4  :c,b24  :e,x4b5 :uninterresting, x16  :more,20' -display_dec_dump 1 -display_hex_dump 0 -rul -col -o ver 
RANGE_NAME       OFFSET   CUMULATI DEC_DUMP                                                         ASCII_DUMP        
                                   0   1   2   3   4   5   6   7   8   9   a   b   c   d   e   f    0123456789abcdef  
header           00000000          035 045 045 045 045 045 045 045 045 045 045 045 045 045 045 045  #--------------- 
header           00000010 00000010 045 045 045 035                                                  ---#             
DATA             00000014                          010 035 032 084                                      .# T         
.c               00 .. 23          --- 035 032 084 value: 2302036                                   .bitfield: -# T  
.e               04 .. 08          --- --- --- 005 value: 5                                         .bitfield: ---.  
>>uninterresting 00000018 00000027 000 000 000 016 skipped: 16 bytes                                                 
more             00000028                                          035 010 035 045 045 045 045 045          #.#----- 
more             00000030 00000008 045 045 045 045 045 045 045 045 045 045 045 045                  ------------     

You can also have both dumps simulteanously. Check the display option of the hdr command.

Colors and Headers

Showing the column header and the rulers

  $>hdr -col -rul your_file

You can also insert a header anywhere, except as a first range since -col and -rul are made for that, by adding a header range in your range definition.

  $> hdr -r 'zzz,50: show_header,@,red: data,20' -o ver -col -rul some_file
  RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP        
                                     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345  
  zzz              00000000 00000000 44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  Data::HexDump::R 
  zzz              00000010 00000010 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== 
  zzz              00000020 00000020 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  =========..'Hexa 
  zzz              00000030 00000030 64 65                                            de               
  RANGE_NAME       OFFSET   CUMULATI HEX_DUMP                                         ASCII_DUMP        
                                   0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345  
  data             00000032 00000000       63 69 61 6c 20 52 61 6e 67 65 20 44 75 6d    cial Range Dum 
  data             00000040 0000000e 70 65 72 27 20 20                                per'            

Defining your own colors

Define a custom color file containing a data in the format below andd pass it as argument to option --colors

  {
  HTML =>
        {
        # custom color name
        custom => 'color:#f00; background-color:#ff0; ',
        
        # the normal ansi color
        reset => '',
        white => 'color:#aaa; ',
        black => 'color:#000; ',
        green => 'color:#0a0; ',
        ...
        },
  ANSI =>
        {
        # custom color name, it's a good idea to define them on both formats
        custom => 'bright_red on_bright_yellow; ',
        
        reset => '',
        white => 'white',
        ...
        },
  }

Colorize only the range you give color to

By default hdr will assign colors to ranges you don't give a color to.

  hdr -r 'A,10:B,10,red:C,10:D,10,green' -col -rul
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_  
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345         
00000000 63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 A, B,  
00000010 62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 B, C, D,  
00000020 63 34 30 62 64 65 31 39                          c40bde19         D,  

If you only want the ranges you gave a color to to be colorized,use the --color bw option

  hdr -r 'A,10:B,10,red:C,10:D,10,green' -col -rul -color bw
OFFSET   HEX_DUMP                                         ASCII_DUMP       RANGE_  
         0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f   0123456789012345         
00000000 63 6f 6d 6d 69 74 20 31 39 39 61 34 62 31 32 37  commit 199a4b127 A, B,  
00000010 62 32 39 66 39 31 64 32 65 36 33 66 39 35 66 38  b29f91d2e63f95f8 B, C, D,  
00000020 63 34 30 62 64 65 31 39                          c40bde19         D,  

Input

from file or stdin

  hdr -r 'magic cooookie, 5,  ...'  -rul -o ver -col my_file

or pipe your data into hdr

  cat my_file | hdr -r 'magic, 5,  ...' -rul -o ver -col

Too little data

hdr will display one or more warnings is there is not enough data for all the defined ranges.

  Warning: range 'xxx' size 32 was shortened to 8 due to maximum size limit
  Warning: More ranges to display but no more data.

Too much data

hdr only uses the part that is used by your range descriptions. The rest is ignored.

If you want to use an existing range description that contains many range definitions, you can use the --maximum_size option to limit the amount of data hdr will use.

  $>hdr -r 'A,50:B,50:C,50:D,50' --maximum_size 100

will display a warning:

  Warning: More ranges to display but no more data.

and this dump:

00000000 44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  Data::HexDump::R A,  
00000010 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== A,  
00000020 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  =========..'Hexa A,  
00000030 64 65 63 69 61 6c 20 52 61 6e 67 65 20 44 75 6d  decial Range Dum A, B,  
00000040 70 65 72 27 20 20 77 69 74 68 20 63 6f 6c 6f 72  per'  with color B,  
00000050 2c 20 62 69 74 66 69 65 6c 64 73 20 61 6e 64 20  , bitfields and  B,  
00000060 73 6b 69 70                                      skip             B,  

Bitfields

  $> hdr  -r 'header,1 :DATA,4  :c,b9  :d,b3 :e,x4b5  :the end,8 :x,b3 :data,47 :v,x90b8' -o ver

Vertical mode

header           00000000 00000000 44                                               D                
DATA             00000001 00000000    61 74 61 3a                                    ata:            
.c               00 .. 08          -- -- 01 3a    -----------------------100111010  .bitfield: --.:  
.d               00 .. 02          -- -- -- 02    -----------------------------010  .bitfield: ---.  
the end          00000005 00000000                3a 48 65 78 44 75 6d 70                :HexDump    
.x               00 .. 02          -- -- -- 00    -----------------------------000  .bitfield: ---.  
data             0000000d 00000000                                        3a 3a 52               ::R 
data             00000010 00000003 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== 
data             00000020 00000013 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  =========..'Hexa 
data             00000030 00000023 64 65 63 69 61 6c 20 52 61 6e 67 65              decial Range     
.v               90 .. 97          -- -- -- 59    01011001------------------------  .bitfield: ---Y  

Horizontal mode

By default, bitmaps are not displayed in horizontal mode as they tend to clutter the display. You can force the display of bitfields in vertical mode by seting the --display_bitfields option to true.

without --display_bitfields option set:

00000000 44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  Data::HexDump::R header, DATA, the end, data,  
00000010 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== data,  
00000020 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  =========..'Hexa data,  
00000030 64 65 63 69 61 6c 20 52 61 6e 67 65              decial Range     data,  

with --display_bitfields option set to true:

00000000          44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  Data::HexDump::R header, DATA, the end, data,  
00 .. 08 DATA     -- -- 01 3a    -----------------------100111010  .bitfield: --.:  .c               
00 .. 02 DATA     -- -- -- 02    -----------------------------010  .bitfield: ---.  .d               
00 .. 02 the end  -- -- -- 00    -----------------------------000  .bitfield: ---.  .x               
00000010          61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== data,  
00000020          3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  =========..'Hexa data,  
00000030          64 65 63 69 61 6c 20 52 61 6e 67 65              decial Range     data,  
90 .. 97 data     -- -- -- 59    01011001------------------------  .bitfield: ---Y  .v               

Not enough data for the bitfield

If you define an offset and a bit amout that sets the bitfields outside the data range, or you define a bitfield with more than 32 bits, The bitfields will be displayed as an error message.

   hdr  -r 'x,b3 :A, 1 : y,X3x2b16 :z, b33'
data             00000000 00000000 44                                               D                
DATA             00000001 00000000    61 74 61 3a                                    ata:            
.c               00 .. 49          Error: bitfield is more than 32 bits long (50)   Error: bitfield  
.d               30 .. 32          Error: 33 bits needed but only 32 bits available Error: 33 bits n 
.e               04 .. 08          -- -- -- 13    -----------------------10011----  .bitfield: ---.  
the end          00000005 00000000                3a 48 65 78 44 75 6d 70                :HexDump    

Bitfield offset

The format for a bitfield is [XInteger][xInteger]b[Integer]. You can use X for byte offset and x for bit offset. If no size is given after b, the bitfield consists of a single bit

Examples:

  • X1x1b - second bit of the second byte

  • X3x8b4 - 4 first bits of the fifth byte

  • X4b8 - fifth byte

  • X1x0b8 - second byte

  hdr  -r 'data,4 :X1b1,X1b1 :X3x8b4,X3x8b4 :X4b8,X4b8 :X1x0b8,X1x0b8 :end data, 8' -o ver 

Remember that bit zero is on the right side by default. Also notice that since we have no fifth byte an error is displayed.

data             00000000 00000000 44 61 74 61                                      Data             
.X1b1            08 .. 08          -- -- -- 00    -----------------------0--------  .bitfield: ---.  
.X3x8b4          32 .. 35          Error: 36 bits needed but only 32 bits available Error: 36 bits n 
.X4b8            32 .. 39          Error: 40 bits needed but only 32 bits available Error: 40 bits n 
.X1x0b8          08 .. 15          -- -- -- 74    ----------------01110100--------  .bitfield: ---t  
end data         00000004 00000000             3a 3a 48 65 78 44 75 6d                  ::HexDum     

With BIT_ZERO_ON_LEFT set

data             00000000 00000000 44 61 74 61                                      Data             
.X1b1            08 .. 08          -- -- -- 00    --------0-----------------------  .bitfield: ---.  
.X3x8b4          32 .. 35          Error: 36 bits needed but only 32 bits available Error: 36 bits n 
.X4b8            32 .. 39          Error: 40 bits needed but only 32 bits available Error: 40 bits n 
.X1x0b8          08 .. 15          -- -- -- 61    --------01100001----------------  .bitfield: ---a  
end data         00000004 00000000             3a 3a 48 65 78 44 75 6d                  ::HexDum     

Skip Ranges

Say you have data with data that you are not interrested in seeing in the hexdump.

  $>hdr -r 'magic, 5  :other,37  :uninterresting,45     :more, 20'
00000000 44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  Data::HexDump::R magic, other,  
00000010 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== other,  
00000020 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  =========..'Hexa other, uninterresting,  
00000030 64 65 63 69 61 6c 20 52 61 6e 67 65 20 44 75 6d  decial Range Dum uninterresting,  
00000040 70 65 72 27 20 20 77 69 74 68 20 63 6f 6c 6f 72  per'  with color uninterresting,  
00000050 2c 20 62 69 74 66 69 65 6c 64 73 20 61 6e 64 20  , bitfields and  uninterresting, more,  
00000060 73 6b 69 70 20 72 61 6e 67 65 73                 skip ranges      more,  

Change the uninterresting range definition to skip the data.

  $>hdr -r 'magic, 5  :other,37  :uninterresting,X45     :more, 20'
00000000 44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  Data::HexDump::R magic, other,  
00000010 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== other,  
00000020 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a                    =========.       other, >>uninterresting,  
00000050                      65 6c 64 73 20 61 6e 64 20         elds and  >>uninterresting, more,  
00000060 73 6b 69 70 20 72 61 6e 67 65 73                 skip ranges      more,  

In vertical mode

magic            00000000 00000000 44 61 74 61 3a                                   Data:            
other            00000005 00000000                3a 48 65 78 44 75 6d 70 3a 3a 52       :HexDump::R 
other            00000010 0000000b 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== 
other            00000020 0000001b 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a                    =========.       
>>uninterresting 0000002a 00000056 00 00 00 2d bytes skipped                                         
more             00000057 00000000                      65 6c 64 73 20 61 6e 64 20         elds and  
more             00000060 00000009 73 6b 69 70 20 72 61 6e 67 65 73                 skip ranges      

Comment

You can use comments to introduce the ranges that are going to be displayed next and thus make it easier to understand

        [
          ['data header', 5, 'blue on_cyan'],
          ['Next is a very interresting data section', #, 'bright_white on_red'],
          ['data', 20, 'blue on_bright_yellow'],
        ] ;


  $>hdr -o vertical -r 'magic, 5  :first interresting data section,#, black on_bright_cyan    :other,37  :something,45 :second interresting data section,#,black on_bright_cyan   :other,37  :something,45 :We are done, #, black on_bright_cyan  :more, 20'
magic            00000000 00000000 44 61 74 61 3a                                   Data:            
"first interresting data section" 
other            00000005 00000000                3a 48 65 78 44 75 6d 70 3a 3a 52       :HexDump::R 
other            00000010 0000000b 61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  ange.=========== 
other            00000020 0000001b 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a                    =========.       
something        0000002a 00000000                               0a 27 48 65 78 61            .'Hexa 
something        00000030 00000006 64 65 63 69 61 6c 20 52 61 6e 67 65 20 44 75 6d  decial Range Dum 
something        00000040 00000016 70 65 72 27 20 20 77 69 74 68 20 63 6f 6c 6f 72  per'  with color 
something        00000050 00000026 2c 20 62 69 74 66 69                             , bitfi          
"second interresting data section" 
other            00000057 00000000                      65 6c 64 73 20 61 6e 64 20         elds and  
other            00000060 00000009 73 6b 69 70 20 72 61 6e 67 65 73 0a 0a 0a 49 4e  skip ranges...IN 
other            00000070 00000019 53 54 41 4c 4c 41 54 49 4f 4e 0a 2d              STALLATION.-     
something        0000007c 00000000                                     2d 2d 2d 2d              ---- 
something        00000080 00000004 2d 2d 2d 2d 2d 2d 2d 0a 0a 54 6f 20 69 6e 73 74  -------..To inst 
something        00000090 00000014 61 6c 6c 20 74 68 69 73 20 6d 6f 64 75 6c 65 20  all this module  
something        000000a0 00000024 74 79 70 65 20 74 68 65 20                       type the         
"We are done" 
more             000000a9 00000000                            66 6f 6c 6c 6f 77 69           followi 
more             000000b0 00000007 6e 67 3a 0a 0a 20 20 20 70 65 72 6c 20           ng:..   perl     

Advanced Usage

First, read the module documentation.

dynamic range generation

The parser below will alternatively return a 'from 0' or 'from 1' range description depending on the first byte of the data to display

 sub my_parser 
        {
        my ($dumper, $data, $offset) = @_ ;
        
        my $first_byte = unpack ("x$offset C", $data) ;
        
        $offset < length($data)
                ?  $first_byte == ord(0)
                        ? ['from "0"', 5, 'bright_yellow']
                        : ['from "1"', 3, 'bright_green']
                : undef ;
        }       
 my $data = '01' x 20 ;
 print $hdr->dump(\&my_parser, $data) ;
00000000 30 31 30 31 30 31 30 31 30 31 30 31 30 31 30 31  0101010101010101 5:from "0", 3:from "1", 5:from "0", 3:from "1",  
00000010 30 31 30 31 30 31 30 31 30 31 30 31 30 31 30 31  0101010101010101 5:from "0", 3:from "1", 5:from "0", 3:from "1",  
00000020 30 31 30 31 30 31 30 31                          01010101         5:from "0", 3:from "1",  

IPV4 header

Below is the range description for an IPV4 header:

 use strict ;
 use warnings ;
 
 my $IPV4_header =
        [
        ['IPV4_header', '#', 'bright_white on_red'],
                [
                        ['Version, Header length,  Differentiated Services, Total Length', '#', 'blue on_cyan'],
                        ['First 32 bits', 4], 
                        ['Version', 'b4'],
                        ['Header length', 'x4b4'],
                        ['Differentiated Services', 'x8b8'],
                                # Differentiated Services (DS) 
                                # # bit 3: 0 = Normal Delay, 1 = Low Delay
                                # # bit 4: 0 = Normal Throughput, 1 = High Throughput
                                # # bit 5: 0 = Normal Reliability, 1 = High Reliability
                                # # bit 6: 0 = Normal Cost, 1 = Minimize Monetary Cost
                                # (defined by RFC 1349)
                                # # bit 7: never defined
                        ['Total length', 'x16b16'],
                ],

                [
                        ['Identification, Flags, Fragment Offset', '#', 'blue on_cyan'],
                        ['Second 32 bits', 4], 
                        ['Identification', 'b16'],
                        
                        ['reserved', 'x16b1'],
                        ["Don't Fragment", 'x17b1'],
                        ['More Fragments', 'x18b1'],
                                # bit 0: Reserved; must be zero. As an April Fools joke, proposed for use in RFC 3514 as the "Evil bit".
                                # bit 1: Don't Fragment (DF)
                                # bit 2: More Fragments (MF)

                        ['Fragment Offset', 'x19b13'],
                ],

                [
                        ['Time to Live, Protocol, Header Checksum', '#', 'blue on_cyan'],
                        ['Third 32 bits', 4], 
                        ['Time to Live', 'b8'],
                        ['Protocol', 'x8b8'],
                        ['Header Checksum', 'x16b16'],
                        [],
                ],
                
                ['Addresses and options', '#', 'blue on_cyan'],
                ['Source Address', 4],
                ['Destination Address', 4], 
                [
                sub
                        {
                        my ($self, $data, $offset, $size, $range) = @_ ;
                        
                        my ($header_length_chunk) = grep {$_->{NAME} eq 'Header length'} @{$self->{GATHERED}} ;
                        my $header_length = ord(substr($header_length_chunk->{DATA}, 0, 1)) & 0x0f  ;
                        
                        if($header_length > 5)
                                {
                                return ['Options', 4, undef ,'header length > 5, extracting options'] ;
                                }
                        else
                                {
                                # Note that we could choose to return a comment range instead
                                #~ return ['header length < 5, no options', '#', undef, undef] ;
                                
                                return undef, 'Skipping option field header length < 5';
                                }
                        }
                ],
                ['Data', '#', 'blue on_cyan'],
                ['Data', 128] # display some of the data
        ] ;

Note that the Options field is an optional field that is present when the Header length is greater than 5. For the Options field, we define subroutine which verifies the value of the Header length and selectively returns a range description or nothing.

In vertical mode

 $> hdr -r t/parse_ipv4.pl -display_ascii_dump 0 -bit_zero_on_left -o ver
"IPV4_header" 
"Version, Header length,  Differentiated Services, Total Length" 
First 32 bits    00000000 00000000 44 61 74 61                                      
.Version         00 .. 03          -- -- -- 04    0100----------------------------  
.Header length   04 .. 07          -- -- -- 04    ----0100------------------------  
.Differentiated  08 .. 15          -- -- -- 61    --------01100001----------------  
.Total length    16 .. 31          -- -- 74 61    ----------------0111010001100001  
"Identification, Flags, Fragment Offset" 
Second 32 bits   00000004 00000000             3a 3a 48 65                          
.Identification  00 .. 15          -- -- 3a 3a    0011101000111010----------------  
.reserved        16 .. 16          -- -- -- 00    ----------------0---------------  
.Don't Fragment  17 .. 17          -- -- -- 01    -----------------1--------------  
.More Fragments  18 .. 18          -- -- -- 00    ------------------0-------------  
.Fragment Offset 19 .. 31          -- -- 08 65    -------------------0100001100101  
"Time to Live, Protocol, Header Checksum" 
Third 32 bits    00000008 00000000                         78 44 75 6d              
.Time to Live    00 .. 07          -- -- -- 78    01111000------------------------  
.Protocol        08 .. 15          -- -- -- 44    --------01000100----------------  
.Header Checksum 16 .. 31          -- -- 75 6d    ----------------0111010101101101  
"Addresses and options" 
Source Address   0000000c 00000000                                     70 3a 3a 52  
Destination Addr 00000010 00000000 61 6e 67 65                                      
"Data" 
Data             00000014 00000000             0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  
Data             00000020 0000000c 3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  
Data             00000030 0000001c 64 65 63 69 61 6c 20 52 61 6e 67 65 20 44 75 6d  
Data             00000040 0000002c 70 65 72 27 20 20 77 69 74 68 20 63 6f 6c 6f 72  
Data             00000050 0000003c 2c 20 62 69 74 66 69 65 6c 64 73 20 61 6e 64 20  
Data             00000060 0000004c 73 6b 69 70 20 72 61 6e 67 65 73 0a 0a 0a 49 4e  
Data             00000070 0000005c 53 54 41 4c 4c 41 54 49 4f 4e 0a 2d 2d 2d 2d 2d  
Data             00000080 0000006c 2d 2d 2d 2d 2d 2d 2d 0a 0a 54 6f 20 69 6e 73 74  
Data             00000090 0000007c 61 6c 6c 20                                      

In horizontal mode

 $> hdr -r t/parse_ipv4.pl -display_ascii_dump 0 -bit_zero_on_left -display_bitfields 1 -display_comment_range 0
00000000          44 61 74 61 3a 3a 48 65 78 44 75 6d 70 3a 3a 52  First 32 bits, Second 32 bits, Third 32 bits, Source Address,  
00 .. 03 First 32 -- -- -- 04    0100----------------------------  .Version         
04 .. 07 First 32 -- -- -- 04    ----0100------------------------  .Header length   
08 .. 15 First 32 -- -- -- 61    --------01100001----------------  .Differentiated  
16 .. 31 First 32 -- -- 74 61    ----------------0111010001100001  .Total length    
00 .. 15 Second 3 -- -- 3a 3a    0011101000111010----------------  .Identification  
16 .. 16 Second 3 -- -- -- 00    ----------------0---------------  .reserved        
17 .. 17 Second 3 -- -- -- 01    -----------------1--------------  .Don't Fragment  
18 .. 18 Second 3 -- -- -- 00    ------------------0-------------  .More Fragments  
19 .. 31 Second 3 -- -- 08 65    -------------------0100001100101  .Fragment Offset 
00 .. 07 Third 32 -- -- -- 78    01111000------------------------  .Time to Live    
08 .. 15 Third 32 -- -- -- 44    --------01000100----------------  .Protocol        
16 .. 31 Third 32 -- -- 75 6d    ----------------0111010101101101  .Header Checksum 
00000010          61 6e 67 65 0a 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d 3d  Destination Addr, Data,  
00000020          3d 3d 3d 3d 3d 3d 3d 3d 3d 0a 0a 27 48 65 78 61  Data,  
00000030          64 65 63 69 61 6c 20 52 61 6e 67 65 20 44 75 6d  Data,  
00000040          70 65 72 27 20 20 77 69 74 68 20 63 6f 6c 6f 72  Data,  
00000050          2c 20 62 69 74 66 69 65 6c 64 73 20 61 6e 64 20  Data,  
00000060          73 6b 69 70 20 72 61 6e 67 65 73 0a 0a 0a 49 4e  Data,  
00000070          53 54 41 4c 4c 41 54 49 4f 4e 0a 2d 2d 2d 2d 2d  Data,  
00000080          2d 2d 2d 2d 2d 2d 2d 0a 0a 54 6f 20 69 6e 73 74  Data,  
00000090          61 6c 6c 20                                      Data,  

Comments that help

Remember that you can, while generating your ranges, assign the comment field

  ['range_name', 4, 'red', 'comment of great help']
  

You can also pass a sub reference in place of the comment. The sub will be called just before the range is displayed. It is passed information about the data to be displayed (data and offset). This allows you to dynamically generate a comment string.

        ['size', 1, undef, 
                sub
                        {
                        my ($self, $data, $offset, $size, $range) = @_ ;
                        
                        my $size = ord(substr($data, $offset, 1))  ;
                        
                        for($size)
                                {
                                $_ == 1 and do {return 'size is: S' } ;
                                $_ == 2 and do {return  'size is: M'} ;
                                $_ == 3 and do {return 'size is: L'} ;
                                $_ == 4 and do {return  'size is: XL'} ;
                                
                                return 'Error in size range!' ;
                                }
                        }
        ],
        

Which would generate :

 ...
 size             00000022 00000000 32                                               2                Error in size range!
 size             00000023 00000000 01                                               .                size is: S
 ...

Let the computer do the heavy lifting

hdr may be a nice tool to run from the command line but you can also use it in shell scripts to generate reports. I generally write pod so I let hdr generate my pod snippets with embeded html hexdumps.

 printf "=head1 MY REPORT\n\n" > report.pod
 printf "=head2 The data\n\n" >> report.pod
 
 hdr -r 'A,55:B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -format HTML -display_command_line -start_tag '=begin html' -end_tag '=end html' >> report.pod
 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -format HTML -display_command_line -start_tag '=begin html' -end_tag '=end html' >> report.pod
 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -offset_start 55 -format HTML -display_command_line -start_tag '=begin html' -end_tag '=end html' >> report.pod

 echo "=head2 Nicely synchronized\n\n" >> report.pod 
 hdr -r 'B,5:bf,x3b5:C,18:D,10'  Changes -col -rul -offset 55 -offset_start 55 -format HTML -display_command_line -start_color bright_yellow -start_tag '=begin html' -end_tag '=end html' >> report.pod
 
 pod2html report.pod > report.html

Bash completion script

hdr can generate a bash completion script

 $> hdr --generate_bash_completion > hdr_completion
 $> source hdr_completion

Helping out

You contribute when you:

  • report errors

  • send some examples of your data/output

  • send parsers or other scripts in which you use hdr or Data::HexDump::Range

  • send new requirement, patches, documentation enhancements

  • pass the word around so more people use hdr and I get more feedback

  • plant a tree, save a cat/dog or both, bike instead for driving, spread some love around you, :)

AUTHOR

        Nadim ibn hamouda el Khemir
        CPAN ID: NKH
        mailto: nadim@cpan.org

COPYRIGHT & LICENSE

Copyright 2010 Nadim Khemir.

This program is free software; you can redistribute it and/or modify it under the terms of either:

  • the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or

  • the Artistic License version 2.0.

SUPPORT

You can also look for information at:

SEE ALSO

hexd from libma http://www.ioplex.com/~miallen/libmba/