The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Convert-BER-XS-0.7
River stage zero No dependents
++
/ Convert::BER::XS

NAME

Convert::BER::XS - very low level BER en-/decoding

SYNOPSIS

 use Convert::BER::XS ':all';

 my $ber = ber_decode $buf
    or die "unable to decode SNMP message";

 # The above results in a data structure consisting of
 # (class, tag, # constructed, data)
 # tuples. Below is such a message, SNMPv1 trap
 # with a Cisco mac change notification.
 # Did you know that Cisco is in the news almost
 # every week because # of some backdoor password
 # or other extremely stupid security bug?

 [ ASN_UNIVERSAL, ASN_SEQUENCE, 1,
   [
      [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 0 ], # snmp version 1
      [ ASN_UNIVERSAL, 4, 0, "public" ], # community
      [ ASN_CONTEXT, 4, 1, # CHOICE, constructed - trap PDU
         [
            [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.2" ], # enterprise oid
            [ ASN_APPLICATION, 0, 0, "\x0a\x00\x00\x01" ], # SNMP IpAddress, 10.0.0.1
            [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 6 ], # generic trap
            [ ASN_UNIVERSAL, ASN_INTEGER32, 0, 1 ], # specific trap
            [ ASN_APPLICATION, ASN_TIMETICKS, 0, 1817903850 ], # SNMP TimeTicks
            [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # the varbindlist
               [
                  [ ASN_UNIVERSAL, ASN_SEQUENCE, 1, # a single varbind, "key value" pair
                     [
                        [ ASN_UNIVERSAL, ASN_OBJECT_IDENTIFIER, 0, "1.3.6.1.4.1.9.9.215.1.1.8.1.2.1" ],
                        [ ASN_UNIVERSAL, ASN_OCTET_STRING, 0, "...data..." # the value
                        ]
                     ]
                  ],
                  ...

 # let's decode it a bit with some helper functions

 my $msg = ber_is_seq $ber
    or die "SNMP message does not start with a sequence";

 ber_is $msg->[0], ASN_UNIVERSAL, ASN_INTEGER32, 0
    or die "SNMP message does not start with snmp version\n";

 # message is SNMP v1 or v2c?
 if ($msg->[0][BER_DATA] == 0 || $msg->[0][BER_DATA] == 1) {

    # message is v1 trap?
    if (ber_is $msg->[2], ASN_CONTEXT, 4, 1) {
       my $trap = $msg->[2][BER_DATA];

       # check whether trap is a cisco mac notification mac changed message
       if (
          (ber_is_oid $trap->[0], "1.3.6.1.4.1.9.9.215.2") # cmnInterfaceObjects
          and (ber_is_i32 $trap->[2], 6)
          and (ber_is_i32 $trap->[3], 1) # mac changed msg
       ) {
          ... and so on

 # finally, let's encode it again and hope it results in the same bit pattern

 my $buf = ber_encode $ber;

DESCRIPTION

WARNING: Before release 1.0, the API is not considered stable in any way.

This module implements a very low level BER/DER en-/decoder.

If is tuned for low memory and high speed, while still maintaining some level of user-friendlyness.

Currently, not much is documented, as this is an initial release to reserve CPAN namespace, stay tuned for a few days.

ASN.1/BER/DER/... BASICS

ASN.1 is a strange language that can be sed to describe protocols and data structures. It supports various mappings to JSON, XML, but most importantly, to a various binary encodings such as BER, that is the topic of this module, and is used in SNMP or LDAP for example.

While ASN.1 defines a schema that is useful to interpret encoded data, the BER encoding is actually somewhat self-describing: you might not know whether something is a string or a number or a sequence or something else, but you can nevertheless decode the overall structure, even if you end up with just a binary blob for the actual value.

This works because BER values are tagged with a type and a namespace, and also have a flag that says whther a value consists of subvalues (is "constructed") or not (is "primitive").

Tags are simple integers, and ASN.1 defines a somewhat weird assortment of those - for example, you have 32 bit signed integers and 16(!) different string types, but there is no unsigned32 type for example. Different applications work around this in different ways, for example, SNMP defines application-specific Gauge32, Counter32 and Unsigned32, which are mapped to two different tags: you can distinguish between Counter32 and the others, but not between Gause32 and Unsigned32, without the ASN.1 schema.

Ugh.

DECODED BER REPRESENTATION

This module represents every BER value as a 4-element tuple (actually an array-reference):

   [CLASS, TAG, CONSTRUCTED, DATA]

To avoid non-descriptive hardcoded array index numbers, this module defines symbolic constants to access these members: BER_CLASS, BER_TAG, BER_CONSTRUCTED and BER_DATA.

Also, the first three members are integers with a little caveat: for performance reasons, these are readonly and shared, so you must not modify them (increment, assign to them etc.) in any way. You may modify the DATA member, and you may re-assign the array itself, e.g.:

   $ber = ber_decode $binbuf;

   # the following is NOT legal:
   $ber->[BER_CLASS] = ASN_PRIVATE; # ERROR, CLASS/TAG/CONSTRUCTED are READ ONLY(!)

   # but all of the following are fine:
   $ber->[BER_DATA] = "string";
   $ber->[BER_DATA] = [ASN_UNIVERSAL, ASN_INTEGER32, 0, 123];
   @$ber = (ASN_APPLICATION, SNMP_TIMETICKS, 0, 1000);

CLASS is something like a namespace for TAGs - there is the ASN_UNIVERSAL namespace which defines tags common to all ASN.1 implementations, the ASN_APPLICATION namespace which defines tags for specific applications (for example, the SNMP Unsigned32 type is in this namespace), a special-purpose context namespace (ASN_CONTEXT, used e.g. for CHOICE) and a private namespace (ASN_PRIVATE).

The meaning of the TAG depends on the namespace, and defines a (partial) interpretation of the data value. For example, right now, SNMP application namespace knowledge ix hardcoded into this module, so it knows that SNMP Unsigned32 values need to be decoded into actual perl integers.

The most common tags in the ASN_UNIVERSAL namespace are ASN_INTEGER32, ASN_BIT_STRING, ASN_NULL, ASN_OCTET_STRING, ASN_OBJECT_IDENTIFIER, ASN_SEQUENCE, ASN_SET and ASN_IA5_STRING.

The most common tags in SNMP's ASN_APPLICATION namespace are SNMP_IPADDRESS, SNMP_COUNTER32, SNMP_UNSIGNED32, SNMP_TIMETICKS, SNMP_OPAQUE and SNMP_COUNTER64.

The CONSTRUCTED flag is really just a boolean - if it is false, the the value is "primitive" and contains no subvalues, kind of like a non-reference perl scalar. IF it is true, then the value is "constructed" which just means it contains a list of subvalues which this module will en-/decode as BER tuples themselves.

The DATA value is either a reference to an array of further tuples (if the value is CONSTRUCTED), some decoded representation of the value, if this module knows how to decode it (e.g. for the integer types above) or a binary string with the raw octets if this module doesn't know how to interpret the namespace/tag.

Thus, you can always decode a BER data structure and at worst you get a string in place of some nice decoded value.

See the SYNOPSIS for an example of such an encoded tuple representation.

DECODING AND ENCODING

$tuple = ber_decoded $bindata

Decodes binary BER data in $bindata and returns the resulting BER tuple. Croaks on any decoding error, so the returned $tuple is always valid.

$bindata = ber_encode $tuple

Encodes the BER tuple into a BER/DER data structure.

HELPER FUNCTIONS

Working with a 4-tuple for every value can be annoying. Or, rather, is annoying. To reduce this a bit, this module defines a number of helper functions, both to match BER tuples and to conmstruct BER tuples:

MATCH HELPERS

Thse functions accept a BER tuple as first argument and either paertially or fully match it. They often come in two forms, one which exactly matches a value, and one which only matches the type and returns the value.

They do check whether valid tuples are passed in and croak otherwise. As a ease-of-use exception, they usually also accept undef instead of a tuple reference. in which case they silently fail to match.

$bool = ber_is $tuple, $class, $tag, $constructed, $data

This takes a BER $tuple and matches its elements agains the privded values, all of which are optional - values that are either missing or undef will be ignored, the others will be matched exactly (e.g. as if you used == or eq (for $data)).

Some examples:

   ber_is $tuple, ASN_UNIVERSAL, ASN_SEQUENCE, 1
      orf die "tuple is not an ASN SEQUENCE";

   ber_is $tuple, ASN_UNIVERSAL, ASN_NULL
      or die "tuple is not an ASN NULL value";

   ber_is $tuple, ASN_UNIVERSAL, ASN_INTEGER32, 0, 50
      or die "BER integer must be 50";
$seq = ber_is_seq $tuple

Returns the sequence members (the array of subvalues) if the $tuple is an ASN SEQUENCE, i.e. the BER_DATA member. If the $tuple is not a sequence it returns undef. For example, SNMP version 1/2c/3 packets all consist of an outer SEQUENCE value:

   my $ber = ber_decode $snmp_data;

   my $snmp = ber_is_seq $ber
      or die "SNMP packet invalid: does not start with SEQUENCE";

   # now we know $snmp is a sequence, so decode the SNMP version

   my $version = ber_is_i32 $snmp->[0]
      or die "SNMP packet invalid: does not start with version number";
$bool = ber_is_i32 $tuple, $i32

Returns a true value if the $tuple represents an ASN INTEGER32 with the value $i32.

$i32 = ber_is_i32 $tuple

Returns true (and extracts the integer value) if the $tuple is an ASN INTEGER32. For 0, this function returns a special value that is 0 but true.

$bool = ber_is_oid $tuple, $oid_string

Returns true if the $tuple represents an ASN_OBJECT_IDENTIFIER that exactly matches $oid_string. Example:

   ber_is_oid $tuple, "1.3.6.1.4"
      or die "oid must be 1.3.6.1.4";
$oid = ber_is_oid $tuple

Returns true (and extracts the OID string) if the $tuple is an ASN OBJECT IDENTIFIER. Otherwise, it returns undef.

CONSTRUCTION HELPERS

$tuple = ber_i32 $value

Constructs a new ASN_INTEGER32 tuple.

RELATIONSHIP TO Convert::BER and Convert::ASN1

This module is not the XS version of Convert::BER, but a different take at doing the same thing. I imagine this module would be a good base for speeding up either of these, or write a similar module, or write your own LDAP or SNMP module for example.

PROFILES

While any BER data can be correctly encoded and decoded out of the box, it can be inconvenient to have to manually decode some values into a "better" format: for instance, SNMP TimeTicks values are decoded into the raw octet strings of their BER representation, which is quite hard to decode. With profiles, you can change which class/tag combinations map to which decoder function inside ber_decode (and of course also which encoder functions are used in ber_encode).

This works by mapping specific class/tag combinations to an internal "ber type".

The default profile supports the standard ASN.1 types, but no application-specific ones. This means that class/tag combinations not in the base set of ASN.1 are decoded into their raw octet strings.

Convert::BER::XS defines two profile variables you cna use out of the box:

$Convert::BER::XS::DEFAULT_PROFILE

This is the default profile, i.e. the profile that is used when no profile is specified for de-/encoding.

You cna modify it, but remember that this modifies the defaults for all callers that rely on the defauit profile.

$Convert::BER::XS::SNMP_PROFILE

A profile with mappings for SNMP-specific application tags added. This is useful when de-/encoding SNMP data.

Example: $ber = ber_decode $data, $Convert::BER::XS::SNMP_PROFILE;

The Convert::BER::XS::Profile class

$profile = new Convert::BER::XS::Profile

Create a new profile. The profile will be identical to the default profile.

$profile->set ($class, $tag, $type)

Sets the mapping for the given $class/$tag combination to $type, which must be one of the BER_TYPE_* constants.

Note that currently, the mapping is stored in a flat array, so large values of $tag will consume large amounts of memory.

Example: $profile = new Convert::BER::XS::Profile; $profile->set (ASN_APPLICATION, SNMP_COUNTER32, BER_TYPE_INT); $ber = ber_decode $data, $profile;

$type = $profile->get ($class, $tag)

Returns the BER type mapped to the given $class/$tag combination.

BER TYPES

This lists the predefined BER types - you can map any CLASS/TAG combination to any BER_TYPE_*.

BER_TYPE_BYTES

The raw octets of the value. This is the default type for unknown tags and de-/encodes the value as if it were an octet string, i.e. by copying the raw bytes.

BER_TYPE_UTF8

Like BER_TYPE_BYTES, but decodes the value as if it were a UTF-8 string (without validation!) and encodes a perl unicode string into a UTF-8 BER string.

BER_TYPE_UCS2

Similar to BER_TYPE_UTF8, but treats the BER value as UCS-2 encoded string.

BER_TYPE_UCS4

Similar to BER_TYPE_UTF8, but treats the BER value as UCS-4 encoded string.

BER_TYPE_INT

Encodes and decodes a BER integer value to a perl integer scalar. This should correctly handle 64 bit signed and unsigned values.

BER_TYPE_OID

Encodes and decodes an OBJECT IDENTIFIER into dotted form without leading dot, e.g. 1.3.6.1.213.

BER_TYPE_RELOID

Same as BER_TYPE_OID but uses relative OID encoding: ASN.1 has this hack of encoding the first two OID components into a single integer in a weird attempt to save an insignificant amount of space in an otherwise wasteful encoding, and relative OIDs are basically OIDs without this hack. The practical difference is that the second component of an OID can only have the values 1..40, while relative OIDs do not have this restriction.

BER_TYPE_NULL

Decodes an ASN_NULL value into undef, and always encodes a ASN_NULL type, regardless of the perl value.

BER_TYPE_BOOL

Decodes an ASN_BOOLEAN value into 0 or 1, and encodes a perl boolean value into an ASN_BOOLEAN.

BER_TYPE_REAL

Decodes/encodes a BER real value. NOT IMPLEMENTED.

BER_TYPE_IPADDRESS

Decodes/encodes a four byte string into an IOv4 dotted-quad address string in perl. Given ther obsolete nature of this type, this is a low-effort implementation that simply uses sprintf and sscanf-style conversion, so it won't handle all string forms supported by inet_aton.

BER_TYPE_CROAK

Always croaks when encountered during encoding or decoding - the default behaviour when encountering an unknown type is to treat it as BER_TYPE_BYTES. When you don't want that but instead prefer a hard error for some types, then CyBER_TYPE_CROAK> is for you.

LIMITATIONS

This module can only en-/decode 64 bit signed and unsigned integers, and only when your perl supports those.

OBJECT IDENTIFIEERS cannot have unlimited length, although the limit is much larger than e.g. the one imposed by SNMP or other protocols.

REAL values are not supported and will croak.

This module has undergone little to no testing so far.

AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://software.schmorp.de/pkg/Convert-BER-XS