XS::Framework::Manual::Typemap - XS::Framework C++ typemap API reference
Typemap is a concept, i.e. C++ struct with certain expectations, described below.
Typemaps are reponsible for transferring objects between perl scripts and C++ code. In the most general case typemap should be a full-specialization of C++ policy class, i.e. stateless class with group of static methods in the xs namespace. The specialization should be for the target C++ class, which objects should be available in perl scripts, e.g.
xs
namespace xs { struct Typemap<MyClass> { static [inline] MyClass in(SV* arg) { ... } static [inline] Sv out(const MyClass& var, const Sv& proto = {}) { ... } static [inline] void destroy (const MyClass&, SV*) { ... } } } // end of namespace xs
First af all for MyClass value or pointer semantics should be defined. Objects with value semantics are allocated on C++ stack and their destructor is automatically called when object is goes out of scope; it is suitable for short-living transient objects like integer, string etc., and for std::shared_ptr / panda::iptr. A typemap specialization should start as
MyClass
std::shared_ptr
panda::iptr
struct Typemap<MyClass> { ...
For objects with pointer semantics the only possible option is to return pointer to MyClass and then later, may be, clean it up in destroy method (see below). A typemap specialization should start as:
destroy
struct Typemap<MyClass*> { ...
As the objects with value semantics is usually applicable for light-weight primitive objects structures with negligible construction and destruction costs (including std::shared_ptr), the classes with pointer semantics is the most general case.
static [inline] MyClass in(SV* arg) { ... static [inline] MyClass* in(SV* arg) { ...
in method is responsible for returning C++ object from perl SV* arg. It's intention is to prepare C++ object to work with (external) C++ API, without being encumbered with Perl details. For object with value semantics it usually return the (transient) object copy, e.g. configuration object created from perl Hashes; but in the case of pointer semantics the in method should extract previously constructed object from perl SV*, i.e. return pointer MyClass*. Beware to do not construct object on heap and return pointer to it in the in method, as this leads to memory leak because the XS::framework will not delete the pointer.
in
delete
static [inline] Sv out(const MyClass& var, const Sv& proto = {}) { ... static [inline] Sv out(const MyClass* var, const Sv& proto = {}) { ...
The out method is responsible for returning Perl wrapper from C++ object to let it be useable from perl scripts. The wrapper should be returned in form of Sv (see XS::Framework::Sv). Usually it should return reference to blessed perl object (newRV), however it might return any other suitable perl SV*, e.g. number, string, or reference to Array/Hash. The proto argument serves as a hint without any restrictions on it's usage or interpretation. In future specializations (see TypemapObject below) it is used as information which final class an object MyClass should be blessed into, i.e. MyPackage::MyClass.
out
proto
TypemapObject
MyPackage::MyClass
static [inline] void destroy (const MyClass&, SV*) { }
The <destroy> method is responsible for cleaning up C++ object when Perl's SV* wrapper (returned in out method), is about to be destroyed in Perl: it is not always enough to have MyClass destructor, as destoy method is called the link between C++ and perl is going to be deleted. For transient objects like string, int, date, etc. which do not have some permanent connection with Perl's SV* it is OK to let the method be empty. For pointer-semantics objects it depends: if your XS-code is responsible for object instantiation, then in the sake of resource leaks prevension it should delete it, i.e.
destoy
static [inline] void destroy (const MyClass* obj, SV*) { delete obj; }
If MyClass object was somehow customly constructed and expect it to be customly destructed, that can be done as
static [inline] void destroy (const MyClass* obj, SV*) { obj->dec_refcounter(); // or GlobalPool::release(obj) }
The perl scalar SV* might be used as place to store additional information, assosicated with the concrete C++ object instance, e.g. the used memory pool, then the memory pool can be extracted from SV* and object can be deleted with it.
static [inline] void destroy (const MyClass* obj, SV* sv) { MyPool* pool = ...; // somehow get pool pointer from SV* pool->release(obj); }
There might be other uses cases. The destroy method is unconditionally inserted for XS-classes in DESTROY at pre-processing step by Parse::XS if there is input typemap. So, the method destroy have to be defined, but it can be empty. The destroy method might be invoked a few times in case of inheritance of XS-adapters, this issues is already solved in TypemapObject.
DESTROY
Let's summarize the for pointer semantics case: in - unwraps / extracs C++ object, out wraps/packs C++ object in perl scalar, destroy deletes Perl and C++ association. A reader might ask the question: where there my C++ object is created? The answer is: the object instantiation left out of typemap scope by design. C++ object instantiation should be done in user's XS-code (aka XS-adapter).
Hovewer, writing typemap specialization for MyClass from scratch is a bit cumbersome because devepoler still have to implement that wrap/unwrap code in using perl API. There are some ready-made and tested patterns, which will help you greatly with that. Go on reading the following sections.
The TypemapBase is basic implementation of concept TypeMap. It ships with empty destroy method; it is expected that client typemap will inherit from TypemapBase and provide required methods:
TypemapBase
template<> struct Typemap<MyClass*>: public TypemapBase<MyClass*> { static inline MyClass* in (SV*) { ... } static inline Sv out (const MyClass* var, const Sv& proto = {}) { ... } };
struct Typemap<MyFinalClass>: TypemapObject<MyBaseClass, MyFinalClass, LifetimePolicy, StoragePolicy, CastingPolicy>;
TypemapObject is helper class in creating typemaps, when there is a blessed Perl object and corresponding C++ object, i.e. one object to one object relationship. In the best case all behaviour can be specified via policies without need of additional code writing.
It is expected that custom typemap will inherit TypemapObject, i.e. in common cases it should be enough to have something like:
template <> struct Typemap<geos::geom::Triangle*>: TypemapObject<geos::geom::Triangle*, geos::geom::Triangle*, ObjectTypePtr, ObjectStorageMG, StaticCast>{ static panda::string_view package() {return "Geo::Geos::Triangle"; } };
CastingPolicy defines how void* is casted to MyFinalClass. In most of cases the StaticCast should be used as it has zero-runtime costs; in rare cases, i.e. when MyFinalClass participates in virtual inheritance, the DynamicCast should be used. Under the hood the DynamicCast uses panda::dyn_cast which is faster implementation than standard dynamic_cast.
CastingPolicy
void*
MyFinalClass
StaticCast
DynamicCast
panda::dyn_cast
dynamic_cast
StoragePolicy defines how C++ object pointers can be stored in Perl SVs. It is available both for value-semantics object and pointer-semantic objects, however for value-semantics it might be a little bit tricky to do. Shipped implementations: ObjectStorageIV, ObjectStorageMG, ObjectStorageMGBackref
StoragePolicy
ObjectStorageIV
ObjectStorageMG
ObjectStorageMGBackref
LifetimePolicy defines how C++ defines convertion rules (between void* and MyClass), defines deletion (see destroy method in generic typemap), upcast (from DerivedClass to BasicClass) and downcast (from BasicClass to DerivedClass) policies. Shipped implementations: ObjectTypePtr, ObjectTypeForeignPtr, ObjectTypeRefcntPtr, ObjectTypeSharedPtr.
LifetimePolicy
ObjectTypePtr
ObjectTypeForeignPtr
ObjectTypeRefcntPtr
ObjectTypeSharedPtr
MyBaseClass and MyFinalClass are needed to define the most specific and the most generic classes handled via typemap. If no class hierarchy have to be managed (i.e. in Perl only <MyFinalClass> should be visible), then MyBaseClass can be the same as MyFinalClass.
MyBaseClass
static [inline] MyClass in(SV* arg) { ... } static [inline] Sv out(const MyClass& var, const Sv& proto = {}) { ... } static [inline] Sv create (const MyClass& var, const Sv& proto = Sv()) { ... } static [inline] void destroy (const MyClass&, SV*) { ... } static [inline] void dispose (const MyClass&, SV*) { ... } static [inline] MyClass dup (const MyClass& obj) { return obj; }
in(SV* arg) does the same as in typemap, i.e. returns C++ from Perl SV*. It is implemented as: 1) delegate to StoragePolicy to get the void*, and then to LifetimePolicy to convert MyBasicType* and then upcast it to MyFinalType*.
in(SV* arg)
MyBasicType*
MyFinalType*
out(const TYPE& var, const Sv& proto = Sv()) does the same as in typemap, i.e. returns Perls SV* from C++ object. In implementation it asks StoragePolicy to wrap/pack MyClass object in Perl SV* and ends up in create method below for classes without BackRefs support.
out(const TYPE& var, const Sv& proto = Sv())
create
The create method is similar to out method described above: it is responsible for creating Perl wrapper from C++ object. That distinguish is irrelevant unless MyClass is supposed to support BackRefs; in the last case, i.e. if MyClass is supports BackRefs, if it is possible out returns previously created and cached SV* wrapper, otherwise it calls create method. It is actually a bit more complex, see out behaviour below.
destroy(const TYPE& var, const Sv& proto = Sv()) - does the same as in typemap, i.e. cleans up C++ object when Perls SV* is about to be deleted. It is implemented in safe manner, i.e. real object deletion (via dispose if StoragePolicy) is executed only once, despite the fact that it migth invoked many times from DESTROY method in XS-adapter due to inheritance. If the class is auto-disposable (defined in StoragePolicy), it is actually destroyed via dispose method. In other words, after the method invokation C++ object and Perls SV* might be destroyed.
destroy(const TYPE& var, const Sv& proto = Sv())
dispose
dispose(const TYPE& var, SV* arg) - does the same as destroy method in typemap; i.e. unconditionally cleans up C++ object and Perl SV*. The actual job is delegated to LifetimePolicy::destroy
dispose(const TYPE& var, SV* arg)
LifetimePolicy::destroy
package() it is expected to be overriden in user-defined typemap. It should return the string for perl class; it is invoked only when Sv hint argument is not supplied in out or create methods. Unless overriden by user the package() method will throw runtime exception.
package()
Sv hint
dup(const MyClass& obj is called when perl starts another thread, otherwise it is never invoked. It should return MyClass* object pointer, which will be attached to SV* duplicate in the new perl thread. The default behaviour is retain policy, i.e. it returns exatcly the same object as in main thread. This implies siginificant restriction on MyClass: it should be thread-safe and ref-counted; the restrictions cannot be checked at compile time, and if the conditions aren't met Perl interpreter will crash at runtime. Another possible policy is clone, i.e. underlying C++ object should return copy of itself, and that copy will be attached to SV*. The last possible policy is skip, i.e. when undef will be attached to cloned SV*; to implement it <CLONE_SKIP> should be defined in XS-adapter and it should return true.
dup(const MyClass& obj
undef
true
ObjectStorageIV ObjectStorageMG ObjectStorageMGBackref
XS::Framework ships three policies, defining how C++ object pointers can be stored in Perl SVs. They are applicable for both for value semantics and pointer semantics, hovewer for value-semantics it might be a little bit tricky to do, i.e. value have to be stored over pointer.
Storage policy concept has the following methods. They are needed only if custom (non-shipped with XS::Framework) policy is supplied.
template <class TYPEMAP, class TYPE> struct ObjectStorage { static const bool auto_disposable = false; static inline void* get (SV* arg) { ... } static inline void set (SV* arg, void* ptr) { ... } static Sv out (const TYPE& var, const Sv& proto) { ... } };
The auto_disposable property defines, whether the destroy method of TypemapObject should be forwarded to LifetimePolicy::destroy. If underlying perl SV* is able to do self-cleaning and invoke LifetimePolicy::destroy, then this property should be set to true. Otherwise, there is a need of assitance to do proper cleaning custom (i.e. ObjectStorageIV), the property should be set to false.
auto_disposable
false
The get and set methods are responsible for storing/retriving underlying object pointer (downcasted to void*) in/from Perl SV*.
get
set
out(const TYPE& var, const Sv& proto = Sv()) does the same as in typemap, i.e. returns Perls SV* from C++ object. If there is no BackRefs support, then it should forward call to TYPEMAP::create, otherwise, if possible, it should extract Backref and return it. Backrefs support is non-trivial topic, see ObjectStorageMGBackref source code to get the ideas.
TYPEMAP::create
ObjectStorageIV relies on the equality sizeof(MyClass*) == sizeof(IV). So, it stores pointer as integer in Perls SV*. This is the most frequently used approach in CPAN, hovewer, in general it is not the best in terms on performance.
sizeof(MyClass*) == sizeof(IV)
This is non-autodisposable policy, i.e. you have to insert DESTROY method in XS-adapter to help it to be forwarded to LifetimePolicy::destoy, otherwise, if there was memory allocations for C++ object, there will be memory leak, as there is no other way to delete underlying C++ object. The actual object deletion is performed in LifetimePolicy::destroy.
LifetimePolicy::destoy
As the XS-adapter DESTROY method conforms to general perl rules, it is executed in eval(), which leads to significant slow down on object deletion. Other storage policies do not suffer from that.
eval()
There is another restriction of this policy: inheritance. As there is no more free space in underlying perl SV*, the descedant Perl classes can only add methods, but no data, i.e. to store datat the SV* should be upgraded to HashRef or ArrayRef, but that's not possible. The only possible way to overcome that is use inheritance in C++. Due to space restriction there might be issues with multiple inheritance, as there is need to store additional pointer. Other storage policies do not suffer from constraint.
This storage policy has smallest memory footprint and no additional memory allocations, i.e. Perl object creation is the fastest.
ObjectStorageMG uses Perl Magic to store pointer. It have to allocate additional memory for Perl Magic, hence, object construction is a bit slower; also that Magic occupies some memory, in other words it has some non-zero costs to use that storage policy.
The SV* itself remains free and it can be used in Child classes defined in Perl, i.e.
package MyPerlClass; use base qw/MyCPPClass/; sub new { my ($class, $data) = @_; # construct C++ object and XS-wrapper my $obj = SUPER::new($class, $data); # extend/upgrade UNDEF to Hash. XS::Framework::obj2hv($obj); $obj->{extended_field} = $data->{extended_field}; return $obj; }
This is autodisposable storage policy, i.e. custom destruction actions are guaranteed to be invoked via LifetimePolicy::destroy. That's why it is NOT recommended to have DESTOY method in XS-adapter, as it has significant performance costs due to eval context execution.
DESTOY
eval
Basically it is the same as <ObjectStorageMG> with the abilitity to cache BackRef and return it on demand via out method by the cost of slightly increased memory footprint in perl descendant classes and slightly slower object construction.
Under certain conditions, i.e. when perl object of descendant perl class has been created in perl and frequently accessed from other XS-adapter, the storage out method returns cached object, which is much faster then allocating and initializing perl wrapper.
[*1] It allows to use C++ inheritance, of course, and even perl single inheritance, but as it gives not hash, you cannot store additional properties in Perl inheritance, which makes it almost useless.
[*2] For ObjectStorageIV you have to write your own custom destroy method, which is executed in Perl's eval closure, which is quite slow; for ObjectStorageMG* you can (but we do not recommend) write destroy method, which will be the same speed slow too, instead the destroy method can be written in liftetime object policy, which is executed in very fast way as usual C++ object destructror.
ObjectStorageMG*
[*3] It heavily depends on property access pattern in your program: when (a), the property XS-object or descendent XS-object class (i.e. with perl implementation) has been created from Perl, and (b) is is accessed frequently as property of some other XS-container, then for ObjectStorageMGBackref the accessor gets significant boost, because it always returns cashed object instead of allocating Perl-wrapper for it on each invocation.
ObjectTypePtr ObjectTypeForeignPtr ObjectTypeRefcntPtr ObjectTypeSharedPtr
LifetimePolicy defines how C++ defines convertion, deletion, upcasting and downcasting rules. As there is no need to write custom LifetimePolicy the concept is not described here. Let us know if we are wrong here.
LifetimePolicy is orthogonal to StoragePolicy, i.e. they can be used in any combination independently.
It assumes that XS-adapter creates object pointer MyClass* on heap and trasfers ownership to Perl. The policy will invoke delete on the object when it's SV* is about to be destroyed.
MyClass*
It assumes that XS-adapter returns object pointer MyClass* without transferring ownership to Perl. The deletion policy is empty. It might be useful when singleton object is returned from C++ to Perl.
It assumes that XS-adapter returns <MyClass*> to Perl, i.e. object ownership is shared between C++ and Perl. Another assumption is that MyClass is ref-counted object, i.e. it should support the following operations via ADL (argument dependent lookup):
void refcnt_inc(MyClass*); void refcnt_dec(MyClass*); std::uint32_t refcnt_get(MyClass*);
It assumes that XS-adapter returns std::shared_ptr<MyClass> to Perl, i.e. object ownership is shared between C++ and Perl.
The out method has optional const Sv& proto = {} parameter, which instructs how make the C++ pointer accessible in Perl.
const Sv& proto = {}
In that case SV* pointing to undef is created, and it is blessed into package() returned by the TypemapObject.
This is automatically applied behaviour by ParseXS for the output parameters, i.e. ClassXXX below:
ClassXXX
ClassXXX* ClassYYY::create_xxx()
This is also applied when there is a code in xs-adapters like:
ClassXXX* item = ... ; // get somehow pointer to ClassXXX RETVAL = xs::out<>(item);
In that case SV* pointing to undef is created, and it is blessed into the specified package name. This is needed to follow standart Perl inheritance model, i.e. respect when the current class might be derived in Perl or xs-adapter. In other words it works similar to:
package My::XXX; sub new { my $class = shift; my $obj = {}; return bless $obj => $class; } package My::YYY; use base qw/My::XXX; my $obj_a = My::XXX->new; my $obj_b = My::YYY->new;
In that case SV* the pointer to C++ object is attached to the existing SV*, the new SV* will not be created and the existing SV* will be returned as result of the operation.
The typical use case is C3-mixin and next::method awareness. It is similar to the perl code
next::method
sub new { my $class = shift; my $obj = $class->next::method(@_); ... return $obj; }
Ususally in xs-adapter the proto is obtained via
..::new(...) PROTO = Stash::from_name(CLASS).call_next(cv, &ST(1), items-1); RETVAL = new MyClass(); // ParseXS from XS::Framework will invoke xs::out automatically
In that it behaves the same as with the empty proto, but instead of using undef as base value to bless the default typemap's package(), it uses the provided object as base to bless. This is similar to
return bless {} => 'MyClass'; # or: return bless [] => 'MyClass'
This might be useful when it is known that the class will be extended from Perl, i.e. to avoid XS::Framework::obj2hv invocation in Perl descedant class.
If the default behaviour is not desirable, i.e. there is need to return blessed hashref and tolerate possible descendant classes, then the new method should be like:
new
Myclass* Myclass::new(...) { RETVAL = new Myclass(...) PROTO = Stash::from_name(CLASS).bless(Hash::create()); }
Beware of the orded above: the C++ class have to be created first, and only then Perl SV* wrapper should be created next. If the order is reversed and C++ class throws an exception in constructor, the C++ object is not constructed, and there is nothing to attach to Perl SV*; but in the SV* desctruction it will be detected (invalid object reference), and the corresponding warning will be shown.
It is possible to have type map for some const class, e.g.
struct Typemap<const MyClass*>: TypemapObject<const MyClass*, const MyClass*, ... >
It has the following sense: if C++ API returns const object, then corresponding Perl SV* wrapper will be marked as read only. Attempt to invoke non-const method on read-only SV* leads to runtime exception.
const-methods in xs-adapters should be marked as const in method attributes section.
const
Out of the box XS::Framework is shipped with the patrial specializations, when there is no need to write dependend/deduceable typemaps, when there is some basic typemap. For example, typemap<T*> is deduced from typemap<const T*>. This is written as:
typemap<const T*> -> typemap<T*>
Here are all rules:
typemap<const T*> -> typemap<T*> typemap<T*> -> Typemap<iptr<T>> typemap<T*> -> Typemap<iptr<T>&> typemap<T*> -> Typemap<T&>
List of predefined typemap:
int8_t int16_t int32_t int64_t uint8_t uint16_t uint32_t uint64_t float double bool Simple Ref Glob Array Hash Stash std::string panda::string_view std::vector<T> std::map<Key, T>
Perl class written in XS (file with .xs/xsi extension).
The instance of Perl object (SV*), created from perl script, but stored outside of Perl (i.e. in C++/C/XS). The tricky part is to temporally "prolong" SV* lifetime in some C/C++ contrainer outside of Perl interpreter.
To install XS::Framework, copy and paste the appropriate command in to your terminal.
cpanm
cpanm XS::Framework
CPAN shell
perl -MCPAN -e shell install XS::Framework
For more information on module installation, please visit the detailed CPAN module installation guide.