NAME
XS::Framework::Manual::Typemap - XS::Framework C++ typemap API reference
C++ typemaps
Generic typemaps
Typemap is a concept, i.e. C++ struct with certain expectations, described below.
Typemaps are reponsible for transferring objects between perl scripts and C++ code. In the most general case typemap should be a full-specialization of C++ policy class, i.e. stateless class with group of static methods in the xs
namespace. The specialization should be for the target C++ class, which objects should be available in perl scripts, e.g.
namespace xs {
struct Typemap<MyClass> {
static [inline] MyClass in(SV* arg) { ... }
static [inline] Sv out(const MyClass& var, const Sv& proto = {}) { ... }
static [inline] void destroy (const MyClass&, SV*) { ... }
}
} // end of namespace xs
First af all for MyClass
value or pointer semantics should be defined. Objects with value semantics are allocated on C++ stack and their destructor is automatically called when object is goes out of scope; it is suitable for short-living transient objects like integer, string etc., and for std::shared_ptr
/ panda::iptr
. A typemap specialization should start as
struct Typemap<MyClass> { ...
For objects with pointer semantics the only possible option is to return pointer to MyClass
and then later, may be, clean it up in destroy
method (see below). A typemap specialization should start as:
struct Typemap<MyClass*> { ...
As the objects with value semantics is usually applicable for light-weight primitive objects structures with negligible construction and destruction costs (including std::shared_ptr), the classes with pointer semantics is the most general case.
static [inline] MyClass in(SV* arg) { ...
static [inline] MyClass* in(SV* arg) { ...
in
method is responsible for returning C++ object from perl SV* arg. It's intention is to prepare C++ object to work with (external) C++ API, without being encumbered with Perl details. For object with value semantics it usually return the (transient) object copy, e.g. configuration object created from perl Hashes; but in the case of pointer semantics the in
method should extract previously constructed object from perl SV*, i.e. return pointer MyClass*. Beware to do not construct object on heap and return pointer to it in the in
method, as this leads to memory leak because the XS::framework will not delete
the pointer.
static [inline] Sv out(const MyClass& var, const Sv& proto = {}) { ...
static [inline] Sv out(const MyClass* var, const Sv& proto = {}) { ...
The out
method is responsible for returning Perl wrapper from C++ object to let it be useable from perl scripts. The wrapper should be returned in form of Sv (see XS::Framework::Sv). Usually it should return reference to blessed perl object (newRV), however it might return any other suitable perl SV*, e.g. number, string, or reference to Array/Hash. The proto
argument serves as a hint without any restrictions on it's usage or interpretation. In future specializations (see TypemapObject
below) it is used as information which final class an object MyClass should be blessed into, i.e. MyPackage::MyClass
.
static [inline] void destroy (const MyClass&, SV*) { }
The <destroy> method is responsible for cleaning up C++ object when Perl's SV* wrapper (returned in out
method), is about to be destroyed in Perl: it is not always enough to have MyClass destructor, as destoy
method is called the link between C++ and perl is going to be deleted. For transient objects like string, int, date, etc. which do not have some permanent connection with Perl's SV* it is OK to let the method be empty. For pointer-semantics objects it depends: if your XS-code is responsible for object instantiation, then in the sake of resource leaks prevension it should delete it, i.e.
static [inline] void destroy (const MyClass* obj, SV*) { delete obj; }
If MyClass object was somehow customly constructed and expect it to be customly destructed, that can be done as
static [inline] void destroy (const MyClass* obj, SV*) {
obj->dec_refcounter();
// or GlobalPool::release(obj)
}
The perl scalar SV* might be used as place to store additional information, assosicated with the concrete C++ object instance, e.g. the used memory pool, then the memory pool can be extracted from SV* and object can be deleted with it.
static [inline] void destroy (const MyClass* obj, SV* sv) {
MyPool* pool = ...; // somehow get pool pointer from SV*
pool->release(obj);
}
There might be other uses cases. The destroy
method is unconditionally inserted for XS-classes in DESTROY
at pre-processing step by Parse::XS if there is input typemap. So, the method destroy
have to be defined, but it can be empty. The destroy
method might be invoked a few times in case of inheritance of XS-adapters, this issues is already solved in TypemapObject.
Let's summarize the for pointer semantics case: in
- unwraps / extracs C++ object, out
wraps/packs C++ object in perl scalar, destroy
deletes Perl and C++ association. A reader might ask the question: where there my C++ object is created? The answer is: the object instantiation left out of typemap scope by design. C++ object instantiation should be done in user's XS-code (aka XS-adapter).
Hovewer, writing typemap specialization for MyClass
from scratch is a bit cumbersome because devepoler still have to implement that wrap/unwrap code in using perl API. There are some ready-made and tested patterns, which will help you greatly with that. Go on reading the following sections.
TypemapBase
The TypemapBase
is basic implementation of concept TypeMap. It ships with empty destroy
method; it is expected that client typemap will inherit from TypemapBase
and provide required methods:
template<>
struct Typemap<MyClass*>: public TypemapBase<MyClass*> {
static inline MyClass* in (SV*) { ... }
static inline Sv out (const MyClass* var, const Sv& proto = {}) { ... }
};
TypemapObject
struct Typemap<MyFinalClass>: TypemapObject<MyBaseClass, MyFinalClass, LifetimePolicy, StoragePolicy, CastingPolicy>;
TypemapObject is helper class in creating typemaps, when there is a blessed Perl object and corresponding C++ object, i.e. one object to one object relationship. In the best case all behaviour can be specified via policies without need of additional code writing.
It is expected that custom typemap will inherit TypemapObject
, i.e. in common cases it should be enough to have something like:
template <>
struct Typemap<geos::geom::Triangle*>: TypemapObject<geos::geom::Triangle*, geos::geom::Triangle*, ObjectTypePtr, ObjectStorageMG, StaticCast>{
static panda::string_view package() {return "Geo::Geos::Triangle"; }
};
Template parameters overview
CastingPolicy
defines how void*
is casted to MyFinalClass
. In most of cases the StaticCast
should be used as it has zero-runtime costs; in rare cases, i.e. when MyFinalClass
participates in virtual inheritance, the DynamicCast
should be used. Under the hood the DynamicCast
uses panda::dyn_cast
which is faster implementation than standard dynamic_cast
.
StoragePolicy
defines how C++ object pointers can be stored in Perl SVs. It is available both for value-semantics object and pointer-semantic objects, however for value-semantics it might be a little bit tricky to do. Shipped implementations: ObjectStorageIV
, ObjectStorageMG
, ObjectStorageMGBackref
LifetimePolicy
defines how C++ defines convertion rules (between void*
and MyClass
), defines deletion (see destroy
method in generic typemap), upcast (from DerivedClass to BasicClass) and downcast (from BasicClass to DerivedClass) policies. Shipped implementations: ObjectTypePtr
, ObjectTypeForeignPtr
, ObjectTypeRefcntPtr
, ObjectTypeSharedPtr
.
MyBaseClass
and MyFinalClass
are needed to define the most specific and the most generic classes handled via typemap. If no class hierarchy have to be managed (i.e. in Perl only <MyFinalClass> should be visible), then MyBaseClass
can be the same as MyFinalClass
.
methods
static [inline] MyClass in(SV* arg) { ... }
static [inline] Sv out(const MyClass& var, const Sv& proto = {}) { ... }
static [inline] Sv create (const MyClass& var, const Sv& proto = Sv()) { ... }
static [inline] void destroy (const MyClass&, SV*) { ... }
static [inline] void dispose (const MyClass&, SV*) { ... }
static [inline] MyClass dup (const MyClass& obj) { return obj; }
in(SV* arg)
does the same as in typemap, i.e. returns C++ from Perl SV*. It is implemented as: 1) delegate to StoragePolicy
to get the void*
, and then to LifetimePolicy
to convert MyBasicType*
and then upcast it to MyFinalType*
.
out(const TYPE& var, const Sv& proto = Sv())
does the same as in typemap, i.e. returns Perls SV* from C++ object. In implementation it asks StoragePolicy
to wrap/pack MyClass object in Perl SV* and ends up in create
method below for classes without BackRefs support.
The create
method is similar to out
method described above: it is responsible for creating Perl wrapper from C++ object. That distinguish is irrelevant unless MyClass
is supposed to support BackRefs; in the last case, i.e. if MyClass
is supports BackRefs, if it is possible out
returns previously created and cached SV* wrapper, otherwise it calls create
method. It is actually a bit more complex, see out behaviour below.
destroy(const TYPE& var, const Sv& proto = Sv())
- does the same as in typemap, i.e. cleans up C++ object when Perls SV* is about to be deleted. It is implemented in safe manner, i.e. real object deletion (via dispose
if StoragePolicy
) is executed only once, despite the fact that it migth invoked many times from DESTROY
method in XS-adapter due to inheritance. If the class is auto-disposable (defined in StoragePolicy
), it is actually destroyed via dispose
method. In other words, after the method invokation C++ object and Perls SV* might be destroyed.
dispose(const TYPE& var, SV* arg)
- does the same as destroy
method in typemap; i.e. unconditionally cleans up C++ object and Perl SV*. The actual job is delegated to LifetimePolicy::destroy
package()
it is expected to be overriden in user-defined typemap. It should return the string for perl class; it is invoked only when Sv hint
argument is not supplied in out
or create
methods. Unless overriden by user the package()
method will throw runtime exception.
dup(const MyClass& obj
is called when perl starts another thread, otherwise it is never invoked. It should return MyClass* object pointer, which will be attached to SV* duplicate in the new perl thread. The default behaviour is retain policy, i.e. it returns exatcly the same object as in main thread. This implies siginificant restriction on MyClass: it should be thread-safe and ref-counted; the restrictions cannot be checked at compile time, and if the conditions aren't met Perl interpreter will crash at runtime. Another possible policy is clone, i.e. underlying C++ object should return copy of itself, and that copy will be attached to SV*. The last possible policy is skip, i.e. when undef
will be attached to cloned SV*; to implement it <CLONE_SKIP> should be defined in XS-adapter and it should return true
.
Wrapping C++ objects into Perl SV* aka storage policy
ObjectStorageIV
ObjectStorageMG
ObjectStorageMGBackref
XS::Framework ships three policies, defining how C++ object pointers can be stored in Perl SVs. They are applicable for both for value semantics and pointer semantics, hovewer for value-semantics it might be a little bit tricky to do, i.e. value have to be stored over pointer.
Storage policy concept
Storage policy concept has the following methods. They are needed only if custom (non-shipped with XS::Framework) policy is supplied.
template <class TYPEMAP, class TYPE>
struct ObjectStorage {
static const bool auto_disposable = false;
static inline void* get (SV* arg) { ... }
static inline void set (SV* arg, void* ptr) { ... }
static Sv out (const TYPE& var, const Sv& proto) { ... }
};
The auto_disposable
property defines, whether the destroy
method of TypemapObject
should be forwarded to LifetimePolicy::destroy
. If underlying perl SV* is able to do self-cleaning and invoke LifetimePolicy::destroy
, then this property should be set to true
. Otherwise, there is a need of assitance to do proper cleaning custom (i.e. ObjectStorageIV
), the property should be set to false
.
The get
and set
methods are responsible for storing/retriving underlying object pointer (downcasted to void*
) in/from Perl SV*.
out(const TYPE& var, const Sv& proto = Sv())
does the same as in typemap, i.e. returns Perls SV* from C++ object. If there is no BackRefs support, then it should forward call to TYPEMAP::create
, otherwise, if possible, it should extract Backref and return it. Backrefs support is non-trivial topic, see ObjectStorageMGBackref
source code to get the ideas.
ObjectStorageIV
ObjectStorageIV
relies on the equality sizeof(MyClass*) == sizeof(IV)
. So, it stores pointer as integer in Perls SV*. This is the most frequently used approach in CPAN, hovewer, in general it is not the best in terms on performance.
This is non-autodisposable policy, i.e. you have to insert DESTROY
method in XS-adapter to help it to be forwarded to LifetimePolicy::destoy
, otherwise, if there was memory allocations for C++ object, there will be memory leak, as there is no other way to delete underlying C++ object. The actual object deletion is performed in LifetimePolicy::destroy
.
As the XS-adapter DESTROY
method conforms to general perl rules, it is executed in eval()
, which leads to significant slow down on object deletion. Other storage policies do not suffer from that.
There is another restriction of this policy: inheritance. As there is no more free space in underlying perl SV*, the descedant Perl classes can only add methods, but no data, i.e. to store datat the SV* should be upgraded to HashRef or ArrayRef, but that's not possible. The only possible way to overcome that is use inheritance in C++. Due to space restriction there might be issues with multiple inheritance, as there is need to store additional pointer. Other storage policies do not suffer from constraint.
This storage policy has smallest memory footprint and no additional memory allocations, i.e. Perl object creation is the fastest.
ObjectStorageMG
ObjectStorageMG
uses Perl Magic to store pointer. It have to allocate additional memory for Perl Magic, hence, object construction is a bit slower; also that Magic occupies some memory, in other words it has some non-zero costs to use that storage policy.
The SV* itself remains free and it can be used in Child classes defined in Perl, i.e.
package MyPerlClass;
use base qw/MyCPPClass/;
sub new {
my ($class, $data) = @_;
# construct C++ object and XS-wrapper
my $obj = SUPER::new($class, $data);
# extend/upgrade UNDEF to Hash.
XS::Framework::obj2hv($obj);
$obj->{extended_field} = $data->{extended_field};
return $obj;
}
This is autodisposable storage policy, i.e. custom destruction actions are guaranteed to be invoked via LifetimePolicy::destroy
. That's why it is NOT recommended to have DESTOY
method in XS-adapter, as it has significant performance costs due to eval
context execution.
ObjectStorageMGBackref
Basically it is the same as <ObjectStorageMG> with the abilitity to cache BackRef and return it on demand via out
method by the cost of slightly increased memory footprint in perl descendant classes and slightly slower object construction.
Under certain conditions, i.e. when perl object of descendant perl class has been created in perl and frequently accessed from other XS-adapter, the storage out
method returns cached object, which is much faster then allocating and initializing perl wrapper.
Storage policy feature matrix
[*1] It allows to use C++ inheritance, of course, and even perl single inheritance, but as it gives not hash, you cannot store additional properties in Perl inheritance, which makes it almost useless.
[*2] For ObjectStorageIV
you have to write your own custom destroy
method, which is executed in Perl's eval closure, which is quite slow; for ObjectStorageMG*
you can (but we do not recommend) write destroy
method, which will be the same speed slow too, instead the destroy
method can be written in liftetime object policy, which is executed in very fast way as usual C++ object destructror.
[*3] It heavily depends on property access pattern in your program: when (a), the property XS-object or descendent XS-object class (i.e. with perl implementation) has been created from Perl, and (b) is is accessed frequently as property of some other XS-container, then for ObjectStorageMGBackref
the accessor gets significant boost, because it always returns cashed object instead of allocating Perl-wrapper for it on each invocation.
LifetimePolicy for managing C++ objects
ObjectTypePtr
ObjectTypeForeignPtr
ObjectTypeRefcntPtr
ObjectTypeSharedPtr
LifetimePolicy
defines how C++ defines convertion, deletion, upcasting and downcasting rules. As there is no need to write custom LifetimePolicy the concept is not described here. Let us know if we are wrong here.
LifetimePolicy
is orthogonal to StoragePolicy
, i.e. they can be used in any combination independently.
ObjectTypePtr
It assumes that XS-adapter creates object pointer MyClass*
on heap and trasfers ownership to Perl. The policy will invoke delete
on the object when it's SV* is about to be destroyed.
ObjectTypeForeignPtr
It assumes that XS-adapter returns object pointer MyClass*
without transferring ownership to Perl. The deletion policy is empty. It might be useful when singleton object is returned from C++ to Perl.
ObjectTypeRefcntPtr
It assumes that XS-adapter returns <MyClass*> to Perl, i.e. object ownership is shared between C++ and Perl. Another assumption is that MyClass
is ref-counted object, i.e. it should support the following operations via ADL (argument dependent lookup):
void refcnt_inc(MyClass*);
void refcnt_dec(MyClass*);
std::uint32_t refcnt_get(MyClass*);
ObjectTypeSharedPtr
It assumes that XS-adapter returns std::shared_ptr<MyClass> to Perl, i.e. object ownership is shared between C++ and Perl.
C++ typemap lifetime policy feature matrix
out
behaviour
The out
method has optional const Sv& proto = {}
parameter, which instructs how make the C++ pointer accessible in Perl.
- it might be empty
-
In that case SV* pointing to
undef
is created, and it is blessed intopackage()
returned by the TypemapObject.This is automatically applied behaviour by ParseXS for the output parameters, i.e.
ClassXXX
below:ClassXXX* ClassYYY::create_xxx()
This is also applied when there is a code in xs-adapters like:
ClassXXX* item = ... ; // get somehow pointer to ClassXXX RETVAL = xs::out<>(item);
- it might contain Perl package name (in form of string or Stash)
-
In that case SV* pointing to
undef
is created, and it is blessed into the specified package name. This is needed to follow standart Perl inheritance model, i.e. respect when the current class might be derived in Perl or xs-adapter. In other words it works similar to:package My::XXX; sub new { my $class = shift; my $obj = {}; return bless $obj => $class; } package My::YYY; use base qw/My::XXX; my $obj_a = My::XXX->new; my $obj_b = My::YYY->new;
- it might be already blessed Perl object
-
In that case SV* the pointer to C++ object is attached to the existing SV*, the new SV* will not be created and the existing SV* will be returned as result of the operation.
The typical use case is C3-mixin and
next::method
awareness. It is similar to the perl codesub new { my $class = shift; my $obj = $class->next::method(@_); ... return $obj; }
Ususally in xs-adapter the
proto
is obtained via..::new(...) PROTO = Stash::from_name(CLASS).call_next(cv, &ST(1), items-1); RETVAL = new MyClass(); // ParseXS from XS::Framework will invoke xs::out automatically
- it might be something else (ArrayRef or HashRef)
-
In that it behaves the same as with the empty
proto
, but instead of usingundef
as base value to bless the default typemap's package(), it uses the provided object as base to bless. This is similar toreturn bless {} => 'MyClass'; # or: return bless [] => 'MyClass'
This might be useful when it is known that the class will be extended from Perl, i.e. to avoid XS::Framework::obj2hv invocation in Perl descedant class.
If the default behaviour is not desirable, i.e. there is need to return blessed hashref and tolerate possible descendant classes, then the new
method should be like:
Myclass* Myclass::new(...) {
RETVAL = new Myclass(...)
PROTO = Stash::from_name(CLASS).bless(Hash::create());
}
Beware of the orded above: the C++ class have to be created first, and only then Perl SV* wrapper should be created next. If the order is reversed and C++ class throws an exception in constructor, the C++ object is not constructed, and there is nothing to attach to Perl SV*; but in the SV* desctruction it will be detected (invalid object reference), and the corresponding warning will be shown.
const TypemapObject
It is possible to have type map for some const class, e.g.
struct Typemap<const MyClass*>: TypemapObject<const MyClass*, const MyClass*, ... >
It has the following sense: if C++ API returns const object, then corresponding Perl SV* wrapper will be marked as read only. Attempt to invoke non-const method on read-only SV* leads to runtime exception.
const-methods in xs-adapters should be marked as const
in method attributes section.
Auto-deduced typemaps
Out of the box XS::Framework is shipped with the patrial specializations, when there is no need to write dependend/deduceable typemaps, when there is some basic typemap. For example, typemap<T*> is deduced from typemap<const T*>. This is written as:
typemap<const T*> -> typemap<T*>
Here are all rules:
typemap<const T*> -> typemap<T*>
typemap<T*> -> Typemap<iptr<T>>
typemap<T*> -> Typemap<iptr<T>&>
typemap<T*> -> Typemap<T&>
List of predefined typemap:
int8_t
int16_t
int32_t
int64_t
uint8_t
uint16_t
uint32_t
uint64_t
float
double
bool
Simple
Ref
Glob
Array
Hash
Stash
std::string
panda::string_view
std::vector<T>
std::map<Key, T>
GLOSSARY
XS-adapter, XS-wrapper
Perl class written in XS (file with .xs/xsi extension).
BackRef
The instance of Perl object (SV*), created from perl script, but stored outside of Perl (i.e. in C++/C/XS). The tricky part is to temporally "prolong" SV* lifetime in some C/C++ contrainer outside of Perl interpreter.