Parse::StringTokenizer - Extract fields from strings No symbols are exported by default.
use Parse::StringTokenizer; my $tokenizer = String::Tokenizer->new(); my $str = 'a b c'; die unless 'a' eq $tokenizer->shift($str); die unless 'b c' eq $str;
Nothing exported by default
This module requires these other modules and libraries:
Error::Programatic Perl::Module
Similiar in spirit to strtok.
strtok
Default delimiter is whitespace (regex \s).
\s
Default quotes which protect the delimiter are: '"
'"
options:
-delim => $delimiter # Token separator -quotes => $quotes # Quote characters which protect the # delimiter, and will be ignored -contained => $chars # Character pairs which protect the delimiter -keywords => \@keywords # Words which will not be split -preserve => 1 # Preserve quotes which surround each field
By default, the following is considered three tokens:
one 'and a two' "and a three"
-preserve
TODO: when packing the field will be double quoted if it contains the primary delimiter. If the field is quoted, then do not double-quote
Returns the new field count.
Quotes and contained segments can have escaped characters, like:
'don\'t break me'
right now, it comes back as
dont\'t break me
TODO, unescape it so it becomes
dont't break me
the difficulty is that you DO NOT want to unescape the backslashes here
"my quotes \'are\' different"
which means reworking the RE such that the leading and tailing quotes are NOT removed, i.e., they remain part of the segment. and a subsequent step added to `unpack` which knows what to do by the presence of leading and trailing characters.
Ryan Gies <ryangies@cpan.org>
Copyright (C) 2014-2016 by Ryan Gies. All rights reserved. Copyright (C) 2006-2013 by Livesite Networks, LLC. All rights reserved. Copyright (C) 2000-2005 by Ryan Gies. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. * Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. * The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. To the best of our knowledge, no patented algorithms have been used. However, we do not have the resources to carry out a patent search, and therefore cannot give any guarantee of the above statement.
To install Data::Hub, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Data::Hub
CPAN shell
perl -MCPAN -e shell install Data::Hub
For more information on module installation, please visit the detailed CPAN module installation guide.