Treex::Tool::EnglishMorpho::Lemmatizer - rule based lemmatizer for English


version 2.20151102


 use Treex::Tool::EnglishMorpho::Lemmatizer;
 my $lemmatizer    = Treex::Tool::EnglishMorpho::Lemmatizer->new();
 my ($word,  $tag) = qw( goes VBZ );
 my ($lemma, $neg) = $lemmatizer->lemmatize($word, $tag);
 # $lemma = 'go', $neg = 0
 ($lemma, $neg) = $lemmatizer->lemmatize('unhappy', 'JJ');
 # $lemma = 'happy', $neg = 1



Accepts pair of word and tag. Produces pair with its lemma and indication if word was negation



  • noun -s (dogs -> dog, ponies -> pony,..., mice -> mouse)

  • verb -s (does -> do,...)

  • verb -ing

  • verb -ed, -en

  • adjective/adverb -er

  • adjective/adverb -est

  • cut off negative prefixes (un|in|im|non|dis|il|ir)

Input requirements


doesn't should be tokenized as two words: does and n't (It will be lemmatized as do and not).


Correct tagging (Penn style) is quite crucial for Lemmatizer to work. For example it doesn't change words with tags NN and NNP (it changes only NNS and NNPS). So (pence, NN) -> pence, but (pence, NNS) -> penny.

Differences from the previous implementation

Modul PEDT::MorphologyAnalysis uses Morpha (written in Flex) and in some cases gives different lemmatization.

Adverbs and adjectives.

Morpha leaves comparatives and superlatives unchanged. PEDT::MorphologyAnalysis does only basic analysis (later -> lat).

Capitalization of proper names
Changes of NN
Latin words

Declination of words with latin origin is not covered by any Lemmatizer rules on purpose. There are few widely known english words with latin origin which are (or should be) covered by exception files (f.e. indices NNS -> index). In my opinion, it is better, especially for translation purposes, to leave the other latin words unchanged. Mostly they will have the same form also in the target language (biological terms like Spheniscidae). BTW: Errors made by Morpha latin fallbacks are sometimes funny: sci-fi -> sci-fus, Mitsubishi -> mitsubishus, Shanghai -> shanghaus,...


  • this POD documentation !!!

  • better list of exceptions

  • change exceptions format from tsv to stored perl hash


Martin Popel <>


Copyright © 2008 - 2011 by Institute of Formal and Applied Linguistics, Charles University in Prague

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

1 POD Error

The following errors were encountered while parsing the POD:

Around line 416:

Non-ASCII character seen before =encoding in '©'. Assuming UTF-8