Russian Morphology for Apache Lucene

Russian and English morphology for Java and Apache Lucene 9.3 framework based on open source dictionary from site АОТ. It uses dictionary base morphology with some heuristics for unknown words. It supports a homonym for example for a Russian word “вина” it gives two variants “вино” and “вина”.

How to use

Build project, by running mvn clean package, this will provide you the latest versions of the artifacts - 1.5, add it to your classpath. You could select which version to use - Russian or English.

Now you can create a Lucene Analyzer:

  RussianAnalayzer russian = new RussianAnalayzer();
  EnglishAnalayzer english = new EnglishAnalayzer();

You can write you own analyzer using filter that convert word in it’s right forms.

  LuceneMorphology luceneMorph = new EnglishLuceneMorphology();
  TokenStream tokenStream = new MorphlogyFilter(result, luceneMorph);

Because usually LuceneMorphology contains a lot data needing for it functionality, it is better didn’t create this object for each MorphologyFilter.

Also if you need get a list of base forms of word, you can use following example

 LuceneMorphology luceneMorph = new EnglishLuceneMorphology();
 List<String> wordBaseForms = luceneMorph.getMorphInfo(word);

Solr

You can use the LuceneMorphology as morphology filter in a Solr schema.xml using a MorphologyFilterFactory:

<fieldType name="content" class="solr.TextField" positionIncrementGap="100">
      <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
		<filter class="org.apache.lucene.analysis.morphology.MorphologyFilterFactory" language="Russian"/>
		<filter class="org.apache.lucene.analysis.morphology.MorphologyFilterFactory" language="English"/>
      </analyzer>
</fieldType>

Just add morphology-1.5.jar in your Solr lib-directories

Restrictions

• It works only with UTF-8.
• It assume what letters е and ё are the same.
• Word forms with prefixes like “наибольший” treated as separate word.

License

Apache License, Version 2.0