# Kuromoji analyzer default character/token filters

**URL:** <https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976>\
**Category:** Elasticsearch\
**Created:** [December 5, 2016, 3:21am UTC](https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976 "2016-12-05T03:21:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Luke3113](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luke3113/32/13587_2.png) [@Luke3113](https://discuss.elastic.co/u/Luke3113)\
**Post date:** [December 5, 2016, 3:21am UTC](https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976/1 "2016-12-05T03:21:04Z")

</div>

Hi I'm using Elasticsearch 5.0.1 with the Kuromoji plugin.  
Until now I've merely used its default configuration by using the following mapping:

`"analyzer": "kuromoji"`

However as stated in the docs ([Kuromoji analyzer](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji-analyzer.html)) it consists of a character filter, tokenizer and various token filters. Some of these seem to be applied in the analyzer's default settings (e.g. kuromoji\_baseform), some not (e.g. kuromoji\_number token filter).

**I would like to know which filters are being used in Kuromojis default setting.** Is there any way to find out with an API-call?  
I've tried looking into the Plugin-sources as this doesn't seem to be documented: [KuromojiAnalyzerProvider.java](https://github.com/elastic/elasticsearch/blob/master/plugins/analysis-kuromoji/src/main/java/org/elasticsearch/index/analysis/KuromojiAnalyzerProvider.java)  
However, there doesn't seem to be any character-filters or tokenizers defined.

Thank you!

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 5, 2016, 5:18am UTC](https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976/2 "2016-12-05T05:18:53Z")

</div>

May be look here:

> <https://github.com/apache/lucene-solr/blob/master/lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/JapaneseAnalyzer.java>

---

<div class="post-metadata">

**Author:** ![Luke3113](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luke3113/32/13587_2.png) [@Luke3113](https://discuss.elastic.co/u/Luke3113)\
**Post date:** [December 5, 2016, 7:29am UTC](https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976/3 "2016-12-05T07:29:00Z")

</div>

Oh great thanks, I didn't know the JapaneseAnalyzer is also part of Kuromoji!  
So it seems from this file that following components are used by default:

- kuromoji\_tokenizer - tokenizer
- kuromoji\_baseform - token filter
- kuromoji\_part\_of\_speech - token filter
- cjk\_width - token filter
- ja\_stop - token filter
- kuromoji\_stemmer - token filter
- lowercase - token filter

Doesn't seem to be used by default:

- kuromoji\_iteration\_mark - character filter
- kuromoji\_number - token filter
- kuromoji\_readingform - token filter

---

<div class="post-metadata">

**Author:** ![johtani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johtani/32/44956_2.png) [@johtani](https://discuss.elastic.co/u/johtani)\
**Post date:** [December 5, 2016, 8:03am UTC](https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976/4 "2016-12-05T08:03:08Z")

</div>

And additional information, `stoptags` and `stopwords` are here.

[https://github.com/apache/lucene-solr/tree/master/lucene/analysis/kuromoji/src/resources/org/apache/lucene/analysis/ja](https://github.com/apache/lucene-solr/tree/master/lucene/analysis/kuromoji/src/resources/org/apache/lucene/analysis/ja)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 2, 2017, 8:03am UTC](https://discuss.elastic.co/t/kuromoji-analyzer-default-character-token-filters/67976/5 "2017-01-02T08:03:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
