How to use my customer lucene analyzer(tokenizer)?


(fancyerii) #1

I want to use my own Chinese analyzer and I can write lucene analyzer class
myself. How can I integrate it to elasticsearch?
I googled and
found http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/custom-analyzers.html.
But it only combine existing tokenizers and filters. I can use tokenizer
writing in java by myself.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Tihomir Lichev) #2

I think you should write your own plugin with your analyzer implementation,
so you can register your analyzer into the ES and use it later on.
You could have a look and see how it done maybe in this
plugin: https://github.com/elasticsearch/elasticsearch-analysis-kuromoji

05 август 2014, вторник, 09:58:16 UTC+3, fanc...@gmail.com написа:

I want to use my own Chinese analyzer and I can write lucene analyzer
class myself. How can I integrate it to elasticsearch?
I googled and found
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/custom-analyzers.html.
But it only combine existing tokenizers and filters. I can use tokenizer
writing in java by myself.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fe02dc31-991c-49f9-9820-63ba96a37d33%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jun Ohtani) #3

Hi,

I think this plugin will be helpful for you.


2014/08/05 15:58 fancyerii@gmail.com:

I want to use my own Chinese analyzer and I can write lucene analyzer
class myself. How can I integrate it to elasticsearch?
I googled and found
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/custom-analyzers.html.
But it only combine existing tokenizers and filters. I can use tokenizer
writing in java by myself.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPW8A5xqnsWa76TJuSAWwxPyrUA8F9cJ6tyTs2ZJxXFpb4vEMg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(art) #4

I have the same question about using an analyzer I have written as a
plug-in for ElasticSearch 1.3.


demonstrates only how to use the tokenizers in combination with the
built-in CustomAnalyzer. They do not show how to use the kuromoji analyzer
itself.

When I try to specify my analyzer for a field, I get errors like this:

MapperParsingException[Analyzer [special_analyzer] not found for field
[foo]];

Can you show an example of how to specify the kuromoji analyzer for a
field? I should then be able to adapt it for use with my plugin analyzer.

Thanks in advance,
Art

On Tuesday, August 5, 2014 12:34:42 AM UTC-7, Jun Ohtani wrote:

Hi,

I think this plugin will be helpful for you.

https://github.com/elasticsearch/elasticsearch-analysis-kuromoji
2014/08/05 15:58 <fanc...@gmail.com <javascript:>>:

I want to use my own Chinese analyzer and I can write lucene analyzer
class myself. How can I integrate it to elasticsearch?
I googled and found
http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/custom-analyzers.html.
But it only combine existing tokenizers and filters. I can use tokenizer
writing in java by myself.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/da795847-3ea2-4afb-9a7b-aefdd6f111a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jun Ohtani) #5

Hi Art,

I wrote an example specifying the kuromoji analyzer("kuromoji") and custom
analyzer("my_analyzer") for a field.

curl -XPUT "http://localhost:9200/kuromoji-sample" -d'
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "kuromoji_tokenizer",
"filter": [
"kuromoji_baseform"
]
}
}
}
}
},
"mappings": {
"sample": {
"properties": {
"title": {
"type": "string",
"analyzer": "my_analyzer"
},
"body" : {
"type": "string",
"analyzer": "kuromoji"
}
}
}
}
}'

I hope that it will be helpful for you.

2014-08-22 9:18 GMT+09:00 art@safeshepherd.com:

I have the same question about using an analyzer I have written as a
plug-in for ElasticSearch 1.3.

https://github.com/elasticsearch/elasticsearch-analysis-kuromoji/blob/es-1.3/README.md
demonstrates only how to use the tokenizers in combination with the
built-in CustomAnalyzer. They do not show how to use the kuromoji analyzer
itself.

When I try to specify my analyzer for a field, I get errors like this:

MapperParsingException[Analyzer [special_analyzer] not found for field
[foo]];

Can you show an example of how to specify the kuromoji analyzer for a
field? I should then be able to adapt it for use with my plugin analyzer.

Thanks in advance,
Art

On Tuesday, August 5, 2014 12:34:42 AM UTC-7, Jun Ohtani wrote:

Hi,

I think this plugin will be helpful for you.

https://github.com/elasticsearch/elasticsearch-analysis-kuromoji
2014/08/05 15:58 fanc...@gmail.com:

I want to use my own Chinese analyzer and I can write lucene analyzer
class myself. How can I integrate it to elasticsearch?
I googled and found http://www.elasticsearch.org/guide/en/
elasticsearch/guide/current/custom-analyzers.html. But it only combine
existing tokenizers and filters. I can use tokenizer writing in java by
myself.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/da795847-3ea2-4afb-9a7b-aefdd6f111a0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/da795847-3ea2-4afb-9a7b-aefdd6f111a0%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Jun Ohtani
blog : http://blog.johtani.info

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPW8A5x9xr_4OoHeBjCO%2BYYJHN3-O5pSk2fB7-v0rETKPMRHkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(art) #6

Thanks Jun, that was helpful. It helped me to realize I had not fully
connected my analyzer plugin.

On Thursday, August 21, 2014 11:23:47 PM UTC-7, Jun Ohtani wrote:

Hi Art,

I wrote an example specifying the kuromoji analyzer("kuromoji") and custom
analyzer("my_analyzer") for a field.

curl -XPUT "http://localhost:9200/kuromoji-sample" -d'
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "kuromoji_tokenizer",
"filter": [
"kuromoji_baseform"
]
}
}
}
}
},
"mappings": {
"sample": {
"properties": {
"title": {
"type": "string",
"analyzer": "my_analyzer"
},
"body" : {
"type": "string",
"analyzer": "kuromoji"
}
}
}
}
}'

I hope that it will be helpful for you.

2014-08-22 9:18 GMT+09:00 <a...@safeshepherd.com <javascript:>>:

I have the same question about using an analyzer I have written as a
plug-in for ElasticSearch 1.3.

https://github.com/elasticsearch/elasticsearch-analysis-kuromoji/blob/es-1.3/README.md
demonstrates only how to use the tokenizers in combination with the
built-in CustomAnalyzer. They do not show how to use the kuromoji analyzer
itself.

When I try to specify my analyzer for a field, I get errors like this:

MapperParsingException[Analyzer [special_analyzer] not found for field
[foo]];

Can you show an example of how to specify the kuromoji analyzer for a
field? I should then be able to adapt it for use with my plugin analyzer.

Thanks in advance,
Art

On Tuesday, August 5, 2014 12:34:42 AM UTC-7, Jun Ohtani wrote:

Hi,

I think this plugin will be helpful for you.

https://github.com/elasticsearch/elasticsearch-analysis-kuromoji
2014/08/05 15:58 fanc...@gmail.com:

I want to use my own Chinese analyzer and I can write lucene analyzer
class myself. How can I integrate it to elasticsearch?
I googled and found http://www.elasticsearch.org/guide/en/
elasticsearch/guide/current/custom-analyzers.html. But it only combine
existing tokenizers and filters. I can use tokenizer writing in java by
myself.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c3fe52cd-8cb5-4c53-b0fe-87183deb45bf%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/da795847-3ea2-4afb-9a7b-aefdd6f111a0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/da795847-3ea2-4afb-9a7b-aefdd6f111a0%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--

Jun Ohtani
blog : http://blog.johtani.info

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a792d08d-534f-4619-bfcb-0f01262b6c51%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #7