Unable to configure custom analyzer with rabbitmq river?

Sam_4 · January 18, 2012, 5:42am

Hi

I am working with elastic search and using RabbitMQ River. I want to
use custom analyzer. When I read about it I found few ways to
configure it but when I executed with the mapping it did not gave me
any parsing exception but also not got the predicted output while
searching.

Tested Output:

I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".

This made me sure that the custom analyzer is not configured
correctly.

I refer https://groups.google.com/group/elasticsearch/browse_thread/thread/bd1a6a94a8c94e20
but got no success when executed this mapping.
I used this mapping https://gist.github.com/1626452 but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks

kimchy · January 18, 2012, 9:04pm

Before you create the river, you need to precreate the index it will index
into, and have the relevant mappings set on it (and settings, like analysis
settings).

On Wed, Jan 18, 2012 at 7:42 AM, sam mishra.sameek@gmail.com wrote:

Hi

I am working with Elasticsearch and using RabbitMQ River. I want to
use custom analyzer. When I read about it I found few ways to
configure it but when I executed with the mapping it did not gave me
any parsing exception but also not got the predicted output while
searching.

Tested Output:

I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".

This made me sure that the custom analyzer is not configured
correctly.

I refer
https://groups.google.com/group/elasticsearch/browse_thread/thread/bd1a6a94a8c94e20
but got no success when executed this mapping.
I used this mapping River Mapping · GitHub but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks

Sam_4 · January 19, 2012, 7:35am

Thanks for reply shay.

As you said I created index with the defined mapping in it and river it.
The implemented class can be viewed at RabbitMQ River With Custom Analyzer · GitHub
.I have mentioned code snippet on gist SearchTest.java
#https://gist.github.com/1638251#file_search_test.java where
I am trying to search data "inc" with using logical name search_analyzer
but again the result is not expected as I included "inc" in Stop Word List
, again I am not sure the mapping I have set is being utilized or not.

I am stuck with the following points:

Can you please tell me the way, to verify the mapping details that are
currently being used on index using java.
If I am following wrong way while searching the data how can I correct
it to make it work using java API.

Thanks

On Thu, Jan 19, 2012 at 2:34 AM, Shay Banon kimchy@gmail.com wrote:

Before you create the river, you need to precreate the index it will index
into, and have the relevant mappings set on it (and settings, like analysis
settings).

On Wed, Jan 18, 2012 at 7:42 AM, sam mishra.sameek@gmail.com wrote:

Hi

I am working with Elasticsearch and using RabbitMQ River. I want to
use custom analyzer. When I read about it I found few ways to
configure it but when I executed with the mapping it did not gave me
any parsing exception but also not got the predicted output while
searching.

Tested Output:

I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".

This made me sure that the custom analyzer is not configured
correctly.

I refer
https://groups.google.com/group/elasticsearch/browse_thread/thread/bd1a6a94a8c94e20
but got no success when executed this mapping.
I used this mapping River Mapping · GitHub but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks

kimchy · January 19, 2012, 6:41pm

You are putting the analyzer config as a mapping. The analyzer config is
placed in the settings of an index (or in the config file), and mapping
define the how the json document ends up being indexed, for example, have a
specific field associated with a custom analyzer defined in your settings.

On Thu, Jan 19, 2012 at 9:35 AM, sameek mishra mishra.sameek@gmail.comwrote:

Thanks for reply shay.

As you said I created index with the defined mapping in it and river it.
The implemented class can be viewed at RabbitMQ River With Custom Analyzer · GitHub
.I have mentioned code snippet on gist SearchTest.java #https://gist.github.com/1638251#file_search_test.java where
I am trying to search data "inc" with using logical name search_analyzer
but again the result is not expected as I included "inc" in Stop Word List
, again I am not sure the mapping I have set is being utilized or not.

I am stuck with the following points:

Can you please tell me the way, to verify the mapping details that
are currently being used on index using java.

If I am following wrong way while searching the data how can I
correct it to make it work using java API.

Thanks

On Thu, Jan 19, 2012 at 2:34 AM, Shay Banon kimchy@gmail.com wrote:

Before you create the river, you need to precreate the index it will
index into, and have the relevant mappings set on it (and settings, like
analysis settings).

On Wed, Jan 18, 2012 at 7:42 AM, sam mishra.sameek@gmail.com wrote:

Hi

I am working with Elasticsearch and using RabbitMQ River. I want to
use custom analyzer. When I read about it I found few ways to
configure it but when I executed with the mapping it did not gave me
any parsing exception but also not got the predicted output while
searching.

Tested Output:

I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".

This made me sure that the custom analyzer is not configured
correctly.

I refer
https://groups.google.com/group/elasticsearch/browse_thread/thread/bd1a6a94a8c94e20
but got no success when executed this mapping.
I used this mapping River Mapping · GitHub but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks

Sam_4 · January 20, 2012, 7:53am

Thanks for your reply and support.the above solution you gave worked
perfectly for me.I have updated the gist that executed perfectly.

gist.github.com

https://gist.github.com/sameek/1638251

Analyzer Setting source

 {  
  "index" : {
"analysis" : {
"analyzer" : {
 "customindexanalyzer" : {
"type":"custom",
"tokenizer" : "whitespace",
"filter" : ["lowercase","asciifolding","length","mystopword"],
"char_filter" :["html_strip"]
 },

This file has been truncated. show original

Mapping index search

"test_type" : {
  "index_analyzer" : "customindexanalyzer",
"search_analyzer" : "customsearchanalyzer"
}

RabbitMqRiverIndexer.java

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package com.arosys.doqeap.esearch;
import com.arosys.configurationreader.ConfigurationReader;
import com.arosys.customexception.CustomException;
import com.arosys.doqeap.exception.ValueNotFoundException;
import com.arosys.logger.LoggerFactory;
import com.rabbitmq.client.Channel;

This file has been truncated. show original

There are more than three files. show original

On Jan 19, 11:41 pm, Shay Banon kim...@gmail.com wrote:

You are putting the analyzer config as a mapping. The analyzer config is
placed in the settings of an index (or in the config file), and mapping
define the how the json document ends up being indexed, for example, have a
specific field associated with a custom analyzer defined in your settings.

On Thu, Jan 19, 2012 at 9:35 AM, sameek mishra mishra.sam...@gmail.comwrote:

Thanks for reply shay.

As you said I created index with the defined mapping in it and river it.
The implemented class can be viewed athttps://gist.github.com/1638251
.I have mentioned code snippet on gist SearchTest.java #https://gist.github.com/1638251#file_search_test.java where
I am trying to search data "inc" with using logical name search_analyzer
but again the result is not expected as I included "inc" in Stop Word List
, again I am not sure the mapping I have set is being utilized or not.

I am stuck with the following points:

Can you please tell me the way, to verify the mapping details that
are currently being used on index using java.

If I am following wrong way while searching the data how can I
correct it to make it work using java API.

Thanks

On Thu, Jan 19, 2012 at 2:34 AM, Shay Banon kim...@gmail.com wrote:

Before you create the river, you need to precreate the index it will
index into, and have the relevant mappings set on it (and settings, like
analysis settings).

On Wed, Jan 18, 2012 at 7:42 AM, sam mishra.sam...@gmail.com wrote:

Hi

I am working with Elasticsearch and using RabbitMQ River. I want to
use custom analyzer. When I read about it I found few ways to
configure it but when I executed with the mapping it did not gave me
any parsing exception but also not got the predicted output while
searching.

Tested Output:

I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".

This made me sure that the custom analyzer is not configured
correctly.

I refer
https://groups.google.com/group/elasticsearch/browse_thread/thread/bd...
but got no success when executed this mapping.
I used this mappinghttps://gist.github.com/1626452but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks

Topic		Replies	Views
Is there is any way to explicitly use custom analyzer on rabbitmq river? Elasticsearch	3	306	July 6, 2017
How to put custom analyzer mapping on rabbitmq river? Elasticsearch	2	315	July 6, 2017
Cannot get ElasticSearch Config to Create Custom Analyzers Elasticsearch	2	352	July 6, 2017
Custom analyzer registered but not used Elasticsearch	1	353	July 6, 2017
Analyzer not found when it is custom Elasticsearch	6	1032	July 5, 2017

Unable to configure custom analyzer with rabbitmq river?

Related topics