How to add filter on searching data on index using java?


(Sam-4) #1

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

        QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =
client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchType.DEFAULT)
.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().actionGet();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(Shay Banon) #2

You don't define it on the query, you define it in the index configuration
with a custom analyzer, and then use that analyzer either by making it the
default one, or specifically applying it in the mapping to specific fields.

On Mon, Jan 16, 2012 at 8:18 AM, sam mishra.sameek@gmail.com wrote:

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

https://gist.github.com/1619313

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

       QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =

client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchType.DEFAULT)

.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().actionGet();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(Sam-4) #3

thanks for reply.could u explain how to define index configuration
with a custom analyzer and also how to make it default one. i have
tried but i did not found any document how to define custom analyzer
with rabbitmq river and make it default.please refer any document or
solution,which help me.

On Jan 17, 2:24 pm, Shay Banon kim...@gmail.com wrote:

You don't define it on the query, you define it in the index configuration
with a custom analyzer, and then use that analyzer either by making it the
default one, or specifically applying it in the mapping to specific fields.

On Mon, Jan 16, 2012 at 8:18 AM, sam mishra.sam...@gmail.com wrote:

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

https://gist.github.com/1619313

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

       QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =

client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchTyp e.DEFAULT)

.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().action Get();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(Shay Banon) #4

When you define the rabbitmq river, you define which index it is going to
index into. Before you define the river, you will need to make sure to
create that index with the relevant mappings you plan to use. Default
analyzer can be defined simply by naming it "default".

Analysis Docs:
http://www.elasticsearch.org/guide/reference/index-modules/analysis/.
Create Index API (with mapping):
http://www.elasticsearch.org/guide/reference/api/admin-indices-create-index.html
.
Associating an analyzer with a field:
http://www.elasticsearch.org/guide/reference/mapping/core-types.html.

On Tue, Jan 17, 2012 at 12:58 PM, sam mishra.sameek@gmail.com wrote:

thanks for reply.could u explain how to define index configuration
with a custom analyzer and also how to make it default one. i have
tried but i did not found any document how to define custom analyzer
with rabbitmq river and make it default.please refer any document or
solution,which help me.

On Jan 17, 2:24 pm, Shay Banon kim...@gmail.com wrote:

You don't define it on the query, you define it in the index
configuration
with a custom analyzer, and then use that analyzer either by making it
the
default one, or specifically applying it in the mapping to specific
fields.

On Mon, Jan 16, 2012 at 8:18 AM, sam mishra.sam...@gmail.com wrote:

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

https://gist.github.com/1619313

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

       QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =

client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchTyp
e.DEFAULT)

.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().action
Get();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(Sam-4) #5

Thanks for quick reply. i have created gist for used mapping:

https://gist.github.com/1626452

when i try to execute it got following exception:

org.elasticsearch.common.jackson.JsonParseException: Unexpected
character ('"' (code 34)): was expecting comma to separate OBJECT
entries
at [Source: [B@1171e30; line: 1, column: 336]
at
org.elasticsearch.common.jackson.JsonParser._constructError(JsonParser.java:
1291)
at
org.elasticsearch.common.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:
385)
at
org.elasticsearch.common.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:
306)
at
org.elasticsearch.common.jackson.impl.Utf8StreamParser.nextToken(Utf8StreamParser.java:
312)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:
46)
at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:
121)
at
org.elasticsearch.client.action.bulk.BulkRequestBuilder.add(BulkRequestBuilder.java:
84)
at org.elasticsearch.river.rabbitmq.RabbitmqRiver
$Consumer.run(RabbitmqRiver.java:216)
at java.lang.Thread.run(Thread.java:662)

please help where the problem in my mapping.

Thanks

On Jan 17, 4:29 pm, Shay Banon kim...@gmail.com wrote:

When you define the rabbitmq river, you define which index it is going to
index into. Before you define the river, you will need to make sure to
create that index with the relevant mappings you plan to use. Default
analyzer can be defined simply by naming it "default".

Analysis Docs:http://www.elasticsearch.org/guide/reference/index-modules/analysis/.
Create Index API (with mapping):http://www.elasticsearch.org/guide/reference/api/admin-indices-create...
.
Associating an analyzer with a field:http://www.elasticsearch.org/guide/reference/mapping/core-types.html.

On Tue, Jan 17, 2012 at 12:58 PM, sam mishra.sam...@gmail.com wrote:

thanks for reply.could u explain how to define index configuration
with a custom analyzer and also how to make it default one. i have
tried but i did not found any document how to define custom analyzer
with rabbitmq river and make it default.please refer any document or
solution,which help me.

On Jan 17, 2:24 pm, Shay Banon kim...@gmail.com wrote:

You don't define it on the query, you define it in the index
configuration
with a custom analyzer, and then use that analyzer either by making it
the
default one, or specifically applying it in the mapping to specific
fields.

On Mon, Jan 16, 2012 at 8:18 AM, sam mishra.sam...@gmail.com wrote:

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

https://gist.github.com/1619313

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

       QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =

client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchTyp
e.DEFAULT)

.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().action
Get();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(Sam-4) #6

I resolved the above syntax error and now it is not giving me any
exception but again the mapping is not working as expected.

Tested Output:
I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".
This made me sure that the custom analyzer is not configured
correctly.

I updated the above mapping and now I am using this mapping
https://gist.github.com/1626452 but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks
On Jan 17, 5:15 pm, sam mishra.sam...@gmail.com wrote:

Thanks for quick reply. i have created gist for used mapping:

https://gist.github.com/1626452

when i try to execute it got following exception:

org.elasticsearch.common.jackson.JsonParseException: Unexpected
character ('"' (code 34)): was expecting comma to separate OBJECT
entries
at [Source: [B@1171e30; line: 1, column: 336]
at
org.elasticsearch.common.jackson.JsonParser._constructError(JsonParser.java :
1291)
at
org.elasticsearch.common.jackson.impl.JsonParserMinimalBase._reportError(Js onParserMinimalBase.java:
385)
at
org.elasticsearch.common.jackson.impl.JsonParserMinimalBase._reportUnexpect edChar(JsonParserMinimalBase.java:
306)
at
org.elasticsearch.common.jackson.impl.Utf8StreamParser.nextToken(Utf8Stream Parser.java:
312)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXCo ntentParser.java:
46)
at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:
121)
at
org.elasticsearch.client.action.bulk.BulkRequestBuilder.add(BulkRequestBuil der.java:
84)
at org.elasticsearch.river.rabbitmq.RabbitmqRiver
$Consumer.run(RabbitmqRiver.java:216)
at java.lang.Thread.run(Thread.java:662)

please help where the problem in my mapping.

Thanks

On Jan 17, 4:29 pm, Shay Banon kim...@gmail.com wrote:

When you define the rabbitmq river, you define which index it is going to
index into. Before you define the river, you will need to make sure to
create that index with the relevant mappings you plan to use. Default
analyzer can be defined simply by naming it "default".

Analysis Docs:http://www.elasticsearch.org/guide/reference/index-modules/analysis/.
Create Index API (with mapping):http://www.elasticsearch.org/guide/reference/api/admin-indices-create...
.
Associating an analyzer with a field:http://www.elasticsearch.org/guide/reference/mapping/core-types.html.

On Tue, Jan 17, 2012 at 12:58 PM, sam mishra.sam...@gmail.com wrote:

thanks for reply.could u explain how to define index configuration
with a custom analyzer and also how to make it default one. i have
tried but i did not found any document how to define custom analyzer
with rabbitmq river and make it default.please refer any document or
solution,which help me.

On Jan 17, 2:24 pm, Shay Banon kim...@gmail.com wrote:

You don't define it on the query, you define it in the index
configuration
with a custom analyzer, and then use that analyzer either by making it
the
default one, or specifically applying it in the mapping to specific
fields.

On Mon, Jan 16, 2012 at 8:18 AM, sam mishra.sam...@gmail.com wrote:

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

https://gist.github.com/1619313

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

       QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =

client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchTyp
e.DEFAULT)

.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().action
Get();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(Sam-4) #7

It is disappointing to me.i didn't find mapping details when define
custom analyzer with rabbitmq,but i did not getting any helpful
response from your side.elastic provided documentation is quite
confusing. please any one help me to get correct mapping for both
index_analyzer and search_analyzer.i not able to find correct mapping.

On Jan 18, 10:47 am, sam mishra.sam...@gmail.com wrote:

I resolved the above syntax error and now it is not giving me any
exception but again the mapping is not working as expected.

Tested Output:
I included "inc" in the stop words file but when I searched "inc" it
should give me 0 hits as the stop word filter, filters the stop words
from the index but I am getting 1 hit as the text I have indexed
contain "inc".
This made me sure that the custom analyzer is not configured
correctly.

I updated the above mapping and now I am using this mappinghttps://gist.github.com/1626452but I am not sure
that this is completely correct.Please tell me where I am going wrong
and what changes are needed to be made.

Thanks
On Jan 17, 5:15 pm, sam mishra.sam...@gmail.com wrote:

Thanks for quick reply. i have created gist for used mapping:

https://gist.github.com/1626452

when i try to execute it got following exception:

org.elasticsearch.common.jackson.JsonParseException: Unexpected
character ('"' (code 34)): was expecting comma to separate OBJECT
entries
at [Source: [B@1171e30; line: 1, column: 336]
at
org.elasticsearch.common.jackson.JsonParser._constructError(JsonParser.java :
1291)
at
org.elasticsearch.common.jackson.impl.JsonParserMinimalBase._reportError(Js onParserMinimalBase.java:
385)
at
org.elasticsearch.common.jackson.impl.JsonParserMinimalBase._reportUnexpect edChar(JsonParserMinimalBase.java:
306)
at
org.elasticsearch.common.jackson.impl.Utf8StreamParser.nextToken(Utf8Stream Parser.java:
312)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXCo ntentParser.java:
46)
at org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:
121)
at
org.elasticsearch.client.action.bulk.BulkRequestBuilder.add(BulkRequestBuil der.java:
84)
at org.elasticsearch.river.rabbitmq.RabbitmqRiver
$Consumer.run(RabbitmqRiver.java:216)
at java.lang.Thread.run(Thread.java:662)

please help where the problem in my mapping.

Thanks

On Jan 17, 4:29 pm, Shay Banon kim...@gmail.com wrote:

When you define the rabbitmq river, you define which index it is going to
index into. Before you define the river, you will need to make sure to
create that index with the relevant mappings you plan to use. Default
analyzer can be defined simply by naming it "default".

Analysis Docs:http://www.elasticsearch.org/guide/reference/index-modules/analysis/.
Create Index API (with mapping):http://www.elasticsearch.org/guide/reference/api/admin-indices-create...
.
Associating an analyzer with a field:http://www.elasticsearch.org/guide/reference/mapping/core-types.html.

On Tue, Jan 17, 2012 at 12:58 PM, sam mishra.sam...@gmail.com wrote:

thanks for reply.could u explain how to define index configuration
with a custom analyzer and also how to make it default one. i have
tried but i did not found any document how to define custom analyzer
with rabbitmq river and make it default.please refer any document or
solution,which help me.

On Jan 17, 2:24 pm, Shay Banon kim...@gmail.com wrote:

You don't define it on the query, you define it in the index
configuration
with a custom analyzer, and then use that analyzer either by making it
the
default one, or specifically applying it in the mapping to specific
fields.

On Mon, Jan 16, 2012 at 8:18 AM, sam mishra.sam...@gmail.com wrote:

Hi

I have used following mapping to define custom analyzer at index level
used mapping:

https://gist.github.com/1619313

I am using following java code to search data:
String indexName="test1";
String indexType="type1";
String data="is";

       QueryStringQueryBuilder querystring = new

QueryStringQueryBuilder(data);
SearchResponse searchResponse =

client.prepareSearch(indexName).setTypes(indexType).setSearchType(SearchTyp
e.DEFAULT)

.setQuery(querystring.analyzer("whitespace")).setSize(100).execute().action
Get();

I have question to you how to define stop word filter and
stopwords_path setting on this search query.please help me.

Thanks


(system) #8