Search Filter for nested json array

Hi!
I am a newbie to elasticsearch.
I have a json document with the following structure
{

I am using the Java API
I am indexing this object like this

IndexResponse response = client.prepareIndex(
ES_INDEX, "discussions",
id).setSource(json_string).execute().actionGet();

When I query it like this it works fine, but it will search only in
title and text fields. It will also find only cases where searchText is
the prefix of text or title. Why?

client.prepareSearch(ES_INDEX).setTypes(type.name())
.setQuery(filteredQuery(matchAllQuery(),
andFilter(boolFilter().must(termFilter("test",
"false")),
orFilter(inFilter("title", searchText),
inFilter("text", searchText)))))
.setSearchType(
SearchType.DFS_QUERY_THEN_FETCH).setFrom(0).setSize(
limit).execute().actionGet();

I can't search in the poll text. Tried nested filter but I am getting
exception below

Exception

nested: QueryParsingException[[tokrest] [nested] nested object under
path [poll] is not of nested type]; }
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:260)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:213)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:110)
at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

Any help and code examples will be appreciated!
Tamar Fraenkel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You can't use nested filters if you did not define nested objects using a mapping first.
See: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

That said, what are you trying to achieve here?
What are you looking for?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 12 mars 2013 à 13:21, Tamar Fraenkel tamar.fraenkel@gmail.com a écrit :

Hi!
I am a newbie to elasticsearch.
I have a json document with the following structure
{
id: "45f9f13c-5234-4a72-8a08-3863deed3755",
title: "My Title",
text: "My Text",
article: {
id: "http://...",
url: null,
title: "Article Title",
excerpt: "...",
imageUrl: "http:",
sourceId: 110,
sourceName: "Source",
sourceIconUrl: "http://",
time: "2012-08-23 17:49:00 +0300",
topics: null,
},
poll: [
{
text: "Option 1",
count: 0,
index: 0
},
{
text: "Option 2",
count: 0,
index: 1
},
{
text: "Option 3",
count: 0,
index: 2
},
{
text: "Option 14",
count: 0,
index: 3
}

I am using the Java API
I am indexing this object like this

IndexResponse response = client.prepareIndex(
ES_INDEX, "discussions", id).setSource(json_string).execute().actionGet();

When I query it like this it works fine, but it will search only in title and text fields. It will also find only cases where searchText is the prefix of text or title. Why?

client.prepareSearch(ES_INDEX).setTypes(type.name())
.setQuery(filteredQuery(matchAllQuery(),
andFilter(boolFilter().must(termFilter("test", "false")),
orFilter(inFilter("title", searchText),
inFilter("text", searchText)))))
.setSearchType(
SearchType.DFS_QUERY_THEN_FETCH).setFrom(0).setSize(
limit).execute().actionGet();

I can't search in the poll text. Tried nested filter but I am getting exception below

Exception

nested: QueryParsingException[[tokrest] [nested] nested object under path [poll] is not of nested type]; }
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:260)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:213)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:110)
at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

Any help and code examples will be appreciated!
Tamar Fraenkel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi
Thanks for the reply.
I actually saw this but have no clue on how to define such mapping using
the java API.
What I am trying to achieve is search for discussion by text match to
either the title, text, or one of the votes texts.
I can probably flatten the Json and solve the issue, but I was hoping I
wouldn't have to.

Also, I am not sure why search works for some words but not others

Tamar
On Mar 12, 2013 3:59 PM, "David Pilato" david@pilato.fr wrote:

You can't use nested filters if you did not define nested objects using a
mapping first.
See: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

That said, what are you trying to achieve here?
What are you looking for?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 12 mars 2013 à 13:21, Tamar Fraenkel tamar.fraenkel@gmail.com a
écrit :

Hi!
I am a newbie to elasticsearch.
I have a json document with the following structure
{

I am using the Java API
I am indexing this object like this

IndexResponse response = client.prepareIndex(
ES_INDEX, "discussions",
id).setSource(json_string).execute().actionGet();

When I query it like this it works fine, but it will search only in
title and text fields. It will also find only cases where searchText
is the prefix of text or title. Why?

client.prepareSearch(ES_INDEX).setTypes(type.name())
.setQuery(filteredQuery(matchAllQuery(),
andFilter(boolFilter().must(termFilter("test",
"false")),
orFilter(inFilter("title", searchText),
inFilter("text", searchText)))))
.setSearchType(

SearchType.DFS_QUERY_THEN_FETCH).setFrom(0).setSize(
limit).execute().actionGet();

I can't search in the poll text. Tried nested filter but I am getting
exception below

Exception

nested: QueryParsingException[[tokrest] [nested] nested object under
path [poll] is not of nested type]; }
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:260)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:213)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:110)
at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

Any help and code examples will be appreciated!
Tamar Fraenkel

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/S_SQpepmw2M/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Just search in poll.text, title, text fields.
It should work and you probably don't need nested docs here.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 12 mars 2013 à 18:59, Tamar Fraenkel tamar.fraenkel@gmail.com a écrit :

Hi
Thanks for the reply.
I actually saw this but have no clue on how to define such mapping using the java API.
What I am trying to achieve is search for discussion by text match to either the title, text, or one of the votes texts.
I can probably flatten the Json and solve the issue, but I was hoping I wouldn't have to.

Also, I am not sure why search works for some words but not others

Tamar

On Mar 12, 2013 3:59 PM, "David Pilato" david@pilato.fr wrote:
You can't use nested filters if you did not define nested objects using a mapping first.
See: http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

That said, what are you trying to achieve here?
What are you looking for?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 12 mars 2013 à 13:21, Tamar Fraenkel tamar.fraenkel@gmail.com a écrit :

Hi!
I am a newbie to elasticsearch.
I have a json document with the following structure
{
id: "45f9f13c-5234-4a72-8a08-3863deed3755",
title: "My Title",
text: "My Text",
article:
{
id: "http://...",
url: null,
title: "Article Title",
excerpt: "...",
imageUrl: "http:",
sourceId: 110,
sourceName: "Source",
sourceIconUrl: "http://",
time: "2012-08-23 17:49:00 +0300",
topics: null,
},
poll:
[
{
text: "Option 1",
count: 0,
index: 0
},
{
text: "Option 2",
count: 0,
index: 1
},
{
text: "Option 3",
count: 0,
index: 2
},
{
text: "Option 14",
count: 0,
index: 3
}

I am using the Java API
I am indexing this object like this

IndexResponse response = client.prepareIndex(
ES_INDEX, "discussions", id).setSource(json_string).execute().actionGet();

When I query it like this it works fine, but it will search only in title and text fields. It will also find only cases where searchText is the prefix of text or title. Why?

client.prepareSearch(ES_INDEX).setTypes(type.name())
.setQuery(filteredQuery(matchAllQuery(),
andFilter(boolFilter().must(termFilter("test", "false")),
orFilter(inFilter("title", searchText),
inFilter("text", searchText)))))
.setSearchType(
SearchType.DFS_QUERY_THEN_FETCH).setFrom(0).setSize(
limit).execute().actionGet();

I can't search in the poll text. Tried nested filter but I am getting exception below

Exception

nested: QueryParsingException[[tokrest] [nested] nested object under path [poll] is not of nested type]; }
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:260)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:213)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:110)
at org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

Any help and code examples will be appreciated!
Tamar Fraenkel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/S_SQpepmw2M/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi!
Managed to make it work:

Before indexing I defined a mapping:
XContentBuilder mappingBuilder = jsonBuilder().
startObject() //all
.startObject(IndexType.discussions.name()) //type
.startObject("properties") //properties
.startObject("title").field("type", "string").endObject()
.startObject("text").field("type", "string").endObject()
.startObject("poll").field("type", "nested").endObject()
.endObject() //properties
.endObject() //type
.endObject(); //all

CreateIndexResponse createIndexResponse =

client.admin().indices().prepareCreate(index_name).addMapping(
index_type, mappingBuilder).execute()
.actionGet();

When indexing:
XContentBuilder builder = jsonBuilder().startObject()
.field("title", getTitle())
.field("text", getText())
.startArray("poll");
for (PollOption poll : getPollList()) {
builder = builder.startObject().field("text", poll.getText()).endObject();
}
builder = builder.endArray().endObject();
IndexResponse response = client.prepareIndex(
index_name, index_type, getId()).setSource(
builder).execute().actionGet();

When Searching:
SearchRequestBuilder srqb =
client.prepareSearch(index_name).setTypes(index_type).
setQuery(filteredQuery(matchAllQuery(),
orFilter(queryFilter(fieldQuery("title", searchText)),
queryFilter(fieldQuery("text", searchText)),
nestedFilter("poll",
fieldQuery("poll.text", searchText)))));

Hope this will help someone.

Tamar

On Wed, Mar 13, 2013 at 10:41 AM, David Pilato david@pilato.fr wrote:

Just search in poll.text, title, text fields.
It should work and you probably don't need nested docs here.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 12 mars 2013 à 18:59, Tamar Fraenkel tamar.fraenkel@gmail.com a
écrit :

Hi
Thanks for the reply.
I actually saw this but have no clue on how to define such mapping using
the java API.
What I am trying to achieve is search for discussion by text match to
either the title, text, or one of the votes texts.
I can probably flatten the Json and solve the issue, but I was hoping I
wouldn't have to.

Also, I am not sure why search works for some words but not others

Tamar
On Mar 12, 2013 3:59 PM, "David Pilato" david@pilato.fr wrote:

You can't use nested filters if you did not define nested objects using a
mapping first.
See:
http://www.elasticsearch.org/guide/reference/mapping/nested-type.html

That said, what are you trying to achieve here?
What are you looking for?

--
David Pilato | Technical Advocate | *Elasticsearch.comhttp://elasticsearch.com/
*
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 12 mars 2013 à 13:21, Tamar Fraenkel tamar.fraenkel@gmail.com a
écrit :

Hi!
I am a newbie to elasticsearch.
I have a json document with the following structure
{

I am using the Java API
I am indexing this object like this

IndexResponse response = client.prepareIndex(
ES_INDEX, "discussions",
id).setSource(json_string).execute().actionGet();

When I query it like this it works fine, but it will search only in
title and text fields. It will also find only cases where searchText
is the prefix of text or title. Why?

client.prepareSearch(ES_INDEX).setTypes(type.name())
.setQuery(filteredQuery(matchAllQuery(),
andFilter(boolFilter().must(termFilter("test",
"false")),
orFilter(inFilter("title", searchText),
inFilter("text", searchText)))))
.setSearchType(

SearchType.DFS_QUERY_THEN_FETCH).setFrom(0).setSize(
limit).execute().actionGet();

I can't search in the poll text. Tried nested filter but I am getting
exception below

Exception

nested: QueryParsingException[[tokrest] [nested] nested object under
path [poll] is not of nested type]; }
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:260)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:213)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteDfs(SearchServiceTransportAction.java:110)
at
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchDfsQueryThenFetchAction.java:86)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

Any help and code examples will be appreciated!
Tamar Fraenkel

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/S_SQpepmw2M/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/S_SQpepmw2M/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Tamar Fraenkel
8 Yitzhak Shalev St., Apt. 2
Jerusalem, Israel
Home: +972-2-6500435
Cell: +972-54-8356490

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.