Search doesn't work when String Types are used in filter

Elastic Stack Elasticsearch
1

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes", "index"
: "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" : "not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hey,

You're using a term filter here with: a*. Most likely that string literal
doesn't exist in your log_entry.text field. You want to perform a wildcard
or prefix search? If so you need to use the wildcard query and wrap the in
a query filter:

Or use the prefix filter (then you don't need to use the *):

Martijn

On 8 February 2013 09:34, mars kommineni.sateesh@gmail.com wrote:

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" :
"not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi,

Thanks for your response.

I tried the wild card as when i used the actual string it did not return
any results.

When i used the Integer Types it returned the results fine. How ever it
returned extra results which doesn't contain the value i specified in the
Query.

Thanks

On Friday, February 8, 2013 2:04:52 PM UTC+5:30, mars wrote:

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" :
"not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

The fields log_entry.text and log_entry.type are not_analyzed
meaning that no text analysis is executed and the string is indexed as is.
When using a term filter for these field, only exact matches (case
sensitive) will make a document a hit in the search result.

I don't what kind of string you put in these fields and what kind of search
experience you expect. But usually for text fields you do want analyse the
text so, that you can find documents by search with individual words. Maybe
just remove the index option for the log_entry.text field (the default
for string fields is to perform text analysis).

On 8 February 2013 17:51, mars kommineni.sateesh@gmail.com wrote:

Hi,

Thanks for your response.

I tried the wild card as when i used the actual string it did not
return any results.

When i used the Integer Types it returned the results fine. How ever it
returned extra results which doesn't contain the value i specified in the
Query.

Thanks

On Friday, February 8, 2013 2:04:52 PM UTC+5:30, mars wrote:

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" :
"not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi,

Sorry i should have posted my query and the Results.

Basically i am firing a Query with no query string ( match all Query) and
only Constraining in the Filter.

The Query i am running is:

{
"from": 1,
"size": 10,
"sort": [
{
"log_entry.reference_type": {
"order": "desc"
}
},
"_score"
],
"query": {
"match_all": {}
},
"filter": {
"term": {
"log_entry.reference_type": "10"
}
}
}

Here is the response i am getting from ES.

{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 9,
"max_score": null,
"hits": [
{
"_index": "site462",
"_type": "notes",
"_id": "1340",
"_score": 1,
"_source": {
"log_entry": {
"_id": 117,
"text": "Another Test Note",
"creation_date": "2012-03-16T09:40:31.725Z",
"modification_date": "2012-03-16T09:40:31.725Z",
"created_by": 342,
"modified_by": 342,
"reference_type": 10
}
},
"sort": [
10,
1
]
},
{
"_index": "site462",
"_type": "notes",
"_id": "1706",
"_score": 1,
"_source": {
"log_entry": {
"_id": 329,
"text": "This is Test Note 1",
"creation_date": "2012-07-04T21:57:57.665Z",
"modification_date": "2012-07-04T22:16:18.644Z",
"created_by": 363,
"modified_by": 363,
"reference_type": 10
}
},
"sort": [
10,
1
]
},
{
"_index": "site462",
"_type": "notes",
"_id": "1703",
"_score": 1,
"_source": {
"log_entry": {
"_id": 37,
"text": "some note",
"creation_date": "2011-12-28T07:29:07.715Z",
"modification_date": "2012-07-03T11:08:53.699Z",
"created_by": 342,
"modified_by": 342,
"reference_type": 70
}
},
"sort": [
10,
1
]
}
]
}
}

On Friday, February 8, 2013 2:04:52 PM UTC+5:30, mars wrote:

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" :
"not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

{
"from": 1,
"size": 10,
"sort": [
{
"log_entry.reference_type": {
"order": "desc"
}
},
"_score"
],
"query": {
"match_all": {}
},
"filter": {
"term": {
"log_entry.text": "Another Test Note"
}
}
}

If you run the above query on the same index, you should get at least one
hit. Currently because the field is not analysed you can only do exact
matches on the log_entry.text field.

Martijn

On 8 February 2013 19:45, mars kommineni.sateesh@gmail.com wrote:

Hi,

Sorry i should have posted my query and the Results.

Basically i am firing a Query with no query string ( match all Query)
and only Constraining in the Filter.

The Query i am running is:

{
"from": 1,
"size": 10,
"sort": [
{
"log_entry.reference_type": {
"order": "desc"
}
},
"_score"
],
"query": {
"match_all": {}
},
"filter": {
"term": {
"log_entry.reference_type": "10"
}
}
}

Here is the response i am getting from ES.

{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 9,
"max_score": null,
"hits": [
{
"_index": "site462",
"_type": "notes",
"_id": "1340",
"_score": 1,
"_source": {
"log_entry": {
"_id": 117,
"text": "Another Test Note",
"creation_date": "2012-03-16T09:40:31.725Z",
"modification_date": "2012-03-16T09:40:31.725Z",
"created_by": 342,
"modified_by": 342,
"reference_type": 10
}
},
"sort": [
10,
1
]
},
{
"_index": "site462",
"_type": "notes",
"_id": "1706",
"_score": 1,
"_source": {
"log_entry": {
"_id": 329,
"text": "This is Test Note 1",
"creation_date": "2012-07-04T21:57:57.665Z",
"modification_date": "2012-07-04T22:16:18.644Z",
"created_by": 363,
"modified_by": 363,
"reference_type": 10
}
},
"sort": [
10,
1
]
},
{
"_index": "site462",
"_type": "notes",
"_id": "1703",
"_score": 1,
"_source": {
"log_entry": {
"_id": 37,
"text": "some note",
"creation_date": "2011-12-28T07:29:07.715Z",
"modification_date": "2012-07-03T11:08:53.699Z",
"created_by": 342,
"modified_by": 342,
"reference_type": 70
}
},
"sort": [
10,
1
]
}
]
}
}

On Friday, February 8, 2013 2:04:52 PM UTC+5:30, mars wrote:

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" :
"not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Met vriendelijke groet,

Martijn van Groningen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Hi ,

I am not doing any search against the text field.

I am trying to get all the documents where reference_type =10 (which id
defined as inter in the mapping with index value set to not_analyzed.

Both the following Queries are returning documents that have
reference_type other than 10 !!

Query 1:

{
"from": 1,
"size": 10,
"query": {
"term": {
"log_entry.reference_type": 10
}
},
"filter": {
"term": {
"log_entry.reference_type": "10"
}
},
"facets": {
"reference_type": {
"terms": {
"field": "reference_type"
}
}
}
}

and

{
"from": 1,
"size": 10,
"query": {
match_all : { }
},
"filter": {
"term": {
"log_entry.reference_type": "10"
}
},
"facets": {
"reference_type": {
"terms": {
"field": "reference_type"
}
}
}
}

which is driving me crazy..:slight_smile:

  1. first of what's the difference between setting the term in Query when
    compared to setting it in the Filter.

Thanks

On Friday, February 8, 2013 2:04:52 PM UTC+5:30, mars wrote:

Hi All,

I am having an Issue of not getting any results when String types are
used in the Filter criteria.

Here is the mapping i am using.

"MyOrderLog" :{
"type" : "object",
"properties" : {
"log_entry":{
"properties" : {
"_id" : {"type" : "integer", "store":"yes"},
"text" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"subject" : {"type" : "string", "store":"yes",
"index" : "not_analyzed","term_vector" : "with_positions_offsets"},
"creation_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"modification_date" : {"type" : "date", "store":"yes", "index" :
"not_analyzed"},
"created_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"modified_by" : {"type" : "integer", "store":"yes", "index" :
"not_analyzed"},
"log_type" : {"type" : "string", "store":"yes", "index" :
"not_analyzed"}
}
}
}
}

And the Query i am using is

{
"from": 1,
"size": 10,
"query": {
"filtered": {
"query": {
"query_string": {
"query" : ""
}
},
"filter": {
"and": [
{
"term": {
"log_entry.text": "a"
}
}
]
}
}
}
}

I tried on "log_entry.log_type" field as well and it's the same empty
result set that's returning.

How ever when i try on integer types of Date types everything works as
expected.

Not sure what i am missing.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.