Query and Filter

Elastic Stack Elasticsearch
1

Trying to compose a query and filter combination to no avail:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message""
}
},
"filter":{
"and":[
{
"term":{
"appId":"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b"
}
},
{
"term":{
"processId":"754311ef-d807-4bb4-8c5e-1b480fb7034f"
}
}
]
}
}
}
}

That parses fine by ES, but never returns the results. I know the two
fields are correct and in my index. If I take off the 'filter', I get the
expected results, but I need the filter to narrow the results. When I
compose the same query using Kibana, it tries to use an 'ffilter' query
which I don't see documented anywhere:

"filter": {
"bool": {
"must": [
{
"terms": {
"_type": [
"event"
]
}
},
{
"fquery": {
"query": {
"query_string": {
"query": "appId:("a32b782c-3c51-4d76-9b01-c4c1ffe53d8b")"
}
},
"_cache": true
}
}
]
}

Any pointers would be most appreciated. Pulling my hair out here.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Chances are your appId and processId fields are analyzed so it is breaking
up the id's. Update your mapping of these fields so it is not analyzed
[1]. Also, you should not use an "and" filter to combine term filters.
Use a boolean filter [2] with must clauses for better performance. Read
why at
Elasticsearch Platform — Find real-time answers at scale | Elastic.

[1]

[2]

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 12:52 PM, Matt Hughes hughes.matt@gmail.com wrote:

Trying to compose a query and filter combination to no avail:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message""
}
},
"filter":{
"and":[
{
"term":{
"appId":"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b"
}
},
{
"term":{
"processId":"754311ef-d807-4bb4-8c5e-1b480fb7034f"
}
}
]
}
}
}
}

That parses fine by ES, but never returns the results. I know the two
fields are correct and in my index. If I take off the 'filter', I get the
expected results, but I need the filter to narrow the results. When I
compose the same query using Kibana, it tries to use an 'ffilter' query
which I don't see documented anywhere:

"filter": {
"bool": {
"must": [
{
"terms": {
"_type": [
"event"
]
}
},
{
"fquery": {
"query": {
"query_string": {
"query": "appId:("a32b782c-3c51-4d76-9b01-c4c1ffe53d8b")"
}
},
"_cache": true
}
}
]
}

Any pointers would be most appreciated. Pulling my hair out here.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoBc0EmeY5yUo0juR5EUuOR%3DmuaROPbYKJJ9u7qP_-HB9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks for the quick reply!

I updated the mappings and confirmed both types read not_analyzed. I also
updated the query to use bool/must:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message from AT by user admin was
generated""
}
},
"filter":{
"bool":{
"must":[
{
"term":{
"where.appId":"12229ac6-8e9a-43ff-ab67-e80f3c585a69"
}
},
{
"term":{
"where.processId":
"bd13dbe5-0a4c-4469-a645-44cb3fde280a"
}
}
]
}
}
}
}
}

Still not getting any hits though. Tried escaping the terms. Is there
anything special about having nested field names like that
'where.processId'?

On Friday, April 18, 2014 4:07:31 PM UTC-4, Matt Weber wrote:

Chances are your appId and processId fields are analyzed so it is breaking
up the id's. Update your mapping of these fields so it is not analyzed
[1]. Also, you should not use an "and" filter to combine term filters.
Use a boolean filter [2] with must clauses for better performance. Read
why at
Elasticsearch Platform — Find real-time answers at scale | Elastic.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 12:52 PM, Matt Hughes <hughe...@gmail.com<javascript:>

wrote:

Trying to compose a query and filter combination to no avail:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message""
}
},
"filter":{
"and":[
{
"term":{
"appId":"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b"
}
},
{
"term":{
"processId":"754311ef-d807-4bb4-8c5e-1b480fb7034f"
}
}
]
}
}
}
}

That parses fine by ES, but never returns the results. I know the two
fields are correct and in my index. If I take off the 'filter', I get the
expected results, but I need the filter to narrow the results. When I
compose the same query using Kibana, it tries to use an 'ffilter' query
which I don't see documented anywhere:

"filter": {

    "bool": {
      "must": [

        {
          "terms": {

            "_type": [
              "event"

            ]
          }
        },
        {

          "fquery": {
            "query": {

              "query_string": {
                "query": "appId:(\"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b\")"

              }
            },
            "_cache": true

          }
        }
      ]
    }

Any pointers would be most appreciated. Pulling my hair out here.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/58feafb8-1110-4630-8cbd-ebfd5fef0809%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Did you reindex your docs after updating the mapping? Can you post your
mapping and original docs?

On Friday, April 18, 2014, Matt Hughes hughes.matt@gmail.com wrote:

Thanks for the quick reply!

I updated the mappings and confirmed both types read not_analyzed. I
also updated the query to use bool/must:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message from AT by user admin was
generated""
}
},
"filter":{
"bool":{
"must":[
{
"term":{
"where.appId":
"12229ac6-8e9a-43ff-ab67-e80f3c585a69"
}
},
{
"term":{
"where.processId":
"bd13dbe5-0a4c-4469-a645-44cb3fde280a"
}
}
]
}
}
}
}
}

Still not getting any hits though. Tried escaping the terms. Is there
anything special about having nested field names like that
'where.processId'?

On Friday, April 18, 2014 4:07:31 PM UTC-4, Matt Weber wrote:

Chances are your appId and processId fields are analyzed so it is
breaking up the id's. Update your mapping of these fields so it is not
analyzed [1]. Also, you should not use an "and" filter to combine term
filters. Use a boolean filter [2] with must clauses for better
performance. Read why at Elasticsearch Platform — Find real-time answers at scale | Elastic
elasticsearch-filter-bitsets/.

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/mapping-core-types.html#string
[2] Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/query-dsl-bool-filter.html

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 12:52 PM, Matt Hughes hughe...@gmail.com wrote:

Trying to compose a query and filter combination to no avail:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message""
}
},
"filter":{
"and":[
{
"term":{
"appId":"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b"
}
},
{
"term":{
"processId":"754311ef-d807-4bb4-8c5e-1b480fb7034f"
}
}
]
}
}
}
}

That parses fine by ES, but never returns the results. I know the two
fields are correct and in my index. If I take off the 'filter', I get the
expected results, but I need the filter to narrow the results. When I
compose the same query using Kibana, it tries to use an 'ffilter' query
which I don't see documented anywhere:

"filter": {

    "bool": {
      "must": [

        {
          "terms": {

            "_type": [
              "event"

            ]
          }
        },
        {

          "fquery": {
            "query"

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com<javascript:_e(%7B%7D,'cvml','elasticsearch%2Bunsubscribe@googlegroups.com');>
.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/58feafb8-1110-4630-8cbd-ebfd5fef0809%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/58feafb8-1110-4630-8cbd-ebfd5fef0809%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Thanks,
Matt Weber

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoDaNmkYnDUpb4yXqSqx1Hd%3Dg7f%2BgXi1%2BuQVRMAjfs3W5A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

Nevermind. It was an error on my part; these changes worked. Thanks again!

On Friday, April 18, 2014 5:51:31 PM UTC-4, Matt Hughes wrote:

Thanks for the quick reply!

I updated the mappings and confirmed both types read not_analyzed. I
also updated the query to use bool/must:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message from AT by user admin was
generated""
}
},
"filter":{
"bool":{
"must":[
{
"term":{
"where.appId":
"12229ac6-8e9a-43ff-ab67-e80f3c585a69"
}
},
{
"term":{
"where.processId":
"bd13dbe5-0a4c-4469-a645-44cb3fde280a"
}
}
]
}
}
}
}
}

Still not getting any hits though. Tried escaping the terms. Is there
anything special about having nested field names like that
'where.processId'?

On Friday, April 18, 2014 4:07:31 PM UTC-4, Matt Weber wrote:

Chances are your appId and processId fields are analyzed so it is
breaking up the id's. Update your mapping of these fields so it is not
analyzed [1]. Also, you should not use an "and" filter to combine term
filters. Use a boolean filter [2] with must clauses for better
performance. Read why at
Elasticsearch Platform — Find real-time answers at scale | Elastic
.

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 12:52 PM, Matt Hughes hughe...@gmail.com wrote:

Trying to compose a query and filter combination to no avail:

{
"from":0,
"size":200,
"query":{
"filtered":{
"query":{
"query_string":{
"fields":[
"_all"
],
"query":""Test message""
}
},
"filter":{
"and":[
{
"term":{
"appId":"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b"
}
},
{
"term":{
"processId":"754311ef-d807-4bb4-8c5e-1b480fb7034f"
}
}
]
}
}
}
}

That parses fine by ES, but never returns the results. I know the two
fields are correct and in my index. If I take off the 'filter', I get the
expected results, but I need the filter to narrow the results. When I
compose the same query using Kibana, it tries to use an 'ffilter' query
which I don't see documented anywhere:

"filter": {

    "bool": {
      "must": [

        {
          "terms": {

            "_type": [
              "event"

            ]
          }
        },
        {

          "fquery": {
            "query": {

              "query_string": {
                "query": "appId:(\"a32b782c-3c51-4d76-9b01-c4c1ffe53d8b\")"

              }
            },
            "_cache": true

          }
        }
      ]
    }

Any pointers would be most appreciated. Pulling my hair out here.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/666c3b42-222d-420b-9997-5b660713396d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4a88afad-971d-4d3a-8ddf-a947ff82c99d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.