Yesterday we had an issue where we were getting results we did not expect
from a query that has been running for some time. The gist of the query is:
{
"query" : {
"filtered" : {
"filter" : {
"and" : [
{
"terms" : {
"customer_id" : [
932,
798
]
}
},
{
"range" : {
"event_epoch" : {
"to" : 13800451266,
"from" : 0
}
}
}
]
},
"query" : {
"match_all" : {}
}
}
},
"size" : 500
}
Upon running this query we were returning customer_id's that were not in
the terms list (835 to be exact). The mapping for the index is:
{
"events_v4" : {
"state" : "open",
"settings" : {
"index.analysis.analyzer.stem.filter.1" : "lowercase",
"index.number_of_replicas" : "2",
"index.analysis.analyzer.stem.filter.2" : "stop",
"index.analysis.analyzer.stem.filter.0" : "standard",
"index.analysis.analyzer.stem.tokenizer" : "standard",
"index.analysis.analyzer.stem.filter.3" : "porter_stem",
"index.number_of_shards" : "5",
"index.version.created" : "900199"
},
"mappings" : {
"default" : {
"dynamic_templates" : [
{
"not_analyzed_template" : {
"mapping" : {
"index" : "not_analyzed",
"type" : "string"
},
"match_mapping_type" : "string",
"match" : "_id"
}
}
],
"properties" : {}
},
"event" : {
"dynamic_templates" : [
{
"not_analyzed_template" : {
"mapping" : {
"index" : "not_analyzed",
"type" : "string"
},
"match_mapping_type" : "string",
"match" : "_id"
}
}
],
"properties" : {
"resources" : {
"index" : "not_analyzed",
"omit_norms" : true,
"store" : true,
"index_options" : "docs",
"type" : "string"
},
"resource_ids" : {
"index" : "not_analyzed",
"omit_norms" : true,
"store" : true,
"index_options" : "docs",
"type" : "string"
},
"event_category" : {
"index" : "not_analyzed",
"omit_norms" : true,
"store" : true,
"index_options" : "docs",
"type" : "string"
},
"details" : {
},
"event_type" : {
"index" : "not_analyzed",
"omit_norms" : true,
"store" : true,
"index_options" : "docs",
"type" : "string"
},
"event_id" : {
"index" : "not_analyzed",
"omit_norms" : true,
"store" : true,
"index_options" : "docs",
"type" : "string"
},
"resource_types" : {
"index" : "not_analyzed",
"omit_norms" : true,
"store" : true,
"index_options" : "docs",
"type" : "string"
},
"event_epoch" : {
"store" : true,
"type" : "long"
},
"customer_id" : {
"store" : true,
"type" : "integer"
}
}
}
},
"aliases" : ["events"]
}
}
Looking back at the problem today, we are no longer returning erroneous
results with the same query. We noted that two fixes to the problem were
replacing the AND filter with bool -> must in addition to changing the
execution mode of the terms filter to bool.
Does anyone have some insight as to why a Terms filter over an integer
mapping with a plain execution mode wrapped in an AND filter would return
results outside of the list of specified terms?
Thanks,
Matthew
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.