Querystring filter dosn't work

Elastic Stack Elasticsearch
1

Hi,

I have some issue with a filtered query. If I use the query:
"query": {
"query_string": {
"query": "uri:video\-ondemand/video/flv/test/*"
}
}

I get the results as exspected. But if I use it as a filtered query

{
"query":{
"filtered": {
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "very complex string"
}
}
]
}
},
"_cache": "false",
"filter": {
"bool": {
"must": [
{
"fquery": {
"query": {
"query_string": {
"query": "uri:video\-ondemand/video/flv/test/*"
}
}
}
},
{
"range": {
"@timestamp": {
"from": "2014-12-01T00:00:00.000+01",
"to": "2014-12-01T23:59:59.999+01"
}
}
}
]
}
}
}
},
"sort": {"@timestamp": {"order": "asc"}}
}

I get results with other values in uri: example
"video-ondemand/event/mp4/....."
If I move the uri query in the first query string (addes AND
uri:video\-ondemand/video/flv/test/*) then I get no results.
The uri is an not_analyzed string.

Anyone an idea why it dosnt work?

best regards
Messias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/35c4d535-2cf3-4eb5-8e0c-4042b7c71694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

I have no idea but for sure, I'd prefer using a term filter on uri field and I would not set _cache.
But may be you could GIST a full example we can reproduce?

David

Le 26 janv. 2015 à 08:47, Messias torsten.schubert@gl-systemhaus.de a écrit :

Hi,

I have some issue with a filtered query. If I use the query:
"query": {
"query_string": {
"query": "uri:video\-ondemand/video/flv/test/*"
}
}

I get the results as exspected. But if I use it as a filtered query

{
"query":{
"filtered": {
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "very complex string"
}
}
]
}
},
"_cache": "false",
"filter": {
"bool": {
"must": [
{
"fquery": {
"query": {
"query_string": {
"query": "uri:video\-ondemand/video/flv/test/*"
}
}
}
},
{
"range": {
"@timestamp": {
"from": "2014-12-01T00:00:00.000+01",
"to": "2014-12-01T23:59:59.999+01"
}
}
}
]
}
}
}
},
"sort": {"@timestamp": {"order": "asc"}}
}

I get results with other values in uri: example "video-ondemand/event/mp4/....."
If I move the uri query in the first query string (addes AND uri:video\-ondemand/video/flv/test/*) then I get no results.
The uri is an not_analyzed string.

Anyone an idea why it dosnt work?

best regards
Messias

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/35c4d535-2cf3-4eb5-8e0c-4042b7c71694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/31249962-CA2B-41DD-90B5-F2827255215B%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

3

So here more details:

{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "uri:video\-ondemand/video/flv/test/" # one thing with
video-ondemand I get an Error ( nested: JsonParseException[Unrecognized
character escape '-' (code 45))
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"from": "2014-12-01T00:00:00.001",
"to": "2014-12-01T23:59:59.999"
}
}
}
]
}
}
}
},
"size": 10
}

Get results like this:
{
"took": 12,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 4,
"failed": 0
},
"hits": {
"total": 81189,
"max_score": 0.13291985,
"hits": [
{
"_index": "test",
"_type": "log",
"_id": "idstring",
"_score": 0.13291985,
"_source": {
"@version": "1",
"@timestamp": "2014-12-01T20:26:17.000Z",
"type": "log",
"tags": [
"original"
],
"timestamp": "2014-12-01 20:26:17",
"sc_status": 0,
"cs_method": "GET",
"uri": "/video-ondemand/video/flv/test/2014/some.mp4",
"name": "Other",
"os": "Other",
"os_name": "Other",
"device": "HTC Streaming P"
}
},
{
"_index": "test",
"_type": "log",
"_id": "docid",
"_score": 0.13291954,
"_source": {
"@version": "1",
"@timestamp": "2014-12-01T20:39:06.000Z",
"type": "log",
"tags": [
"original"
],
"timestamp": "2014-12-01 20:39:06",
"uri":
"/video-ondemand/video/flv/test/2014/another.mp4",
"name": "Other",
"os": "Other",
"sc_status": 404,
"cs_method": "GET",
"os_name": "Other",
"device": "Other"
}
},
{
"_index": "test",
"_type": "log",
"_id": "docid",
"_score": 0.13291954,
"_source": {
"@version": "1",
"@timestamp": "2014-12-01T20:39:07.000Z",
"type": "log",
"tags": [
"original"
],
"timestamp": "2014-12-01 20:39:07",
"uri": "/video-ondemand/video/flv/test/2014/super.mp4",
"sc_status": 0,
"cs_method": "GET",
"name": "Other",
"os": "Other",
"os_name": "Other",
"device": "Other"
}
},
.......

with follow query:
{
"query":{
"filtered": {
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "(sc_status:206 OR
sc_status:200 OR sc_status:0) AND cs_method:GET AND cs_uri:.mp4"
}
}
]
}
},
"_cache": "false",
"filter": {
"bool": {
"must": [
{
"fquery": {
"query": {
"query_string": {
"query":
"cs_uri:video\-ondemand/video/flv/test/*"
}
}
}
},
{
"range": {
"@timestamp": {
"from":
"2014-12-01T00:00:00.000+01",
"to":
"2014-12-01T23:59:59.999+01"
}
}
}
]
}
}
}
},
"sort": {"@timestamp": {"order": "asc"}}
}

I get follow results:
{
"took": 492,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 4,
"failed": 0
},
"hits": {
"total": 213476,
"max_score": null,
"hits": [
{
"_index": "test",
"_type": "log",
"_id": "docid",
"_score": null,
"_source": {
"@version": "1",
"@timestamp": "2014-11-30T23:00:00.000Z",
"type": "log",
"tags": [
"original"
],
"timestamp": "2014-11-30 23:00:00",
"cs_method": "GET",
"cs_uri": "/video-ondemand/event/mp4/filename.mp4",
"sc_status": 206,
"name": "Mobile Safari",
"os": "iOS 5.1.1",
"os_name": "iOS",
"os_major": "5",
"os_minor": "1",
"device": "iPad",
"browser": "Mobile Safari %{major}"
},
"sort": [
1417388400000
]
},
{
"_index": "test",
"_type": "log",
"_id": "docid",
"_score": null,
"_source": {
"@version": "1",
"@timestamp": "2014-11-30T23:00:01.000Z",
"type": "log",
"tags": [
"original"
],
"timestamp": "2014-11-30 23:00:01",
"cs_method": "GET",
"cs_uri": "/video-ondemand/event/mp4/123/file.mp4",
"sc_status": 0,
"name": "Mobile Safari",
"os": "iOS 5.1.1",
"os_name": "iOS",
"os_major": "5",
"os_minor": "1",
"device": "iPad",
"browser": "Mobile Safari %{major}"
},
"sort": [
1417388401000
]
},
{
"_index": "test",
"_type": "log",
"_id": "docid",
"_score": null,
"_source": {
"@version": "1",
"@timestamp": "2014-11-30T23:00:01.000Z",
"type": "log",
"tags": [
"original"
],
"timestamp": "2014-11-30 23:00:01",
"cs_method": "GET",
"cs_uri": "/video-ondemand/events/mp4/23/file.mp4",
"sc_status": 206,
"name": "Mobile Safari",
"os": "iOS 8.1.1",
"os_name": "iOS",
"os_major": "8",
"os_minor": "1",
"device": "iPhone",
"browser": "Mobile Safari %{major}"
},
"sort": [
1417388401000
]
},
....

the results with events shouldn't be in.

regards
Messias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/564148e2-853b-4c3c-a695-d91ec6196548%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

I have done test and found out following:

{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "uri:video\-ondemand/video/flv/test/"
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"from": "2014-12-01T00:00:00.001",
"to": "2014-12-01T23:59:59.999"
}
}
}
]
}
}
}
},
"size": 100
}

"total": 81189

{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "uri:video-ondemand/video/flv/test/" ## without escape
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"from": "2014-12-01T00:00:00.001",
"to": "2014-12-01T23:59:59.999"
}
}
}
]
}
}
}
},
"size": 100
}

"total": 81189

{
"query":{
"filtered": {
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "(sc_status:206 OR sc_status:200 OR sc_status:0) AND cs_method:GET
AND uri:.mp4"
}
}
]
}
},
"_cache": "false",
"filter": {
"bool": {
"must": [
{
"fquery": {
"query": {
"query_string": {
"query": "uri:video-ondemand/video/flv/test/*"
}
}
}
},
{
"range": {
"@timestamp": {
"from": "2014-12-01T00:00:00.000+01",
"to": "2014-12-01T23:59:59.999+01"
}
}
}
]
}
}
}
},
"sort": {"@timestamp": {"order": "asc"}}
}
"total": 216739

the last one should have less than the first 2 (they have sc_status:404).
But I dont know why.

best regards
Messias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a74bd4ef-1698-4926-836a-6cf47111ebe5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Now I have found the right query:

You have to double escape the reserved characters.

e.g. "uri:\/video\-ondemand\/video\/flv\/test\/*" with this query all
works as expected,

Best regards
Messias

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7f1f967e-2e0c-4094-a9ad-72cbd50de7fe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.