hi,
We're in the process of moving from the logstash builtin template to our own in order to have more control of our fields and data types. I'm seeing some problems with wildcard searches using our new index that I'd very much like to some expert advise on.
Firstly, our es template looks like this:
{
"company_app_log": {
"order": 0,
"template": "company_app_log*",
"settings": {
"index": {
"refresh_interval": "5s"
}
},
"mappings": {
"formattederror": {
"dynamic_templates": [
{
"notanalyzed": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string",
"match": "*"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
},
"errorid": {
"type": "integer"
}
}
},
"telemetry": {
"dynamic_templates": [
{
"notanalyzed": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string",
"match": "*"
}
}
],
"properties": {
"@timestamp": {
"type": "date"
}
}
},
"http_measurement": {
"dynamic_templates": [
{
"notanalyzed": {
"mapping": {
"index": "not_analyzed",
"type": "string"
},
"match_mapping_type": "string",
"match": "*"
}
}
],
"properties": {
"httpstatuscode": {
"type": "integer"
},
"@timestamp": {
"type": "date"
},
"elapsedmilliseconds": {
"type": "integer"
}
}
}
},
"aliases": {}
}
}
One document in the index looks like this:
{
"_index": "company_log-uat-2016.11",
"_type": "http_measurement",
"_id": "AVhTY6gacmfALXNtkch5",
"_score": null,
"_source": {
"Level": "Information",
"@version": "1",
"@timestamp": "2016-11-11T11:42:48.481Z",
"count": 1,
"fields": null,
"beat": {
"hostname": "Z49OS2SWB114T",
"name": "Z49OS2SWB114T"
},
"input_type": "log",
"tags": [
"environment:uat",
"servicefamily:services-external",
"servicegroup:api",
"beats_input_codec_json_applied",
"http_measurement"
],
"offset": 3615,
"type": "log",
"host": "Z49OS2SWB114T",
"logtype": "http_measurement",
"computername": "Z49OS2SWB114T",
"httpmethod": "GET",
"requesturi": "https://company.com/api/web/asset/1595262/play?protocol=HLS",
"httpstatusstring": "OK",
"httpstatuscode": 200,
"elapsedmilliseconds": 221
}
}
I'm struggling to understand how I can wildcard search in Kibana using this data.
In regular "logstashed" indices I could for eample use this query:
requesturi.raw: "*web*"
but the corresponding
requesturi: "*web*"
doesnt provide any results from this index. If I remove the quotation marks it works, but that leaves me unable to search using any terms containing special characters, such as:
requesturi: "*api/web/asset/*"
Am I doing something wrong in my template?
For the record, we're not on ELK 5.0, we're still running the previous version of the stack.