Hi,
I'm trying to use the elastic filter to get some data from a index. However the elastic filter doesn't appear to be doing anything. When starting logstash I see New ElasticSearch filter client {:hosts=>["https://.url.found.io:9243"]}
but the queried data doesn't appear. There aren't any errors in the logs either.
I had similar issues with the elastic input before, where the Docs claim elastic DSL is fully supported but in reality only very basic queries work and anything including bool filters etc. doesn't work.
config
filter {
elasticsearch {
hosts => ["https://url.found.io:9243"]
password => "pass"
user => "user"
query_template => "template.json"
fields => { "hdg" => "HEADING" }
}
}
output {
stdout {
codec => rubydebug
}
}
template (confirmed this gives the correct results in the Kibana console)
{
"query": {
"bool" : {
"filter": [
{
"range" : {
"@timestamp" : {
"lt": "%{[@timestamp]}"
}
}
}
],
"must" : {
"term" : { "ID" : "%{[ID]}" }
},
"should": [{
"exists": {
"field": "hdg"
}
},
{
"exists": {
"field": "hdg2"
}
}
],
"minimum_should_match" : 1
}
},
"size": 1,
"_source": {
"includes": [
"@timestamp",
"name",
"hdg",
"ID"]
},
"sort": [
{
"@timestamp": {
"order": "desc"
}
}
]
}
The original document I'm ingesting into Logstash contains the ID
and @timestamp
fields. How can I further troubleshoot this issue?