Date filter is not working in kibana using DSL query

[irobot678 (meNotHuman)]

HI ,

i am trying to a write DSL query for particular date range and its not working and i dont know why ?

i have Time (kibana uploaded time) , message and date (the log generated date) as shown in the below image .

now i want to write a DSL query to a particular date range

i tried with the below code its not working

{
"query": {
"range": {
"date": {
"gte": "2019-04-12 17:48:40,579",
"lt": "2019-04-12 17:48:48,247"
}
}
}
}

and if i write a filter to display logs only to particular date its working as shown in below image, which says the date field is valid

query :

output :

so someone please guide me how to write query that works to display logs of particular date range .

[tylersmalley]

What version of Kibana is this for?

Can you tell me more about the date field you're wanting to filter by? Is it not the configured Time Filter field for the index pattern? If so, you can just use the timepicker in the top right which is currently configured for "This week"

[irobot678 (meNotHuman)]

HI Tyler,

My kibana version is 6.6.2

The date field we are talking about its a date inside log message

we used grok filter which extracts the date from log message as a separate field

logstash.conf
input {
file {
path => "/home/*.log"
start_position => "beginning"
}
}
filter {
grok {
match => ["message","%{TIMESTAMP_ISO8601:date} [%{LOGLEVEL:log-level}] %{DATA:component} %{DATA:class} %{DATA:method} %{DATA:imei} %{DATA:token} %{GREEDYDATA:message}"]
}
}
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}

sample log :
2019-04-29 13:05:31:663 [INFO] DTLS-Connection-Handler-64 o.e.l.s.b.d.BootstrapRedisSecurityStoreImpl getByIdentity Params - identity:7ce2471eced27dee9f3761df8123ba15

output :
{
"method" => "getByIdentity",
"path" => "/home/logs/b.log",
"class" => "o.e.l.s.b.d.BootstrapRedisSecurityStoreImpl",
"host" => "qolsys-desktop",
"@timestamp" => 2019-04-30T11:10:02.512Z,
"tags" => [
[0] "_grokparsefailure"
],

"date" => "2019-04-29 13:05:31:663",

"message" => [
[0] "2019-04-29 13:05:31:663 [INFO] DTLS-Connection-Handler-64 o.e.l.s.b.d.BootstrapRedisSecurityStoreImpl getByIdentity Params - identity:7ce2471eced27dee9f3761df8123ba15",
[1] "Params - identity:7ce2471eced27dee9f3761df8123ba15"
],
"log-level" => "INFO",
"@version" => "1",
"component" => "DTLS-Connection-Handler-64"
}

The output shows date field now i want to write an dsl query to display all the logs for a particular range of date field

the timepicker is used to sort by logs by uploaded time so it wont be useful

in my case log generated time is in date field and log uploaded time by kibana is in @timestamp and all i want to do filter out the date field .

[tylersmalley]

Is there a reason you wouldn't want the time picker to filter by the timestamp of the actual log message and not the time it was ingested into ES? It seems like this would make more sense in which case you just need to instead extract the timestamp as @timestamp and not date.

[tylersmalley]

If you want to keep your current setup you can use lucene query syntax to search for the date range in the query bar:

date:[2019-04-12T17:48:40,579Z TO 2019-04-12T17:48:48,247Z]

[irobot678 (meNotHuman)]

HI tyler ,

i tried this
date:[2019-04-12T17:48:40,579Z TO 2019-04-12T17:48:48,247Z]
but its not working and its showing a following error

Discover: Expected "*", "", "\n", "\r", "\t", [\ \t\r\n] or end of input but ":" found.

Error: Expected "*", "\", "\n", "\r", "\t", [\ \t\r\n] or end of input but ":" found.
KbnError@http://localhost:5601/bundles/commons.bundle.js:3:10413
_flatten/<@http://localhost:5601/bundles/commons.bundle.js:3:1247185
processQueue@http://localhost:5601/dlls/vendors.bundle.dll.js:434:199687
scheduleProcessQueue/<@http://localhost:5601/dlls/vendors.bundle.dll.js:434:200650
$digest@http://localhost:5601/dlls/vendors.bundle.dll.js:434:210412
$evalAsync/<@http://localhost:5601/dlls/vendors.bundle.dll.js:434:212933
completeOutstandingRequest@http://localhost:5601/dlls/vendors.bundle.dll.js:434:64422
Browser/self.defer/timeoutId<@http://localhost:5601/dlls/vendors.bundle.dll.js:434:67267