Hi,
i have recently started using elasticsearch and am completely lost on how to sort the documents based on a field value for which I have defined the type as "date" and a custom format as "yyyy-MM-dd HH:mm:ss -HHmm". I have define the index mapping at time of index creation as :
curl -XPOST localhost:9200/myindex -d '{
"mappings" : {
"type1" : {
"properties": {
"logtime": {
"type": "date",
"format":"yyyy-MM-dd HH:mm:ss -HHmm",
"index" : "not_analyzed"
},
"hostname": {
"type": "string"
},
"level": {
"type": "string"
},
"log": {
"type": "string"
},
"user": {
"type": "string"
}
}
}
}
}'
I inserted few documenst as :
{"hostname" : "node1", "level" : "info", "log" : "this is dummy log1", "logtime" : "2016-05-01 02:05:06 -0500", "user" : "bdmin"}
{"hostname" : "node1", "level" : "info", "log" : "this is dummy log1", "logtime" : "2016-05-01 02:03:06 -0500", "user" : "admin"}
{"hostname" : "node1", "level" : "info", "log" : "this is dummy log1", "logtime" : "2016-05-01 02:07:06 -0500", "user" : "gdmin"}
{"hostname" : "node1", "level" : "info", "log" : "this is dummy log1", "logtime" : "2016-05-01 02:01:06 -0500", "user" : "cdmin"}
When I do a search of docs and sort on field "logtime", the results are not sorted and the score value suggests as if all docs were inserted with same timestamp which is not the one contained in logtime.
Query:
curl -XGET "localhost:9200/myindex/type1/_search?sort=logtime:desc&fields=logtime,user&pretty"
Result:
"fields" : { "logtime" : [ "2016-05-01 02:05:06 -0500" ], "user" : [ "bdmin" ] }, "sort" : [ 1462078806000 ]}
"fields" : { "logtime" : [ "2016-05-01 02:03:06 -0500" ], "user" : [ "admin" ] }, "sort" : [ 1462078806000 ]}
"fields" : { "logtime" : [ "2016-05-01 02:07:06 -0500" ], "user" : [ "gdmin" ] }, "sort" : [ 1462078806000 ]}
"fields" : { "logtime" : [ "2016-05-01 02:01:06 -0500" ], "user" : [ "cdmin" ] }, "sort" : [ 1462078806000 ]}
But if I do sort on field "user", the results are lexicographically sorted as expected. Query:
curl -XGET "localhost:9200/myindex/type1/_search?sort=user:desc&fields=logtime,user&pretty"
Result =
"fields" : {"logtime" : [ "2016-05-01 02:07:06 -0500" ],"user" : [ "gdmin" ]}, "sort" : [ "gdmin" ] }
"fields" : {"logtime" : [ "2016-05-01 02:01:06 -0500" ],"user" : [ "cdmin" ]}, "sort" : [ "cdmin" ] }
"fields" : {"logtime" : [ "2016-05-01 02:05:06 -0500" ],"user" : [ "bdmin" ]}, "sort" : [ "bdmin" ] }
"fields" : {"logtime" : [ "2016-05-01 02:03:06 -0500" ],"user" : [ "admin" ]}, "sort" : [ "admin" ] }
Can anyone please help me in pointing out where am I going wrong or what am I missing here?
In the docs, they have discussed about enabling _timestamp but am not sure how to assign my field "logtime" to this _timestamp field.
PLEASE NOTE that my pipeline uses fluentd to insert docs in es.
Any help is appreciated.
Thanks.
