Kibana TIMESTAMP parse error?

Elastic Stack Kibana
1

Hi,

I am ingesting documents into ES through logstash with the TIMESTAMP field in this format: yyyy-MM-dd HH:mm:ss,SSS .

When i visualize the same in the KIBANA using date histogram it is showing error like this:

Visualize: failed to parse date field [1495747020000] with format [yyyy-MM-dd HH:mm:ss,SSS]

image.png1175×592 32.8 KB

When i seen in my ES logs i am getting like this:

[2017-05-31 22:27:51,519][DEBUG][action.search            ] [Headlok] All shards failed for phase: [query]
RemoteTransportException[[Headlok][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: ElasticsearchParseException[failed to parse date field [1495747020000] with format [yyyy-MM-dd HH:mm:ss,SSS]]; nested: IllegalArgumentException[Invalid format: "1495747020000" is malformed at "0000"];
Caused by: ElasticsearchParseException[failed to parse date field [1495747020000] with format [yyyy-MM-dd HH:mm:ss,SSS]]; nested: IllegalArgumentException[Invalid format: "1495747020000" is malformed at "0000"];
        at org.elasticsearch.common.joda.DateMathParser.parseDateTime(DateMathParser.java:203)
        at org.elasticsearch.common.joda.DateMathParser.parse(DateMathParser.java:67)
        at org.elasticsearch.index.mapper.core.DateFieldMapper$DateFieldType.parseToMilliseconds(DateFieldMapper.java:457)
        at org.elasticsearch.index.mapper.core.DateFieldMapper$DateFieldType.innerRangeQuery(DateFieldMapper.java:440)
        at org.elasticsearch.index.mapper.core.DateFieldMapper$DateFieldType.access$000(DateFieldMapper.java:201)
        at org.elasticsearch.index.mapper.core.DateFieldMapper$DateFieldType$LateParsingQuery.rewrite(DateFieldMapper.java:228)
        at org.apache.lucene.search.BooleanQuery.rewrite(BooleanQuery.java:278)
        at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:837)
        at org.elasticsearch.search.internal.ContextIndexSearcher.rewrite(ContextIndexSearcher.java:81)
        at org.elasticsearch.search.internal.DefaultSearchContext.preProcess(DefaultSearchContext.java:231)
        at org.elasticsearch.search.query.QueryPhase.preProcess(QueryPhase.java:103)
        at org.elasticsearch.search.SearchService.createContext(SearchService.java:689)
        at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:633)
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:377)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368)
        at org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365)
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33)
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IllegalArgumentException: Invalid format: "1495747020000" is malformed at "0000"
        at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187)
        at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:826)
        at org.elasticsearch.common.joda.DateMathParser.parseDateTime(DateMathParser.java:200)
        ... 22 more

How can i avoid this error? Whether i have to do any manipulations in the logstash.conf file?

Thanks

2

Can you share your mappings for the TIMESTAMP field? Are documents with that field in the same format?

3

Thanks @jbudz

These are the mappings of the TIMESTAMP field . Default it is coming as string so i edited the mappings and reindexed it again with the below mapping.

"TIMESTAMP": {
                 "type": "date",
                  "format": "yyyy-MM-dd HH:mm:ss,SSS",
               },​

It is in this format 2017-05-31 22:27:51,519.

Thanks

4

Put Like "yyyy-mm-ddTHH:mm:ss"

or follow link :- http://momentjs.com/docs/#/displaying/format/

5

The format which you had given is not suitable for my TIMESTAMP field.

Thanks

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.