Hi everyone
I used logstash and elasticsearch 6.5 and i want to replace @timestamp to log's time but failed, i don't undertand where i'm wrong .
My log format :
2019-06-27 08:55:22 INFO 43665 SMS-MT [cid:nnx_smppgw_local_deli01] [queue-msgid:46fbddf5-2739-45cf-a0d5-ee7d39a69345] [smpp-msgid:42834c77-c957-4399-bb73-f701cc3c6962] [status:ESME_ROK] [prio:0] [dlr:NO_SMSC_DELIVERY_RECEIPT_REQUESTED] [validity:none] [from:1595] [to:84949191816] [content:'Thoi tiet Binh Thuan 27/6: Nang gian doan, tu chieu toi mua dong vai noi, kha nang mua 60%, 25-33 do, do am 77%. Vung bien: Co mua rao va dong rai rac. Tam nhin xa giam xuong 4-10km trong mua. Gio tay nam cap 4-5, co luc cap 6, giat cap 7. Trong con dong de phong loc xoay. Chi tiet goi 18001195(0d)']
Here is my config on logstash :
input {
beats {
port => 5044
In the Discover page click on the arrow on the left side of an entry. You will see the table view of all fields. Then click on the JSON tab on the right of the table tab. Now you can copy the content of the JSON document.
Well, in your discover view it shows the @timestamp field's value as "June 27th 2019, 16:06:15:271", which is the same as the timestamp in the log message (2019-06-27 16:06:15) - just in a different format.
So do you want the date to be formatted differently? This can be done in the Kibana settings.
Or did you want the message field to not contain the timestamp anymore? If so you could try displaying your logdata field instead of the message field.
I have lots of old log files that need to be uploaded on ELK but if i uploaded old log file on elk , the field time and @timestamp will not display log's time correctly . Then i want to replace @timestamp to log's time .
Your filters already grok the timestamp to the date_time field, which you then map to the @timestamp field using the date filter. That in itself is correct.
Also, In the screen captures you provided, the @timestamp always matched the time given in the message field. What exactly do you think is incorrect? Can you find a doc where the @timestamp field was processed incorrectly?
But the tags would indicate whether a date parse failure occured.
It would be good to determine that. For testing, it would also help if you could comment out the removal of the date_time field, so that you can check what was contained within it, if the date parse failure occurs.
Your date filter is conditional depending on the value of [tags], and if the date filter finds a value in [date_time] but fails to parse it then it will add a value to [tags], so the contents of [tags] are most certainly useful in diagnosing the problem.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.