# Replace @timestamp with log's time Somebody help me please **URL:** \ **Category:** Logstash\ **Created:** [June 27, 2019, 2:39am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705 "2019-06-27T02:39:32Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 27, 2019, 2:39am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/1 "2019-06-27T02:39:32Z")
Hi everyone I used logstash and elasticsearch 6.5 and i want to replace @timestamp to log's time but failed, i don't undertand where i'm wrong . My log format : 2019-06-27 08:55:22 INFO 43665 SMS-MT [cid:nnx\_smppgw\_local\_deli01] [queue-msgid:46fbddf5-2739-45cf-a0d5-ee7d39a69345] [smpp-msgid:42834c77-c957-4399-bb73-f701cc3c6962] [status:ESME\_ROK] [prio:0] [dlr:NO\_SMSC\_DELIVERY\_RECEIPT\_REQUESTED] [validity:none] [from:1595] [to:84949191816] [content:'Thoi tiet Binh Thuan 27/6: Nang gian doan, tu chieu toi mua dong vai noi, kha nang mua 60%, 25-33 do, do am 77%. Vung bien: Co mua rao va dong rai rac. Tam nhin xa giam xuong 4-10km trong mua. Gio tay nam cap 4-5, co luc cap 6, giat cap 7. Trong con dong de phong loc xoay. Chi tiet goi 18001195(0d)'] Here is my config on logstash : input { beats { port =\> 5044 # Set to False if you do not use SSL ssl =\> false # Delete below linesif you do not use SSL #ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt" #ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key" client\_inactivity\_timeout =\> 600 } } filter { if "smpp\_nnx\_2" in [tags]{ grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:date\_time} %{LOGLEVEL:log} %{GREEDYDATA:logdata}" } overwrite =\> ["message"] } date { match =\> ["date\_time", "yyyy-MM-dd HH:mm:ss,SSS"] remove\_field =\> ["date\_time"] } } } output { if "smpp\_nnx\_2" in [tags]{ elasticsearch { hosts =\> ["localhost:9200"] index =\> "smpp\_nnx\_2" document\_type =\> "%{[@metadata][type]}" } stdout { codec =\> rubydebug } } Here is my patern when i creat on kibana ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/9/899c5ab3e5d980759e52e0ef730f13f933e91d7e.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/3/f3b316fdc259bec89773d49c45d7ea03f4c6d875.png) it's don't have any field i creat ! Somebody help me please . ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [June 27, 2019, 5:09am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/2 "2019-06-27T05:09:05Z")
Hi, Can you post the JSON of one of the entries in Kibana? If the date filter fails it adds a tag called ["\_dateparsefailure"](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-tag_on_failure) My guess is that the date\_time format in the date filter fails because you do not have the millisecond precision. Can you try: > ``` > date { > match => ["date_time", "yyyy-MM-dd HH:mm:ss"] > remove_field => ["date_time"] > } > > ``` ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 27, 2019, 7:13am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/3 "2019-06-27T07:13:05Z")
> [@Wolfram\_Haussig](#): > > Can you post the JSON of one of the entries in Kibana? If the date filter fails it adds a tag called ["\_dateparsefailure"](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-tag_on_failure) Hi [Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig) thanks for your help but how can i post the JSON of one of the entries in Kibana ? Can you show me ? i tried to repalce your date filter > date { > match =\> ["date\_time", "yyyy-MM-dd HH:mm:ss"] > remove\_field =\> ["date\_time"] > } > On my logstash config but it doesn't work either > > ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/0/90dc3b8b8fdc9c081be56778d483863de69cd861.png) > ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/1/a17c4353d4b37ebf9e40895443132305e4e3137f.png) > ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/6/d6225cbaa8a93501d2b4ac7a51267c3ca0c5ea41.png) ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [June 27, 2019, 7:16am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/4 "2019-06-27T07:16:37Z")
> [@anhdt061091](#): > > Hi [Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig) thanks for your help but how can i post the JSON of one of the entries in Kibana ? Can you show me ? In the Discover page click on the arrow on the left side of an entry. You will see the table view of all fields. Then click on the JSON tab on the right of the table tab. Now you can copy the content of the JSON document. ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/3/536db6c491a2f5ef240d839fea717e7462d588a0.png) ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 27, 2019, 7:22am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/5 "2019-06-27T07:22:25Z")
Hi [Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig) Here is my JSON in kibana ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/1/e1ab5964a093ded46746600f0091213c60ee588d.png) { "\_index": "smpp\_nnx\_2", "\_type": "doc", "\_id": "IMTGl2sBLXmOKnoJsZ9A", "\_version": 1, "\_score": null, "\_source": { "log": { "file": { "path": "/var/log/jasmin/messages.log" } }, "logdata": " 66196 SMS-MT [cid:nnx\_smppgw\_local\_deli01] [queue-msgid:83bba1b6-0ae4-4f98-9b9e-70a7fa0b475a] [smpp-msgid:0aefca6e-285e-4d86-a4ed-9b88d59af9e1] [status:ESME\_ROK] [prio:0] [dlr:NO\_SMSC\_DELIVERY\_RECEIPT\_REQUESTED] [validity:none] [from:1595] [to:84948870442] [content:'GA bi SOC NHIET khi nang nong keo dai. QK goi ngay 18001195 (0d) de duoc chuyen gia tu van cach ha nhiet do cho ga.']", "@timestamp": "2019-06-27T07:12:09.000Z", "beat": { "version": "6.8.0", "name": "SMPP-NNX-02", "hostname": "SMPP-NNX-02" }, "prospector": { "type": "log" }, "tags": [ "smpp\_nnx\_2", "beats\_input\_codec\_plain\_applied" ], "@version": "1", "input": { "type": "log" }, "message": "2019-06-27 14:12:09 INFO 66196 SMS-MT [cid:nnx\_smppgw\_local\_deli01] [queue-msgid:83bba1b6-0ae4-4f98-9b9e-70a7fa0b475a] [smpp-msgid:0aefca6e-285e-4d86-a4ed-9b88d59af9e1] [status:ESME\_ROK] [prio:0] [dlr:NO\_SMSC\_DELIVERY\_RECEIPT\_REQUESTED] [validity:none] [from:1595] [to:84948870442] [content:'GA bi SOC NHIET khi nang nong keo dai. QK goi ngay 18001195 (0d) de duoc chuyen gia tu van cach ha nhiet do cho ga.']", "source": "/var/log/jasmin/messages.log", "offset": 265390037, "host": { "os": { "version": "7 (Core)", "family": "redhat", "codename": "Core", "name": "CentOS Linux", "platform": "centos" }, "containerized": true, "architecture": "x86\_64", "name": "SMPP-NNX-02", "id": "41cb6e4b7919403591a2b90d176a6308" } }, "fields": { "@timestamp": [ "2019-06-27T07:12:09.000Z" ] }, "sort": [ 1561619529000 ] } ---
**Author:** ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Post date:** [June 27, 2019, 7:34am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/6 "2019-06-27T07:34:55Z")
It looks like your time is parsed correctly now? You have `2019-06-27T07:12:09` in your @timestamp field, and `2019-06-27 14:12:09` in your log message. The offset of 7 hours might be fixed [by specifying the correct timezone in the date filter](https://www.elastic.co/guide/en/logstash/6.8/plugins-filters-date.html#plugins-filters-date-timezone). ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 27, 2019, 7:52am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/7 "2019-06-27T07:52:57Z")
Hi [BennyInc](https://discuss.elastic.co/u/BennyInc)[Benny](https://discuss.elastic.co/u/BennyInc) Thanks for your help but i thought ELK can be replace @timestamp with date\_time i creat ? ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/9/b93a480baa9cbd717338256fd98deb656de47680.png) ---
**Author:** ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Post date:** [June 27, 2019, 8:33am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/8 "2019-06-27T08:33:18Z")
Do you want to replace the value in the `@timestamp` field with the value from the date\_time field you grok'ed? This is already done, I think. Or do you want the field to be called `date_time`? If so, you would need to [specify the target in the date filter](https://www.elastic.co/guide/en/logstash/6.8/plugins-filters-date.html#plugins-filters-date-target) (default is `@timestamp`). ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 27, 2019, 9:09am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/9 "2019-06-27T09:09:22Z")
Hi [BennyInc](https://discuss.elastic.co/u/BennyInc)[Benny](https://discuss.elastic.co/u/BennyInc) i don't think is done because when i add timestamp field it doesn't show the log's time. You can see this pic on Time Field an @Timestamp field i want replace log's time on that . Can you help me please ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/3/f3719920e74cdc038b6cd3b8c3f2ecb05f425f38.png) ---
**Author:** ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Post date:** [June 27, 2019, 9:38am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/10 "2019-06-27T09:38:10Z")
Well, in your discover view it shows the `@timestamp` field's value as "`June 27th 2019, 16:06:15:271`", which is the same as the timestamp in the log message (`2019-06-27 16:06:15`) - just in a different format. So do you want the date to be formatted differently? This can be done in the Kibana settings. Or did you want the `message` field to not contain the timestamp anymore? If so you could try displaying your `logdata` field instead of the `message` field. ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 27, 2019, 10:13am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/11 "2019-06-27T10:13:25Z")
hi [BennyInc](https://discuss.elastic.co/u/BennyInc)[Benny](https://discuss.elastic.co/u/BennyInc) I have lots of old log files that need to be uploaded on ELK but if i uploaded old log file on elk , the field time and @timestamp will not display log's time correctly . Then i want to replace @timestamp to log's time . ---
**Author:** ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Post date:** [June 27, 2019, 10:30am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/12 "2019-06-27T10:30:44Z")
Your filters already grok the timestamp to the `date_time` field, which you then map to the `@timestamp` field using the date filter. That in itself is correct. Also, In the screen captures you provided, the `@timestamp` always matched the time given in the `message` field. What exactly do you think is incorrect? Can you find a doc where the `@timestamp` field was processed incorrectly? ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [June 30, 2019, 1:10pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/13 "2019-06-30T13:10:03Z")
hi [BennyInc](https://discuss.elastic.co/u/BennyInc)[Benny](https://discuss.elastic.co/u/BennyInc) This is an example of the wrong time when I get old log on ELK ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/e/6ef392387685af515c1d2b63a6a16cd6774a62d1.png) Can you fix that for me ? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 30, 2019, 1:44pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/14 "2019-06-30T13:44:11Z")
What is the value of the tags field on those messages? ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [July 1, 2019, 1:46am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/15 "2019-07-01T01:46:09Z")
Hi [Badger](https://discuss.elastic.co/u/Badger) Other values don't matter, I just need to change @timestamp to logs time . ---
**Author:** ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)\ **Post date:** [July 1, 2019, 7:25am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/16 "2019-07-01T07:25:22Z")
> [@anhdt061091](#): > > Other values don't matter, I just need to change @timestamp to logs time . But the tags would indicate whether a date parse failure occured. It would be good to determine that. For testing, it would also help if you could comment out the removal of the date\_time field, so that you can check what was contained within it, if the date parse failure occurs. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 1, 2019, 12:39pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/17 "2019-07-01T12:39:12Z")
> [@anhdt061091](#): > > Other values don't matter, I just need to change @timestamp to logs time . Your date filter is conditional depending on the value of [tags], and if the date filter finds a value in [date\_time] but fails to parse it then it will add a value to [tags], so the contents of [tags] are most certainly useful in diagnosing the problem. ---
**Author:** ![anhdt061091](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@anhdt061091](https://discuss.elastic.co/u/anhdt061091)\ **Post date:** [July 2, 2019, 2:10am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/18 "2019-07-02T02:10:33Z")
Hi [BennyInc](https://discuss.elastic.co/u/BennyInc)[Benny](https://discuss.elastic.co/u/BennyInc) Thanks for your help, i have found a solution for my proplem, now i can replace @timestamp to date\_time. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 30, 2019, 2:10am UTC](https://discuss.elastic.co/t/replace-timestamp-with-logs-time-somebody-help-me-please/187705/19 "2019-07-30T02:10:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.