# Logstash Json filter re-formated the @timestamp field unexpectedly
**URL:** https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027
**Category:** Logstash
**Created:** [June 14, 2020, 4:35pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027 "2020-06-14T16:35:11Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@STAR\_DEVX](https://discuss.elastic.co/u/STAR_DEVX)
#### Post date: [June 14, 2020, 4:35pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/1 "2020-06-14T16:35:12Z")
Hi,
I'm encounter an issue with the logstash json plugin.
In my "message", there's an field called "@timestamp", it's in UNIX epoch float format. However, when I try to use json filter to parse the "message" field, it converts the field to \_@timestamp, with value of scientific notation string.
e.g.
The incoming message:
```auto
"@timestamp": "2020-06-14T16:21:06.562Z",
"message": "{ \"@timestamp\":1592107552.495885,\"AGENT_ID\":\"892a6367-8924-4ed2-9e0a-eced1c7af0e6-S0\"....}"
}
```
This is my json filter
```auto
json {
# convert the document string in message to json obj
source => "message"
# remove the message (json string) once it's converted to json obj
remove_field => ["message"]
}
```
I got a WARN log entry for every message parsed, which may blow our log files, but that's another issue.
```auto
Unrecognized @timestamp value, setting current time to @timestamp, original in _@timestamp field {:value=>"0.1592150846569201e10"}
```
Most importantly ,it conversion the @timestamp field from my message to the scientific notation string!
```auto
"@timestamp": "2020-06-14T16:21:06.562Z",
"_@timestamp": "0.1592150846569201e10"
```
My question:
1. Is there a date match format that I can use out of box to parse the scientific notation? Instead of writing my own ruby code to recover the original float timestamp?
2. Is there a way to suppress overwhelmed WARN logs from json filter, such as
```auto
Unrecognized @timestamp value, setting current time to @timestamp, original in _@timestamp field {:value=>"0.1592150846569201e10"}
```
Thank you!
---
### Author:  [@ptamba](https://discuss.elastic.co/u/ptamba)
#### Post date: [June 14, 2020, 4:56pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/2 "2020-06-14T16:56:57Z")
> [@STAR\_DEVX](#):
>
> My question:
>
> 1. Is there a date match format that I can use out of box to parse the scientific notation? Instead of writing my own ruby code to recover the original float timestamp?
> 2. Is there a way to suppress overwhelmed WARN logs from json filter, such as
1. I haven’t tried this but base on [this](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-match), looks like the UNIX format will parse float epoch
2. you can set the json target to another field then use a date filter to convert the timestamp. that way your filter won’t try to override the timestamp for every entry hence avoiding the warning message . but depending on the amount of fields in the json, you could end-up having a lot of fields under the target
---
### Author:  [@STAR\_DEVX](https://discuss.elastic.co/u/STAR_DEVX)
#### Post date: [June 14, 2020, 5:02pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/3 "2020-06-14T17:02:07Z")
Thank you very much for your prompt reply
#1, I tried to use the following to parse the scientific notation string, but I got a timestampparsefaile.
```auto
date {
match => ["_@timestamp", UNIX]
}
```
It seems that it's good at parsing UNIX float, but NOT the scientific notation format
#2. I thought about that, but the issue is that I don't want all the fields out of json ended up being in an nested json object. To comply with our data model, I had to write another ruby script to move each of them out of the nested object to the root level, which is quite inefficient.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 14, 2020, 5:20pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/4 "2020-06-14T17:20:37Z")
Support for epoch values when coercing @timestamp is discussed [here](https://github.com/elastic/logstash/pull/10369).
Moving values from a field to the root level is pretty cheap. The ruby code is [here](https://discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006/2).
---
### Author:  [@STAR\_DEVX](https://discuss.elastic.co/u/STAR_DEVX)
#### Post date: [June 15, 2020, 5:02am UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/5 "2020-06-15T05:02:26Z")
Thank you @Badger, I implemented a hack before viewing your post above:
Given the message in my use case are usually short, I used gsub replaced the "@timestamp" in the "message" string to another key before passing the the "message" content to the json filter. It works.
```auto
mutate {
# Hack, rename the @timestamp field embedded in the message to another string to avoid conflict when parsing the json
gsub => ["message", "\"@timestamp\"", '"logCreateTime"']
}
json {
# convert the document string in message to json obj
source => "message"
# remove the message (json string) once it's converted to json obj
remove_field => ["message"]
}
```
From your experience, will this be a cheaper or more expensive operation than moving the keys?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 15, 2020, 12:36pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/6 "2020-06-15T12:36:13Z")
Your solution could be cheaper, but I doubt either is going to be expensive enough to care about.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 13, 2020, 12:36pm UTC](https://discuss.elastic.co/t/logstash-json-filter-re-formated-the-timestamp-field-unexpectedly/237027/7 "2020-07-13T12:36:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.