# Logstash - The default timestamp field does not match my log field **URL:** \ **Category:** Logstash\ **Created:** [February 19, 2024, 3:22pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622 "2024-02-19T15:22:43Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [February 19, 2024, 3:22pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/1 "2024-02-19T15:22:43Z")
Hello. I have an existing Elastic stack that is pulling in app and web server logs. For the web site, I have 2 software stacks - 1 is using an older apache format (comma separated values) and 1 is using a newer JSON format. With the JSON format, I'm trying to leverage my existing YAML filter and modifying it to try and pick up the JSON format. I've been able to get everything working except for the timestamp field. It is reflecting the upload time of the log, not the actual log entry. Here is an example of an Apache log entry: {"type":"ihs-access","datetime":"2024-02-15T23:59:59.999 EST","@version":"1","protocol":"HTTP/1.1","remoteHost":"10.10.10.10","port":443,"method":"GET","url":"/this/is/a/path/product/details/abcd","query":"","status":200,"bytes":1826,"elapsedTime":3344,"referer":"-","libertyServer":"serverhost:5443","tealeafId":"-","sessionId":"-","requestId":"-","jsessionId":"-","userAgent":"Apache-HttpClient/4.5.2 (Java/1.7.0)"} Here is a snippet of my logstash pipeline: ```auto input { beats { port => "5044" } } filter { if [fields][log_type] == "weblogs" { json { source => "message" tag_on_failure => ["_jsonparsefailure"] } mutate { replace => { type => "apache_access" } } mutate { rename => { "datetime" => "httpTime" } rename => { "remoteHost" => "httpHostIP" } rename => { "method" => "httpRequestMethod" } rename => { "url" => "httpRequest" } rename => { "query" => "httpRequestQueryParms" } rename => { "status" => "httpResponseCode" } rename => { "bytes" => "httpResponseSize" } rename => { "elapsedTime" => "httpResponseTime" } rename => { "referer" => "httpReferer" } rename => { "jsessionId" => "httpJSessionID" } rename => { "userAgent" => "httpUserAgent" } convert => { "httpResponseCode" => "integer" } convert => { "httpResponseSize" => "integer" } convert => { "httpResponseTime" => "integer" } } date { locale => "en" match => ["httpTime", "yyyy-MM-dd'T'HH:mm:ss:SSSz"] timezone => "US/Eastern" } } ``` I'm changing the field name from "datetime" to "httpTime" and then formatting the "httpTime" field. I'm assuming from the above that my date/time format is wrong on the date match. Can someone please help me format this properly based upon my input format? Thanks! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 19, 2024, 5:07pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/2 "2024-02-19T17:07:17Z")
Try `match => ["httpTime", "yyyy-MM-dd'T'HH:mm:ss.SSS ZZZ"]` ---
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [February 19, 2024, 5:29pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/3 "2024-02-19T17:29:56Z")
Nope, didn't work. Here is the modification: ```auto date { locale => "en" match => ["httpTime", "yyyy-MM-dd'T'HH:mm:ss:SSS ZZZ"] timezone => "US/Eastern" #timezone => "US/Central" } ``` Here is the index snippet: ![index1](https://us1.discourse-cdn.com/elastic/original/3X/8/d/8d41c12c51425e25f41b056635e6bf6584213832.jpeg) ![index2](https://us1.discourse-cdn.com/elastic/original/3X/5/5/55eb30138b00aeac9f0a40385228a419787f7517.jpeg) the @timestamp is still reflecting the time of the log ingestion. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 19, 2024, 5:41pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/4 "2024-02-19T17:41:24Z")
It works for me ``` input { generator { count => 1 lines => [''] } } output { stdout { codec => rubydebug { metadata => false } } } filter { mutate { remove_field => ["event", "host", "xlog"] } mutate { add_field => { "httpTime" => "2024-02-15T23:59:59.999 EST" } } date { locale => "en" match => ["httpTime", "yyyy-MM-dd'T'HH:mm:ss.SSS ZZZ"] } } ``` produces ``` "@timestamp" => 2024-02-16T04:59:59.999Z, "httpTime" => "2024-02-15T23:59:59.999 EST", ``` Since the timezone is specified in [httpTime] the timezone option has no effect. ---
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [February 19, 2024, 8:08pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/5 "2024-02-19T20:08:47Z")
Any idea what else I can try? Not sure why your test works but mine is does not. I removed that timezone option, but the results are the same. I'm at a loss at this point. ---
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [February 20, 2024, 1:16pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/6 "2024-02-20T13:16:36Z")
Ugh. Figured out the final issue. My input vs joda match was off by a single character: input: "datetime":"2024-02-15T23:59:59.999 EST" match (before): match =\> ["httpTime", "yyyy-MM-dd'T'HH:mm:ss:SSS ZZZ"] match (after): match =\> ["httpTime", "yyyy-MM-dd'T'HH:mm:ss.SSS ZZZ"] I had a colon right before the SSS and it should have been a dot. Gheesh! That was easy to miss. @Badger Thanks for all your help narrowing this down! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 19, 2024, 1:17pm UTC](https://discuss.elastic.co/t/logstash-the-default-timestamp-field-does-not-match-my-log-field/353622/7 "2024-03-19T13:17:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.