# Replacing @timestamp with timestamp of my log

**URL:** <https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413>\
**Category:** Logstash\
**Created:** [December 8, 2016, 9:19am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413 "2016-12-08T09:19:53Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 9:19am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/1 "2016-12-08T09:19:53Z")

</div>

I want to replace timestamp with the timestamp of my log.. In place of @timestamp, I want to replace with timestamp\_match.

input {  
beats {  
port =\> "5044"  
}  
}  
filter {  
if [type] == "log" {  
grok {  
match =\> { "message" =\> "[%{TIMESTAMP\_ISO8601:timestamp\_match}] %{LOGLEVEL:log-level} [%{DATA:CONNECTION}]"}  
}  
mutate {  
remove\_field =\> ["message"]  
}  
}  
}  
output {  
elasticsearch { hosts =\> ["localhost:9200"]  
index =\> "changed"  
}  
}  
In the index I have:

"hits": [  
{  
"\_index": "changed",  
"\_type": "log",  
"\_id": "AVjddGVby6hU3wZv0i-y",  
"\_score": 1,  
"\_source": {  
"offset": 15252,  
"timestamp\_match": "2016-10-1 8:38:58,928",  
"input\_type": "log",  
"source": "C:\Users\Desktop\logc\lc.log",  
"type": "log",  
"tags": [  
"beats\_input\_codec\_plain\_applied"  
],  
"CONNECTION": "Result Success ",  
"@timestamp": "2016-12-08T07:55:32.645Z",  
"log-level": "INFO",  
"@version": "1",  
"beat": {  
"hostname": "GT5377",  
"name": "GT5377",  
"version": "5.0.0"  
},  
"host": "GT5377"  
}  
}  
Can anyone help me with this?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 8, 2016, 9:23am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/2 "2016-12-08T09:23:11Z")

</div>

You need to use a date filter, that'll do it 🙂

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 9:25am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/3 "2016-12-08T09:25:23Z")

</div>

I tried using like this .. it doesnt work  
date {  
match =\> ["@timestamp", "[%{TIMESTAMP\_ISO8601}]"]  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2016, 9:30am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/4 "2016-12-08T09:30:54Z")

</div>

It's not the `@timestamp` field you want to parse, it's `timestamp_match`. This might work:

```nohighlight
date {
  match => ["timestamp_match", "ISO8601"]
}

```

Since your date components aren't zero-padded (2016-10-1 instead of 2016-10-01) it might be a little bit more complicated.

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 9:42am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/5 "2016-12-08T09:42:49Z")

</div>

yeah. getting \_dateparsefailure ..

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2016, 9:52am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/6 "2016-12-08T09:52:12Z")

</div>

See the logs for details.

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 9:57am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/7 "2016-12-08T09:57:38Z")

</div>

I changed the log for dateparsefailure.((2016-10-1 instead of 2016-10-01) Dateparsefailure is fixed.  
I want to put this timestamp\_match("timestamp\_match": "2016-10-1 8:38:58,928",) value in place of @timestamp("@timestamp": "2016-12-08T07:55:32.645Z") ..  
I tried with replace option. It doesnt work..  
date {  
match =\> ["timestamp\_match", "ISO8601"]  
target =\> "@timestamp"  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2016, 10:04am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/8 "2016-12-08T10:04:36Z")

</div>

Sorry, I don't understand what you mean. What replace option?

You probably have to specify multiple patterns that the date filter will try one by one:

```nohighlight
date {
  match => ["timestamp_match", "yyyy-MM-dd HH:mm:ss,SSS", "yyyy-MM-d HH:mm:ss,SSS"]
}

```

(And similarly for the month numbers if they don't always have two digits.)

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 10:07am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/9 "2016-12-08T10:07:13Z")

</div>

yeah that two digits for the date issue is fixed..

I just need to put the value of  
"timestamp\_match": "2016-10-1 8:38:58,928",  
"@timestamp": "2016-12-08T07:55:32.645Z",

I am expecting something like :  
"@timestamp":"2016-10-1 8:38:58,928"

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2016, 10:08am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/10 "2016-12-08T10:08:37Z")

</div>

Try the suggestion I gave. If it still doesn't work I need to see the error message from the log.

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 10:14am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/11 "2016-12-08T10:14:43Z")

</div>

yeah that two digits for the date issue is fixed..

I just need to put the value of  
"timestamp\_match": "2016-10-1 8:38:58,928",  
"@timestamp": "2016-12-08T07:55:32.645Z",

I am expecting something like :  
"@timestamp":"2016-10-1 8:38:58,928"

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2016, 10:19am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/12 "2016-12-08T10:19:27Z")

</div>

Add a `stdout { codec => rubydebug }` output to your configuration and show the results.

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 10:25am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/13 "2016-12-08T10:25:53Z")

</div>

{  
"offset" =\> 22866,  
"timestamp\_match" =\> "2016-10-01 12:51:24,484",  
"input\_type" =\> "log",  
"source" =\> "C:\Users\Desktop\logc\lc.log",  
"type" =\> "log",  
"tags" =\> [  
[0] "beats\_input\_codec\_plain\_applied"  
],  
"CONNECTION" =\> "Adding REST interface for Profile",  
"@timestamp" =\> 2016-10-01T07:21:24.484Z,  
"log-level" =\> "INFO",  
"@version" =\> "1",  
"beat" =\> {  
"hostname" =\> "GT5377",  
"name" =\> "GT5377",  
"version" =\> "5.0.0"  
},  
"host" =\> "GT5377"  
}

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 10:26am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/14 "2016-12-08T10:26:49Z")

</div>

I want to put the value of timestamp\_match - in the value of @timestamp.

here is my config :

input {  
beats {  
port =\> "5044"  
}  
}  
filter {  
if [type] == "log" {  
grok {  
match =\> { "message" =\> "[%{TIMESTAMP\_ISO8601:timestamp\_match}] %{LOGLEVEL:log-level} [%{DATA:CONNECTION}]"}  
}  
mutate {  
remove\_field =\> ["message"]  
}  
date {  
match =\> ["timestamp\_match", "ISO8601"]  
target =\> "@timestamp"  
}  
}  
}  
output {  
elasticsearch { hosts =\> ["localhost:9200"]  
index =\> "sswi"  
}  
stdout { codec =\> rubydebug }  
}

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 10:33am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/15 "2016-12-08T10:33:34Z")

</div>

date {  
match =\> ["timestamp\_match", "ISO8601"]  
target =\> "@timestamp"  
}

I mentioned target as @timestamp - but it doesnt replace the value of timestamp\_match date value in the @timestamp.

---

<div class="post-metadata">

**Author:** ![taurs](https://avatars.discourse-cdn.com/v4/letter/t/90ced4/32.png) [@taurs](https://discuss.elastic.co/u/taurs)\
**Post date:** [December 8, 2016, 11:10am UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/16 "2016-12-08T11:10:16Z")

</div>

I also tried to replace this way:  
mutate {  
replace =\> ["@timestamp", "%{timestamp\_match}"]  
}  
It doesnt work too ☹

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [December 8, 2016, 12:09pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/17 "2016-12-08T12:09:21Z")

</div>

The `@timestamp` field is UTC but `timestamp_match` is local time. If your timezone is UTC+5:30 then things are working fine.

---

<div class="post-metadata">

**Author:** ![Bevan](https://avatars.discourse-cdn.com/v4/letter/b/9fc29f/32.png) [@Bevan](https://discuss.elastic.co/u/Bevan)\
**Post date:** [December 8, 2016, 6:55pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/18 "2016-12-08T18:55:22Z")

</div>

We wanted to do the same thing, plus we use a slightly non-standard timestamp syntax.  
Here's what works for us in our indexer:  
grok { match = { "message" =\> "^(?\d{4}-\d{2}-\d{2}@\d{2}:\d{2}:\d{2}(.\d{1,4})?)OTHERSTUFF" } }  
date { match =\> ["logtimestamp", "yyyy-MM-dd@HH:mm:ss.SSS"] }

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 5, 2017, 6:55pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-timestamp-of-my-log/68413/19 "2017-01-05T18:55:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.