# Issue with extracting timestamp value from log file

**URL:** <https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580>\
**Category:** Logstash\
**Created:** [March 9, 2019, 4:16am UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580 "2019-03-09T04:16:18Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![raghunayak](https://avatars.discourse-cdn.com/v4/letter/r/43a26b/32.png) [@raghunayak](https://discuss.elastic.co/u/raghunayak)\
**Post date:** [March 9, 2019, 4:16am UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580/1 "2019-03-09T04:16:18Z")

</div>

Log file 1 format  
2019-02-26 03:00:50.000 -05:00 172.11.0.1 LOCAL:DEBUG some message  
Log file 2 format  
2019-01-12T05:03:04.956-05:00 172.11.0.1 LOCAL:DEBUG some message

I have created a common GROK format for the above files and trying to extract the log timestamp in field called logTimestamp. If I run the files separately one by one it works but if I run them both at the same time using filebeat then it fails. My logstash filter is as below

```
 grok {
        match => { "message" => ["^%{DATA:date} %{DATA:time} %{DATA:offset} %{GREEDYDATA:msgFragments}$"
    							]
    	
    				}
    	}
    	mutate {
    				add_field => { "doc_type" => "utilization" 
    							   "trans_type" => "xx"
    							   "record_type" => "yy" 
    								"parse_tag" => "commonSTRING"
    							 }
    			}
    	if [date] =~ /([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))(?!T)/ {
    			mutate {
    					
    					add_field => { "logTimestamp" => "%{date}T%{time}%{offset}"}
    					}
    				}
    	else {
    		  mutate {	
    					add_field => { "logTimestamp" => "%{date}"}
    				}	
    		 }
    	date {
        match => ["logTimestamp", "ISO8601"]
         }

```

@magnusbaeck -- can you please help me understand what I may be doing wrong ? thanks in advance !

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 9, 2019, 1:13pm UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580/2 "2019-03-09T13:13:26Z")

</div>

I would suggest a _much_ more specific grok filter.

```
    grok { match => { "message" => "^(?<date>[0-9\-]{10}[T][0-9:.]{12}\s*[0-9\-:]{6}) %{IPV4:ip} %{GREEDYDATA:restOfLine}" } }
    date { match => ["date", ISO8601, "yyyy-MM-dd HH:mm:ss.SSS ZZ"] }
```

---

<div class="post-metadata">

**Author:** ![raghunayak](https://avatars.discourse-cdn.com/v4/letter/r/43a26b/32.png) [@raghunayak](https://discuss.elastic.co/u/raghunayak)\
**Post date:** [March 9, 2019, 3:53pm UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580/3 "2019-03-09T15:53:41Z")

</div>

@Badger -- thank you so much ! That regex did the trick! I am new to this scripting, is there any good reference that you can recommend for me to study further ?  
Also I had another question ! I had read elsewhere that regex can sometime be costly operation as it tries to compare pattern ! I am new to this GROK scripting and wanted to make sure that I take performance as well in consideration as I iterate over this. Not sure if there is any easy answer to this but any pointers to good source will help! Do you use any tools to monitor how these expressions perform ?  
Thanks again for your answer...  
regards  
rags

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 9, 2019, 5:10pm UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580/4 "2019-03-09T17:10:49Z")

</div>

> [@raghunayak](#):
>
> Also I had another question ! I had read elsewhere that regex can sometime be costly operation as it tries to compare pattern

It can be. [This post](https://www.elastic.co/blog/do-you-grok-grok) on the elastic blog has some pointers.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 6, 2019, 5:10pm UTC](https://discuss.elastic.co/t/issue-with-extracting-timestamp-value-from-log-file/171580/5 "2019-04-06T17:10:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.