# Optimizing Datefield processing by moving from date{} to ruby code

**URL:** <https://discuss.elastic.co/t/optimizing-datefield-processing-by-moving-from-date-to-ruby-code/215665>\
**Category:** Logstash\
**Created:** [January 20, 2020, 5:31am UTC](https://discuss.elastic.co/t/optimizing-datefield-processing-by-moving-from-date-to-ruby-code/215665 "2020-01-20T05:31:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![scott\_stash](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\
**Post date:** [January 20, 2020, 5:31am UTC](https://discuss.elastic.co/t/optimizing-datefield-processing-by-moving-from-date-to-ruby-code/215665/1 "2020-01-20T05:31:05Z")

</div>

Hello,

Currently in my pipeline I'm parsing out loglines, each log line begins with a timestamp such as: "2020-01-19T00:00:08.127+0800" to a field called logtimestamp, and then later in my pipeline update this to the @timestamp field.

For example..

date {  
match =\> ["logTimestamp", "ISO8601"]  
remove\_field =\> ["logTimestamp"]  
}

This works fine, but feels very slow. Looking at the stats under the pipeline, it is averaging between 0.05 - 0.06 ms/event on the date statement. To put in comparison, the dissect of my log line event takes on average 0.03 ms/event.

I've started looking at the code in timestamp.rb and it looks like its taking this timestamp, converting it to milliseconds and creating a logstash timestamp object.

If I run something like this it appears to drop me down to 0.03-0.04 ms/event:

ruby { code =\> ' event.set("@timestamp", LogStash::Timestamp.parse\_iso8601(event.get("logtimestamp"))) ' }

A second note is that in my case the log files come from different timezones, but I want to have them all saved as UTC, so timezone data is always dropped... so for instance:

event.set("@timestamp", LogStash::Timestamp.parse\_iso8601(event.get("logtimestamp")[0..22]))  
event.remove("logtimestamp")

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 17, 2020, 5:35am UTC](https://discuss.elastic.co/t/optimizing-datefield-processing-by-moving-from-date-to-ruby-code/215665/2 "2020-02-17T05:35:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.