# Logstash date filter in UNIX\_MS from two fields
**URL:** https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580
**Category:** Logstash
**Created:** [September 6, 2017, 12:21pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580 "2017-09-06T12:21:11Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)
#### Post date: [September 6, 2017, 12:21pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/1 "2017-09-06T12:21:12Z")
I have some data from snort which has two subfields, in an !event¬ fields:
"event-microsecond" =\> 289367,
"event-second" =\> 1493741082,
I am trying to compose these into a float and then tell Elasticsearch it's the timestamp field.
filter {
mutate { add\_field =\>
{ "full\_timestamp" =\> "[event][event-second].[event][event-microsecond]" }
}
mutate { convert =\> { "full\_timestamp" =\> "float" } }
date {
locale =\> "en"
match =\> ["full\_timestamp", "UNIX\_MS"]
target =\> "@timestamp"
}
}
give me 0.0 for the full\_timestamp, so I'm doing something wrong when I try to make the new field.
I then tried
`mutate { add_field => { "full_timestamp" => "%{[event][event-second]}.%{[event][event-microsecond]}" } }`
and got
"full\_timestamp" =\> 1493741082.289367,
"@timestamp" =\> 1970-01-18T06:55:41.082Z,
The new field looks about right, but the date is wrong by miles; it should be 2nd May 2017.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 6, 2017, 1:35pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/2 "2017-09-06T13:35:27Z")
UNIX\_MS expects the input to be the time in milliseconds since the epoch, so to get the expected results you'll want `full_timestamp` to contain 1493741082289. You can use a ruby filter to divide `event-microsecond` by 1000 to turn it into milliseconds, then use a mutate filter to simply concatenate `event-second` and `event-microsecond` (with no intervening decimal point). Well, the concatenation could of course also be done with the same ruby filter.
---
### Author:  [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)
#### Post date: [September 6, 2017, 2:03pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/3 "2017-09-06T14:03:34Z")
I don't know much ruby but have tried this which seems to work:
(advice on something neater more than welcome)
```
filter {
ruby {
code => "s = sprintf('%03d', (event.get('[event][event-microsecond]').to_i/1000)); s = s + '000'; event.set('milliseconds', s[0..2])"
add_tag => ["ran the code"]
}
mutate { add_field =>
{ "full_timestamp" => "%{[event][event-second]}%{milliseconds}" }
}
mutate { convert => { "full_timestamp" => "integer" } }
date {
locale => "en"
match => ["full_timestamp", "UNIX_MS"]
target => "@timestamp"
}
}
```
giving:
"milliseconds" =\> "289",
"full\_timestamp" =\> 1493741082289,
"@timestamp" =\> 2017-05-02T16:04:42.289Z,
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 6, 2017, 2:09pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/4 "2017-09-06T14:09:05Z")
Shorter:
```ruby
event.set('full_timestamp', event.get('[event][event-second]').to_s + (event.get('[event][event-microsecond]') / 1000).to_s)
```
---
### Author:  [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)
#### Post date: [September 6, 2017, 2:16pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/5 "2017-09-06T14:16:02Z")
Still leaving the mutate to make it an integer?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 6, 2017, 2:24pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/6 "2017-09-06T14:24:08Z")
No, you can drop that if you want.
---
### Author:  [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)
#### Post date: [September 6, 2017, 2:26pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/7 "2017-09-06T14:26:19Z")
Perfect.
Is the date plugin treating it as an integer, so the ruby?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 6, 2017, 2:52pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/8 "2017-09-06T14:52:01Z")
The date filter parses strings. I'm sure it's fine with parsing an integer field but it'll start with converting the integer into a string.
---
### Author:  [@Frances\_Buontempo](https://discuss.elastic.co/u/Frances_Buontempo)
#### Post date: [September 7, 2017, 12:31pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/9 "2017-09-07T12:31:00Z")
One final observation,
I suspect I need
'[event][event-second]').to\_s + (event.get('[event][event-microsecond]') / 1000000).to\_s)
to change microseconds (I had /1000 before, which is milliseconds)
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [September 7, 2017, 12:59pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/10 "2017-09-07T12:59:30Z")
The desired result is milliseconds so it's correct to divide by 1000. The input is 1493741082 and 289367 and you want 1493741082289.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 5, 2017, 12:59pm UTC](https://discuss.elastic.co/t/logstash-date-filter-in-unix-ms-from-two-fields/99580/11 "2017-10-05T12:59:38Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.