# Timestamp grok pattern help

**URL:** https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393
**Category:** Logstash
**Created:** [December 5, 2017, 4:22pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393 "2017-12-05T16:22:05Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Ravikumar\_G](https://avatars.discourse-cdn.com/v4/letter/r/71e660/32.png) [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)
#### Post date: [December 5, 2017, 4:22pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/1 "2017-12-05T16:22:05Z")

</div>

Hello,

I'm trying to ingest timestamp with millisecond events but failing in kibana any help would be great

actual log

2017-12-05T16:14:10.543997+00:00 ccdn-ats-tk-40405-02 pump4[40837]: Level=Error, subSystem=SESSION, Event=contentNPTToSegmentNPT, item 23 contentNpt -1.000000 less then segment startNpt 0.000000, ODID=c0537bc32ea443958df8461306926324

Grok

#/opt/vista/var/log/pump1.log

if [type] == "pump" {  
grok {  
match =\> {  
"message" =\> [  
"%{NOTSPACE:timestamp} %{NOTSPACE:field\_1} %{NOTSPACE:field\_2} %{GREEDYDATA:raw\_data}",  
"%{NOTSPACE:timestamp},%{GREEDYDATA:raw\_data}"  
]  
}  
tag\_on\_failure =\> ["fail\_in\_grok"]  
}  
date {  
match =\> ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSZ"]  
}  
if [raw\_data] {  
kv {  
field\_split =\> ","  
source =\> "raw\_data"  
transform\_key =\> "lowercase"  
}  
}

```
  # if "fail_in_grok" not in [tags] {
  # mutate {
  # remove_field => ["timestamp"]
  # }
  #
  # }

    mutate {
               convert => {
                           "durationUs" => "integer"
             }
                     }

 }
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 5, 2017, 4:42pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/2 "2017-12-05T16:42:22Z")

</div>

```
match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ"]
```

does it for me.

---

<div class="post-metadata">

### Author: ![Ravikumar\_G](https://avatars.discourse-cdn.com/v4/letter/r/71e660/32.png) [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)
#### Post date: [December 5, 2017, 6:52pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/3 "2017-12-05T18:52:44Z")

</div>

No luck changed to

```
now match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ"]
before match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSZ"]

```

![39 PM](https://us1.discourse-cdn.com/elastic/original/3X/e/f/ef5488752833f38319a02e2536594b71b39c9c3e.png)

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 5, 2017, 7:02pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/4 "2017-12-05T19:02:47Z")

</div>

With

```
input { stdin { } }
output { stdout { codec => rubydebug } }

filter {
  grok {
    match => {
      "message" => ["%{NOTSPACE:timestamp} %{NOTSPACE:field_1} %{NOTSPACE:field_2} %{GREEDYDATA:raw_data}", "%{NOTSPACE:timestamp},%{GREEDYDATA:raw_data}"]
    }
    tag_on_failure => ["fail_in_grok"]
  }
  date {
    match => ["timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ"]
  }
  if [raw_data] {
    kv {
      field_split => ","
      source => "raw_data"
      transform_key => "lowercase"
    }
  }
}
```

and the input line you gave I get

```
{
    "@timestamp" => 2017-12-05T16:14:10.543Z,
         "level" => "Error",
       "field_1" => "ccdn-ats-tk-40405-02",
         " odid" => "c0537bc32ea443958df8461306926324",
       "field_2" => "pump4[40837]:",
      "@version" => "1",
          "host" => "[...]",
    " subsystem" => "SESSION",
        " event" => "contentNPTToSegmentNPT",
       "message" => "2017-12-05T16:14:10.543997+00:00 ccdn-ats-tk-40405-02 pump4[40837]: Level=Error, subSystem=SESSION, Event=contentNPTToSegmentNPT, item 23 contentNpt -1.000000 less then segment startNpt 0.000000, ODID=c0537bc32ea443958df8461306926324",
      "raw_data" => "Level=Error, subSystem=SESSION, Event=contentNPTToSegmentNPT, item 23 contentNpt -1.000000 less then segment startNpt 0.000000, ODID=c0537bc32ea443958df8461306926324",
     "timestamp" => "2017-12-05T16:14:10.543997+00:00"
}
```

What do you get?

---

<div class="post-metadata">

### Author: ![Ravikumar\_G](https://avatars.discourse-cdn.com/v4/letter/r/71e660/32.png) [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)
#### Post date: [December 5, 2017, 7:07pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/6 "2017-12-05T19:07:58Z")

</div>

![Screen Shot 2017-12-05 at 2](https://us1.discourse-cdn.com/elastic/original/3X/a/4/a4710ed49e4ba59b8654d4393e8dca55af38ba89.png)

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 5, 2017, 7:20pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/7 "2017-12-05T19:20:23Z")

</div>

Which worked, right? timestamp and @timestamp have the time from the message.

---

<div class="post-metadata">

### Author: ![Ravikumar\_G](https://avatars.discourse-cdn.com/v4/letter/r/71e660/32.png) [@Ravikumar\_G](https://discuss.elastic.co/u/Ravikumar_G)
#### Post date: [December 5, 2017, 8:15pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/8 "2017-12-05T20:15:11Z")

</div>

Hi Badger,

message filed has the just raw event if you see timestamp top and bottom one still missing

Thanks

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 5, 2017, 9:08pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/9 "2017-12-05T21:08:00Z")

</div>

Not getting it. The message has 2017-12-05T19:03:21.921649. The timestamp and @timestamp fields are set to December 5th 2017, 14:03:21.921. Those are the same if you are Eastern time.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 2, 2018, 9:08pm UTC](https://discuss.elastic.co/t/timestamp-grok-pattern-help/110393/10 "2018-01-02T21:08:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
