Grok: Pattern appears to be matching but not creating new fields

Elastic Stack Logstash
1

Here is my Input

2015-06-02 17:13:51.331 +00:00

Here is the filter and match pattern that i'm using

}
filter {
   grok {
	   match => { "time" => "%{TIMESTAMP_ISO8601: event_timestamp}" }
  }
}

The pattern appears to match using a few online tools to confirm this, but when I import data from an microsoft sql DB with the input I posted above being in the first field which is named "time", no new fields are created when I'm looking at the data under the discover tab in Kibana.

I'm pretty new to Elasticsearch and Logstash and I was wondering if there was something simple that i'm missing or if something with my configuration is wrong?

Thanks for any help!

2

There shouldn't be a space before "event_timestamp", but I think the problem is that there's a space before the timezone offset and the grok patterns involved,

don't allow that AFAICT. If that indeed is the problem you can copy the definition of TIMESTAMP_ISO8601 and make the necessary adjustments:

match => {
  "time" => "(?<event_timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})? %{ISO8601_TIMEZONE}?)"
}
3

Thanks for the response, it does seem like the pattern is actually wrong. Aside from the pattern being incorrect, I noticed that the issue is with the data itself that i'm pulling from our Microsoft sql DB. The data in the DB is marked as "__createdAt" however when the data is pulled in from Logstash into Elasticsearch it appears as "createdat.timestamp" and " createdat.Offset". I suppose my first question is what is splitting my "__createdAt" field into the aforementioned two other fields and when is it happening?

Secondly i'm curious if there is anything I can do about it or how I should be looking for this field in my Grok match patterns?

4

I suppose my first question is what is splitting my "__createdAt" field into the aforementioned two other fields and when is it happening?

You haven't showed us all your configuration so it's hard to tell.

Secondly i'm curious if there is anything I can do about it or how I should be looking for this field in my Grok match patterns?

The date filter parses a single field, so you may need to create a new temporary field with both the timestamp and the offset for the date filter to parse. See below.