# Grok parser date processor problem

**URL:** https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207
**Category:** Elasticsearch
**Created:** [October 24, 2020, 2:21pm UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207 "2020-10-24T14:21:28Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![gizem](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gizem](https://discuss.elastic.co/u/gizem)
#### Post date: [October 24, 2020, 2:21pm UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207/1 "2020-10-24T14:21:28Z")

</div>

Hi,  
I have the grok parser below:

```
{
  "description": "pipeline",
  "processors": [
    {
      "grok": {
        "if": "ctx.message.contains('LOG1')",
        "field": "message",
        "patterns": [
          "\\[%{GREEDYDATA:Time}\\] \\(%{LOGLEVEL:Severity}\\)%{SPACE} (?<Module Name>%{WORD:ModulName}): (?<Log Tag>%{DATA:logTag}) (?<Log Detail>%{GREEDYDATA:LogDetail})\\(%{GREEDYDATA:ErrorLineNumber}\\)"
        ]
      }
    },
	{
	  "date": {
        "field": "Time",
		"formats": ["yyyy.MM.dd HH:mm:ss.SSSSSS"],
		"timezone" : "Europe/Istanbul"
      }
	}
  ]
}

```

When the filebeat (v 7.9.2) started I show the logs:

```
|2020-10-24T16:45:13.807+0300|WARN|[elasticsearch]|elasticsearch/client.go:407|Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbfdd29d9cc8bee8f, ext:1046063582, loc:(*time.Location)(0x607ff60)}, Meta:null, Fields:{"agent":{"ephemeral_id":"d0aa7d92-1001-4370-9256-2d9c06d41723","hostname":"pc","id":"62ab133e-e96b-47ca-84e9-d36936121c53","name":"pc","type":"filebeat","version":"7.9.2"},"ecs":{"version":"1.5.0"},"fields":{"apache":true,"applicationname":"App","logtype":"application","pipeline":"pipeline","type":"application"},"host":{"name":"pc"},"input":{"type":"log"},"log":{"file":{"path":"/home/dev/application/release/current_release/log/performance.log"},"offset":734},"message":" (main.cpp:223)"}, Private:file.State{Id:"native::8139491-64770", PrevId:"", Finished:false, Fileinfo:(*os.fileStat)(0xc002307040), Source:"/home/dev/application/release/current_release/log/performance.log", Offset:756, Timestamp:time.Time{wall:0xbfdd29d9aee17d3d, ext:622096019, loc:(*time.Location)(0x607ff60)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x7c32e3, Device:0xfd02}, IdentifierName:"native"}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"illegal_argument_exception","reason":"field [Time] not present as part of path [Time]"}|
|---|---|---|---|---|
|2020-10-24T16:45:13.807+0300|WARN|[elasticsearch]|elasticsearch/client.go:407|Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbfdd29d9cc8c0562, ext:1046069425, loc:(*time.Location)(0x607ff60)}, Meta:null, Fields:{"agent":{"ephemeral_id":"d0aa7d92-1001-4370-9256-2d9c06d41723","hostname":"pc","id":"62ab133e-e96b-47ca-84e9-d36936121c53","name":"pc","type":"filebeat","version":"7.9.2"},"ecs":{"version":"1.5.0"},"fields":{"apache":true,"applicationname":"App","logtype":"application","pipeline":"pipeline","type":"application"},"host":{"name":pc"},"input":{"type":"log"},"log":{"file":{"path":"/home/test/application/release/current_release/log/performance.log"},"offset":464},"message":"[2020.10.16 16:38:35.044159] (ERROR) Main: \tipcs | grep [user_name] | awk '{ system (\"ipcrm -m \" $2)}' (cMemorySource.cpp:98)"}, Private:file.State{Id:"native::14157687-64770", PrevId:"", Finished:false, Fileinfo:(*os.fileStat)(0xc002307520), Source:"/home/test/application/release/current_release/log/performance.log", Offset:593, Timestamp:time.Time{wall:0xbfdd29d9b3b389f7, ext:702970700, loc:(*time.Location)(0x607ff60)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0xd80777, Device:0xfd02}, IdentifierName:"native"}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=400): {"type":"illegal_argument_exception","reason":"unable to parse date [2020.10.16 16:38:35.044159] (ERROR) Main: \tipcs | grep [user_name]","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2020.10.16 16:38:35.044159] (ERROR) Main: \tipcs | grep [user_name] with format [yyyy.MM.dd HH:mm:ss.SSSSSS]","caused_by":{"type":"date_time_parse_exception","reason":"Text '2020.10.16 16:38:35.044159] (ERROR) Main: \tipcs | grep [user_...' could not be parsed, unparsed text found at index 26"}}}|

```

I can't figure out this. Please help.

Regards.

---

<div class="post-metadata">

### Author: ![gizem](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gizem](https://discuss.elastic.co/u/gizem)
#### Post date: [October 24, 2020, 7:00pm UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207/3 "2020-10-24T19:00:23Z")

</div>

> [@gizem](#):
>
> `2020.10.16 16:38:35.044159`

Is the date unparseble with the parser below?

`yyyy.MM.dd HH:mm:ss.SSSSSS`

Any help?

---

<div class="post-metadata">

### Author: ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [October 26, 2020, 9:29am UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207/4 "2020-10-26T09:29:02Z")

</div>

Try this example

```auto
POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "date": {
          "field": "Time",
          "formats": [
            "yyyy.MM.dd HH:mm:ss.SSSSSS"
          ],
          "timezone": "Europe/Istanbul"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "Time": "2020.10.16 16:38:35.044159"
      }
    }
  ]
}

```

does this work for you? If so, is anything missing compared to your example? Also, what Elasticsearch version are you using?

---

<div class="post-metadata">

### Author: ![gizem](https://avatars.discourse-cdn.com/v4/letter/g/7ea924/32.png) [@gizem](https://discuss.elastic.co/u/gizem)
#### Post date: [October 26, 2020, 10:41am UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207/5 "2020-10-26T10:41:26Z")

</div>

> [@spinscale](#):
>
> ```auto
> {
> "pipeline": {
> "processors": [
> {
> "date": {
> "field": "Time",
> "formats": [
> "yyyy.MM.dd HH:mm:ss.SSSSSS"
> ],
> "timezone": "Europe/Istanbul"
> }
> }
> ]
> },
> "docs": [
> {
> "_source": {
> "Time": "2020.10.16 16:38:35.044159"
> }
> }
> ]
> }
> 
> ```

This works, and I use 7.9.2 version of elk stack.

---

<div class="post-metadata">

### Author: ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [October 26, 2020, 12:23pm UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207/6 "2020-10-26T12:23:04Z")

</div>

can you provide a **full** reproduction, with an original logline plus your pipeline to reproduce locally?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 23, 2020, 12:23pm UTC](https://discuss.elastic.co/t/grok-parser-date-processor-problem/253207/7 "2020-11-23T12:23:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
