# How to setup pipline to extract file using grok in Elastic if the timestamp format is "20Aug21 20:36:07.058931 @fsafsasfdasdsadasd" **URL:** \ **Category:** Elasticsearch\ **Tags:** ingest-pipeline\ **Created:** [February 24, 2022, 3:38am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108 "2022-02-24T03:38:34Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\ **Post date:** [February 24, 2022, 3:38am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/1 "2022-02-24T03:38:34Z")
one of my log ,and the format is like as follows: `20Aug21 20:36:07.058931 @fsafsasfdasdsadasd` I want to extract `20Aug21 20:36:07.058931` to a date type field. I tried: ```auto PUT /_ingest/pipeline/xxx-log { "processors":[ { "grok":{ "field":"message", "patterns":[ "%{MY_DATE:timestamp_xyz}" ], "pattern_definitions":{ "MY_MONTH":"(?:[Jj]an(?:uary|uar)?|[Ff]eb(?:ruary|ruar)?|[Mm](?:a|ä)?r(?:ch|z)?|[Aa]pr(?:il)?|[Mm]a(?:y|i)?|[Jj]un(?:e|i)?|[Jj]ul(?:y|i)?|[Aa]ug(?:ust)?|[Ss]ep(?:tember)?|[Oo](?:c|k)?t(?:ober)?|[Nn]ov(?:ember)?|[Dd]e(?:c|z)(?:ember)?)", "MY_DATE":"%{MONTHDAY}%{MY_MONTH}%{YEAR} %{TIME}" }, "ignore_failure":true } }, { "date": { "field": "timestamp_xyz", "formats": ["ddMMMyyyy HH:mm:ss"], "target_field": "@timestamp-syz" } } ] } ``` but no @timestamp-syz in elastic. can you kindly give some suggestions how to setup this pipeline? for the definition of MY\_MONTH I just refer the [logstash-patterns/grok-patterns at master · hpcugent/logstash-patterns · GitHub](https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns) ```auto MONTH \b(?:Jan(?:uary|uar)?|Feb(?:ruary|ruar)?|M(?:a|ä)?r(?:ch|z)?|Apr(?:il)?|Ma(?:y|i)?|Jun(?:e|i)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|O(?:c|k)?t(?:ober)?|Nov(?:ember)?|De(?:c|z)(?:ember)?)\b ``` only delete the \b in front and rear ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 24, 2022, 4:24am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/2 "2022-02-24T04:24:22Z")
Hi @chenx319 Welcome to the community Assuming your message is `20Aug21 20:36:07.058931 @fsafsasfdasdsadasd` ```auto PUT /_ingest/pipeline/discuss-timestamp { "processors": [ { "dissect": { "field": "message", "pattern": "%{timestamp_xyz} @%{other}" } }, { "date": { "field": "timestamp_xyz", "formats": [ "ddMMMyy HH:mm:ss.SSSSSS" ], "target_field": "@timestamp-syz" } } ] } POST _ingest/pipeline/discuss-timestamp/_simulate { "docs": [ { "_index": "index", "_id": "id", "_source": { "message": "20Aug21 20:36:07.058931 @fsafsasfdasdsadasd" } } ] } ``` Result ```auto { "docs" : [ { "doc" : { "_index" : "index", "_type" : "_doc", "_id" : "id", "_source" : { "other" : "fsafsasfdasdsadasd", "@timestamp-syz" : "2021-08-20T20:36:07.058Z", "message" : "20Aug21 20:36:07.058931 @fsafsasfdasdsadasd", "timestamp_xyz" : "20Aug21 20:36:07.058931" }, "_ingest" : { "timestamp" : "2022-02-24T04:22:35.4143128Z" } } } ] } ``` ---
**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\ **Post date:** [February 24, 2022, 4:54am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/3 "2022-02-24T04:54:59Z")
@stephenb Thanks for your reply! Is this need two processors ? is it possible only in one processors? and I setup discuss-timestamp to filebeat, the timestamp-syz in list in Kibana , but the type is string ,not date ? how to make it is a date type? | Name | Type | Format | Searchable | Aggregatable | Excluded | | --- | --- | --- | --- | --- | --- | | @timestamp-syz | string | | | | | ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 24, 2022, 6:00am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/4 "2022-02-24T06:00:50Z")
2 processors ... They are very efficient. You need to define a mapping and set your fields type to a date type... Although it should pick it up as a date automatically. Did you delete the index and try again?? But in general the safe way is to define and mapping. > **[Date field type | Elasticsearch Guide \[8.0\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html)** ---
**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\ **Post date:** [February 24, 2022, 6:19am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/5 "2022-02-24T06:19:33Z")
> [@stephenb](#): > > Did you delete the index and try again?? i delete the index, and try again, the type is still string . my version is : elasticsearch-7.9.3-windows-x86\_64 filebeat-7.16.3-windows-x86\_64 kibana-7.9.3-windows-x86\_64 ---
**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\ **Post date:** [February 24, 2022, 6:32am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/6 "2022-02-24T06:32:01Z")
> [@stephenb](#): > > But in general the safe way is to define and mapping. this is OK for me Thanks a lot ! ```auto PUT indexcx-7.10.0-ghs-2022.02.24 { "mappings": { "properties": { "@timestamp-syz": { "type": "date" } } } } ``` ---
**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\ **Post date:** [February 26, 2022, 8:29am UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/7 "2022-02-26T08:29:49Z")
@stephenb > [@stephenb](#): > > in general the safe way is to define and mapping Is it possible that I define and mapping a field type in filebeat.yml ? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [February 26, 2022, 4:30pm UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/8 "2022-02-26T16:30:35Z")
> [@chenx319](#): > > Is it possible that I define and mapping a field type in filebeat.yml ? No not really, you would be better off adjusting / adding it to the the \_index\_template also I noticed it is not best practice to name the field with the `@` symbol that is usually reserved for the common `@timestamp` field... you can but normal practice. ---
**Author:** ![chenx319](https://avatars.discourse-cdn.com/v4/letter/c/a4c791/32.png) [@chenx319](https://discuss.elastic.co/u/chenx319)\ **Post date:** [February 27, 2022, 12:30pm UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/9 "2022-02-27T12:30:06Z")
@stephenb Thanks very much for your advice ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 27, 2022, 12:30pm UTC](https://discuss.elastic.co/t/how-to-setup-pipline-to-extract-file-using-grok-in-elastic-if-the-timestamp-format-is-20aug21-2007-058931-fsafsasfdasdsadasd/298108/10 "2022-03-27T12:30:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.