# Can date ingest processor process a String to extract a Date from it

**URL:** <https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493>\
**Category:** Elasticsearch\
**Created:** [December 13, 2017, 7:00am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493 "2017-12-13T07:00:42Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Augustus1995](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@Augustus1995](https://discuss.elastic.co/u/Augustus1995)\
**Post date:** [December 13, 2017, 7:00am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/1 "2017-12-13T07:00:42Z")

</div>

Hi guys  
I find a question about [Can date ingest processor process a String to extract a Date from it.](https://discuss.elastic.co/t/date-transform/85430/2)

And i follow the guide and use **Date Processor** , but my log message is complex

```auto
2017-12-13 11:10:30.000 [pool-10-thread-7] DEBUG c.h.i.c.m.servicemanager.task.PassiveNodeHbTask - PassiveNodeHbTask started

```

And in order to use **Date Processor** i have to use **Grok Processor** to extract date as a new field such as

```
PUT _ingest/pipeline/java-pipeline
{
  "description" : "describe pipeline",
  "processors" : [
    {
      "grok": {
        "field": "message",
        "patterns": ["%{TIMESTAMP_ISO8601:logatime}"]
      },
      "date" : {
        "field" : "logatime",
        "target_field": "@timestamp", 
        "formats" : ["yyyy-MM-dd HH:mm:ss.SSS"], 
        "timezone": "Asia/Shanghai"
      },
      "remove": {
        "field": "logatime"
      }
    }
  ]
}

```

But in this way , i will add an useless field ------ **logatime**

I don't want to add this field if possibile

So can date ingest processor process a complex String to extract a Date from it.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2017, 7:39am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/2 "2017-12-13T07:39:54Z")

</div>

`logatime` is removed by the `remove` processor. So I don't understand what you mean.

But I doubt your grok pattern is correct. I don't think that `%{TIMESTAMP_ISO8601:logatime}` matches `2017-12-13 11:10:30.000 [pool-10-thread-7] DEBUG ...`.

---

<div class="post-metadata">

**Author:** ![Augustus1995](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@Augustus1995](https://discuss.elastic.co/u/Augustus1995)\
**Post date:** [December 13, 2017, 8:10am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/3 "2017-12-13T08:10:23Z")

</div>

> [@dadoonet](#):
>
> .

In fact i use **logatime** as a middle field to help me extract date from the complex String, therefore i add **logatime** at beginning and remove **logatime** finally.

And it extract date as @timestamp from a complex string.

But i wonder is there have a simpler way to extract date directly without middle field. Such as i could use  
regular expression in **date processor**

---

<div class="post-metadata">

**Author:** ![Augustus1995](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@Augustus1995](https://discuss.elastic.co/u/Augustus1995)\
**Post date:** [December 13, 2017, 8:18am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/4 "2017-12-13T08:18:33Z")

</div>

And my grok pattern is incomplete because i only want to extract date. It still work and i will write it later.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2017, 10:11am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/5 "2017-12-13T10:11:16Z")

</div>

> But i wonder is there have a simpler way to extract date directly without middle field.

No but what is your concern of adding and removing a field?

---

<div class="post-metadata">

**Author:** ![Augustus1995](https://avatars.discourse-cdn.com/v4/letter/a/7cd45c/32.png) [@Augustus1995](https://discuss.elastic.co/u/Augustus1995)\
**Post date:** [December 13, 2017, 11:23am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/6 "2017-12-13T11:23:37Z")

</div>

> [@dadoonet](#):
>
> No but what is your concern of adding and removing a field?

Thanks a lot!!!  
i worried it maybe a little slow.  
but if there is no other solution, i will still use a middle field  
Thank you!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 10, 2018, 11:23am UTC](https://discuss.elastic.co/t/can-date-ingest-processor-process-a-string-to-extract-a-date-from-it/111493/7 "2018-01-10T11:23:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.