# Ingest pipeline is giving weird response to timeformat change

**URL:** https://discuss.elastic.co/t/ingest-pipeline-is-giving-weird-response-to-timeformat-change/146389
**Category:** Elasticsearch
**Created:** [August 28, 2018, 3:29pm UTC](https://discuss.elastic.co/t/ingest-pipeline-is-giving-weird-response-to-timeformat-change/146389 "2018-08-28T15:29:20Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Jonathan\_Tan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonathan_tan/32/83546_2.png) [@Jonathan\_Tan](https://discuss.elastic.co/u/Jonathan_Tan)
#### Post date: [August 28, 2018, 3:29pm UTC](https://discuss.elastic.co/t/ingest-pipeline-is-giving-weird-response-to-timeformat-change/146389/1 "2018-08-28T15:29:20Z")

</div>

Hi!

I'm stuck... I'd tested this pipeline in local, and now in a remote server (using the same docker image, mind you, v6.3.1), it's not behaving as I'd expected...

```auto
{
  "description": "Takes the eventTime field and turns it into a date field",
  "processors": [
    {
      "date": {
        "field": "eventTime",
        "target_field": "@timestamp",
        "formats": [
          "YYYY-MM-DD'T'HH:mm:ssZ",
          "YYYY/MM/DD HH:mm:ssZ"
        ]
      }
    }
  ],
  "on_failure": [
    {
      "set": {
        "field": "_index",
        "value": "failed-{{_index}}"
      }
    },
    {
      "set": {
        "field": "error",
        "value": "{{_ingest.on_failure_message}}"
      }
    }
  ]
}

```

Sample input:

```auto
{"eventType":"autosuggest","user":127903,"eventTime":"2018-08-29T00:41:35+1000","searchTerm":"swee"}
{"eventType":"autosuggest","user":127903,"eventTime":"2018-08-29T00:41:35+1000","searchTerm":"sweet "}
{"eventType":"autosuggest","user":127903,"eventTime":"2018-08-29T00:41:35+1000","searchTerm":"sweet so"}

```

Sample output from the index:

As you can see below, it takes an `eventtime` value of `2018-08-29T01:18:36+1000`, and then `@timestamp` turns into `2018-01-28T15:18:36.000Z`.

```auto
{
    "took": 4,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 29158,
        "max_score": 1,
        "hits": [
            {
                "_index": "search-metrics-filebeat-6.3.2-2018.08.28",
                "_type": "doc",
                "_id": "kMgegWUBozxkfJZkg-yB",
                "_score": 1,
                "_source": {
                    "offset": 16073720,
                    "prospector": {
                        "type": "log"
                    },
                    "source": "/usr/share/filebeat/logs/event_metrics_1535467082683.log",
                    "eventType": "autosuggest",
                    "tags": [
                        "json",
                        "search-events"
                    ],
                    "input": {
                        "type": "log"
                    },
                    "@timestamp": "2018-01-28T15:18:36.000Z",
                    "searchTerm": "chic",
                    "beat": {
                        "hostname": "6c703e3fceb2",
                        "name": "6c703e3fceb2",
                        "version": "6.3.2"
                    },
                    "eventTime": "2018-08-29T01:18:36+1000",
                    "host": {
                        "name": "6c703e3fceb2"
                    },
                    "user": 141139
                }
            },
            {
                "_index": "search-metrics-filebeat-6.3.2-2018.08.28",
                "_type": "doc",
                "_id": "m8gegWUBozxkfJZkg-yB",
                "_score": 1,
                "_source": {
                    "offset": 16074859,
                    "prospector": {
                        "type": "log"
                    },
                    "eventType": "autosuggest",
                    "source": "/usr/share/filebeat/logs/event_metrics_1535467082683.log",
                    "tags": [
                        "json",
                        "search-events"
                    ],
                    "input": {
                        "type": "log"
                    },
                    "@timestamp": "2018-01-28T15:18:36.000Z",
                    "searchTerm": "white su",
                    "eventTime": "2018-08-29T01:18:36+1000",
                    "beat": {
                        "hostname": "6c703e3fceb2",
                        "name": "6c703e3fceb2",
                        "version": "6.3.2"
                    },
                    "host": {
                        "name": "6c703e3fceb2"
                    },
                    "user": 103776
                }
            }
        ]
    }
}

```

I've got absolutely no idea why this isn't working... Any chance of some help?  
Thanks!

---

<div class="post-metadata">

### Author: ![Jonathan\_Tan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonathan_tan/32/83546_2.png) [@Jonathan\_Tan](https://discuss.elastic.co/u/Jonathan_Tan)
#### Post date: [August 28, 2018, 3:36pm UTC](https://discuss.elastic.co/t/ingest-pipeline-is-giving-weird-response-to-timeformat-change/146389/2 "2018-08-28T15:36:46Z")

</div>

If I remove the pipeline's processor - so it's just an empty array - the `@timestamp` goes to its original value of "ingested time". So it's definitely something strange happening with the processor. 😕

---

<div class="post-metadata">

### Author: ![Jonathan\_Tan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonathan_tan/32/83546_2.png) [@Jonathan\_Tan](https://discuss.elastic.co/u/Jonathan_Tan)
#### Post date: [August 29, 2018, 10:51am UTC](https://discuss.elastic.co/t/ingest-pipeline-is-giving-weird-response-to-timeformat-change/146389/3 "2018-08-29T10:51:29Z")

</div>

Ok. I figured out why...

I was using `DD` instead of `dd`.  
`DD` = day of the _year_  
`dd` = day of the _month_

Hopefully nobody else runs into such an embarrassing problem!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 26, 2018, 10:51am UTC](https://discuss.elastic.co/t/ingest-pipeline-is-giving-weird-response-to-timeformat-change/146389/4 "2018-09-26T10:51:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
