# Rollup job failed to find field

**URL:** <https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233>\
**Category:** Elasticsearch\
**Created:** [August 13, 2018, 8:03pm UTC](https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233 "2018-08-13T20:03:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paul\_Ainslie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_ainslie/32/55031_2.png) [@Paul\_Ainslie](https://discuss.elastic.co/u/Paul_Ainslie)\
**Post date:** [August 13, 2018, 8:03pm UTC](https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233/1 "2018-08-13T20:03:02Z")

</div>

I tried to create a data rollup on data that was pulled in from Logstash but I was getting error: `failed to find field [@timestamp] and [missing] is not provided`.

This is the error in the /var/log/elasticsearch folder.

```auto
[2018-08-13T19:19:45,394][WARN][o.e.x.r.j.RollupJobTask] Rollup job [ms4] failed with an exception: java.lang.RuntimeException: Shard failures encountered while running indexer for rollup job [ms4]: [shard [[Y7et4P77SK66xlfUYleBoQ][mediaserver_2017-10g-1h-rollup][0]], 
reason [RemoteTransportException[[mgr-elastic-002][5.9.154.55:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[failed to find field [@timestamp] and [missing] is not provided]; ], cause [[mediaserver_2017-10g-1h-rollup/aqNUCpQ8TTGxlFzGLLHxRA] QueryShardException[failed to find field [@timestamp] and [missing] is not provided]

```

I realized that I cannot use hyphens in the `rollup_index`! Once I replaced hyphens with underscores, it worked!

Here's the syntax with a rollup\_index name that failed!

```auto
{
    "index_pattern": "mediaserver_2017_2017-10*",
    "rollup_index": "mediaserver_2017-10g-1h-rollup",
    "cron": "*/10 * * * * ?",
    "page_size" :10000,
    "groups" : {
      "date_histogram": {
        "field": "timestamp",
        "interval": "15m",
        "delay": "10m"
      },
      "terms": {
        "fields": ["account", "stream", "streamtype", "host", "http_host", "clientip_n_agent", "geoip.region_name", "geoip.country_name", "geoip.continent_code", "cache_status", "response", "verb"]
      },
      "histogram": {
        "fields": ["request_time"],
        "interval": 1
      }
    },
    "metrics": [
        {
            "field": "bytes",
            "metrics": ["sum"]
        },
        {
            "field": "request_time",
            "metrics": ["avg","min","max"]
        }
    ]
}

```

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [August 17, 2018, 12:43pm UTC](https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233/2 "2018-08-17T12:43:50Z")

</div>

Ah, I think you ran into the bug fixed by [https://github.com/elastic/elasticsearch/pull/30491](https://github.com/elastic/elasticsearch/pull/30491)

The issue is that the index pattern (`mediaserver_2017_2017-10*`) matches the destination rollup index. So when the job is rolling up data, it accidentally searches the destination rollup index too, which has different field names, and so it throws an exception.

That bug is fixed in 6.4, we throw an exception when you try to create the job telling you that the pattern would match itself.

The reason it worked when you switched from hyphen to underscore is likely because the pattern no longer matched. Sorry for the bug, it's definitely irritating!

---

<div class="post-metadata">

**Author:** ![Paul\_Ainslie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_ainslie/32/55031_2.png) [@Paul\_Ainslie](https://discuss.elastic.co/u/Paul_Ainslie)\
**Post date:** [August 20, 2018, 12:34pm UTC](https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233/3 "2018-08-20T12:34:44Z")

</div>

Ah, that's better!

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [August 22, 2018, 11:32am UTC](https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233/4 "2018-08-22T11:32:21Z")

</div>

Great, glad that fixed it! Sorry for the inconvenience, the situation should be clearer in the future after that bugfix is released 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 19, 2018, 11:32am UTC](https://discuss.elastic.co/t/rollup-job-failed-to-find-field/144233/5 "2018-09-19T11:32:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.