# Rollup not triggered

**URL:** <https://discuss.elastic.co/t/rollup-not-triggered/145930>\
**Category:** Elasticsearch\
**Created:** [August 24, 2018, 2:31pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930 "2018-08-24T14:31:19Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![ben\_f](https://avatars.discourse-cdn.com/v4/letter/b/ad7895/32.png) [@ben\_f](https://discuss.elastic.co/u/ben_f)\
**Post date:** [August 24, 2018, 2:31pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/1 "2018-08-24T14:31:20Z")

</div>

Hi,

We are trying to use the new rollup feature.  
`trigger_count` increases with time, but others metrics are sticking to 0, and the index remain empty.  
Are we missing something?

Thanks !

```
PUT _xpack/rollup/job/testrollup
    {
      "index_pattern": "dailyindex_539_*",
      "rollup_index": "test-rollup",
      "cron": "*/30 * * * * ?",
      "page_size": 1000,
      "groups": {
        "date_histogram": {
          "field": "@timestamp",
          "interval": "1m"
        }
      },
      "metrics": [
        {
          "field": "responsetime",
          "metrics": [
            "min",
            "max",
            "sum"
          ]
        }
      ]
    }

```

`POST _xpack/rollup/job/testrollup/_start`

```
GET _xpack/rollup/job/_all
{
  "jobs": [
    {
      "config": {
        "id": "testrollup",
        "index_pattern": "dailyindex_539_*",
        "rollup_index": "test-rollup",
        "cron": "*/30 * * * * ?",
        "groups": {
          "date_histogram": {
            "interval": "1m",
            "field": "@timestamp",
            "time_zone": "UTC"
          }
        },
        "metrics": [
          {
            "field": "responsetime",
            "metrics": [
              "min",
              "max",
              "sum"
            ]
          }
        ],
        "timeout": "20s",
        "page_size": 1000
      },
      "status": {
        "job_state": "started"
      },
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "rollups_indexed": 0,
        "trigger_count": 424
      }
    }
  ]
}

```

And rollup index remain empty

```
GET _cat/indices/test-rollup,dailyindex_539_*?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open dailyindex_539_2018-08-23 rm-eGdpZRrqp6Wa-PEmHqA 3 1 6427212 0 6.7gb 3.3gb
green open test-rollup lOPCzTGIQEunDvW2YlYUTA 5 1 0 0 2.5kb 1.2kb
```

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [August 28, 2018, 3:37pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/2 "2018-08-28T15:37:02Z")

</div>

Any errors/warnings in the log? Since it's triggering, I suspect there's something going wrong and it's aborting after the first search / before indexing docs.

You might also try deleting the job, deleting `test-rollup`, recreate/restart and see if there are any errors/warnings immediately after job creation.

Sorry for the delay, slipped through my inbox filter 🙂

---

<div class="post-metadata">

**Author:** ![ben\_f](https://avatars.discourse-cdn.com/v4/letter/b/ad7895/32.png) [@ben\_f](https://discuss.elastic.co/u/ben_f)\
**Post date:** [August 28, 2018, 4:43pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/3 "2018-08-28T16:43:41Z")

</div>

Hi,

We couldn't find any error in logs on the master node.  
We have tried to delete/recreate many times and we are getting correct acknowledge.

I'm wondering if we didn't reach that bug: [https://github.com/elastic/elasticsearch/pull/31890](https://github.com/elastic/elasticsearch/pull/31890)  
Here is the format of our timestamp: "@timestamp": "2018-08-28T14:28:48.386Z",

Benjamin

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [August 28, 2018, 4:56pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/4 "2018-08-28T16:56:14Z")

</div>

Very possible. What's the section in the `mapping` look like?

That bug manifests as `SearchPhaseExecutionException` warnings. See this [thread for an example exception](https://discuss.elastic.co/t/rollup-fails-with-elasticsearchparseexception-failed-to-parse-date-field-0-with-format-date-time/145098)

Would you be able to upgrade to 6.4 to try with the bugfix in place?

---

<div class="post-metadata">

**Author:** ![ben\_f](https://avatars.discourse-cdn.com/v4/letter/b/ad7895/32.png) [@ben\_f](https://discuss.elastic.co/u/ben_f)\
**Post date:** [August 28, 2018, 5:30pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/5 "2018-08-28T17:30:28Z")

</div>

> ```
> "properties": {
> "@timestamp": {
> "type": "date",
> "format": "dateOptionalTime"
> },
> 
> ```

Sorry, we actually have errors, I missed it (I only checked the Master Node).  
Here is the error :

```
[2018-08-28T19:00:00,035][DEBUG][o.e.a.s.TransportSearchAction] [es8-main] All shards failed for phase: [query]
org.elasticsearch.ElasticsearchParseException: failed to parse date field [1535475600000] with format [dateOptionalTime]
        at org.elasticsearch.common.joda.DateMathParser.parseDateTime(DateMathParser.java:213) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.common.joda.DateMathParser.parse(DateMathParser.java:66) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToMilliseconds(DateFieldMapper.java:318) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:342) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:459) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:466) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:269) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:888) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:79) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:67) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:50) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:285) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:275) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:67) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:627) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:607) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService.createContext(SearchService.java:569) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:551) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:347) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:333) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:329) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1019) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:724) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-6.3.0.jar:6.3.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.3.0.jar:6.3.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: java.lang.IllegalArgumentException: Unrecognized chars at the end of [1535475600000]: [0000]
        at org.elasticsearch.common.joda.DateMathParser.parseDateTime(DateMathParser.java:209) ~[elasticsearch-6.3.0.jar:6.3.0]

```

We plan to upgrade tomorrow morning.

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [August 28, 2018, 5:33pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/6 "2018-08-28T17:33:14Z")

</div>

Ah yep, looks like that bug. Upgrading should fix it.

I'll talk to Jim and see if we should backport this to 6.3.x as well.

---

<div class="post-metadata">

**Author:** ![ben\_f](https://avatars.discourse-cdn.com/v4/letter/b/ad7895/32.png) [@ben\_f](https://discuss.elastic.co/u/ben_f)\
**Post date:** [August 30, 2018, 8:19am UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/7 "2018-08-30T08:19:55Z")

</div>

Hey,

I confirm the upgrade fixed the issue.  
Thanks !

Benjamin

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [August 30, 2018, 1:32pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/8 "2018-08-30T13:32:56Z")

</div>

Great! Feel free to ping me directly on any other Rollup issues you encounter. I've been keeping an eye on the forums, but a direct ping will notify me quicker 🙂

Or if it looks like a bona fide bug, can always submit directly to github too.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 27, 2018, 1:33pm UTC](https://discuss.elastic.co/t/rollup-not-triggered/145930/9 "2018-09-27T13:33:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
