Rollup Error

[hsalim]

Hello,
I am using rollups, and now encountering following error. Any help is appreciated.

Thanks.
'
Rollup job failed with an exception:
java.lang.NullPointerException: Cannot invoke "org.elasticsearch.search.aggregations.Aggregations.get(String)" because the return value of "org.elasticsearch.action.search.SearchResponse.getAggregations(
)" is null'

[warkolm (Mark Walkom)]

I am not super familiar with rollups, but there's not a lot to go on here.

What version of the stack are you using?
What does your rollup job look like?
What does the full log show>

[hsalim]

Here is more information:
Stack version is 7.8
Rollup Job:
{ "config": { "id": "raw2diaroll1", "index_pattern": "raw2diameter1200bcon-*", "rollup_index": "raw2diaroll1", "cron": "0 * * * * ?", "groups": { "date_histogram": { "calendar_interval": "1m", "field": "@timestamp", "time_zone": "UTC" }, "terms": { "fields": [ "field 1.keyword", "field 2.keyword", "etc...", ] } }, "metrics": [ { "field": "@timestamp", "metrics": [ "value_count" ] } ], "timeout": "20s", "page_size": 1000 }, "status": { "job_state": "started", "current_position": { "@timestamp.date_histogram": 1603925940000, "field1.keyword.terms": null, "field2.keyword.terms": "field values", ...... }, "upgraded_doc_id": true }, "stats": { "pages_processed": 8270, "documents_processed": 3116858, "rollups_indexed": 3066011, "trigger_count": 5196, "index_time_in_ms": 937594, "index_total": 3074, "index_failures": 0, "search_time_in_ms": 2778608, "search_total": 8270, "search_failures": 9, "processing_time_in_ms": 176401, "processing_total": 8261 } }
Log file:
[2021-04-26T19:05:00,002][WARN ][o.e.x.r.j.RollupJobTask ] [node-1] Rollup job [raw2diaroll1] failed with an exception:
java.lang.NullPointerException: Cannot invoke "org.elasticsearch.search.aggregations.Aggregations.get(String)" because the return value of "org.elasticsearch.action.search.SearchResponse.getAggregations(
)" is null
at org.elasticsearch.xpack.rollup.job.RollupIndexer.doProcess(RollupIndexer.java:143) ~[?:?]
at org.elasticsearch.xpack.core.indexing.AsyncTwoPhaseIndexer.onSearchResponse(AsyncTwoPhaseIndexer.java:452) [x-pack-core-7.8.0.jar:7.8.0]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:89) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:83) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:43) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:165) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:518) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:403) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.search.TransportSearchAction.lambda$doExecute$3(TransportSearchAction.java:213) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:114) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:87) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:244) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:88) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:179) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$0(SecurityActionFilter.java:87) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$authorizeRequest$4(SecurityActionFilter.java:173) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$runRequestInterceptors$15(AuthorizationService.java:352) [x-pack-security-7.8.0.jar:7.8.0]
at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.common.util.concurrent.ListenableFuture$1.doRun(ListenableFuture.java:112) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:226) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListener(ListenableFuture.java:106) [elasticsearch-7.8.0.jar:7.8.0]
at org.elasticsearch.common.util.concurrent.ListenableFuture.lambda$done$0(ListenableFuture.java:98) [elasticsearch-7.8.0.jar:7.8.0]
at java.util.ArrayList.forEach(ArrayList.java:1510) [?:?]
.........

[hsalim]

Maybe this will help, I have already rolled up my raw data once and it worked without any issues. As a way of testing I am running the same raw files again, and nothing is happening. 'Current position' for rollup is not getting updated, when I execute GET _rollup/job/raw2diaroll1.