Secondary rollup of existing rollup index

Elastic Stack Elasticsearch
1

Hi,
I have an index which is created from a Rollup job. Its content is as follows:

"hits": {
    "total": 4,
    "max_score": 1,
    "hits": [
      {
        "_index": "my_rollup_1m_2021.08.17",
        "_type": "_doc",
        "_id": "my_rollup_1m$09-UpSrS0X2Nnx3TyDfMtg",
        "_score": 1,
        "_source": {
          "stId.terms._count": 2,
          "timestamp.date_histogram.time_zone": "UTC",
          "timestamp.date_histogram.timestamp": 1629201600000,
          "count.sum.value": 20,
          "_rollup.version": 2,
          "stId.terms.value": 1,
          "timestamp.date_histogram.interval": "1h",
          "timestamp.date_histogram._count": 2,
          "_rollup.id": "my_rollup_1m"
        }
      },
    ....
}

Then I am trying to create a secondary level rollup job, that rolls up the index mentioned above. The secondary rollup job configuration is:

PUT _rollup/job/my_rollup_1h
{
  "id": "my_rollup_1h",
  "index_pattern": "my_rollup_1m_2021.08.17",
  "rollup_index": "<my_rollup_1h_{now/d}>",
  "cron": "0 39 * * * ?",
  "groups": {
    "date_histogram": {
      "interval": "1h",
      "field": "timestamp.date_histogram.timestamp",
      "time_zone": "UTC"
    },
    "terms": {
      "fields": [
        "stId.terms.value"
      ]
    }
  },
  "metrics": [
    {
      "field": "count.sum.value",
      "metrics": [
        "sum"
      ]
    }
  ],
  "timeout": "20s",
  "page_size": 10000
}

Secondary rollup job runs fine as seen from below status:

"status": {
    "job_state": "started",
    "current_position": {
      "st.terms.value.terms": 4,
      "timestamp.date_histogram.timestamp.date_histogram": 1629201600000
    },
    "upgraded_doc_id": true
  },
  "stats": {
    "pages_processed": 2,
    "documents_processed": 4,
    "rollups_indexed": 4,
    "trigger_count": 1,
    "index_time_in_ms": 1,
    "index_total": 1,
    "index_failures": 0,
    "search_time_in_ms": 5,
    "search_total": 2,
    "search_failures": 0
  }

But when i check the seconday rollup index, its always empty. Besides i see this exception in ES:

[2021-08-17T13:39:00,004][WARN ][o.e.x.c.i.AsyncTwoPhaseIndexer] [es-data-0] Error while attempting to bulk index documents: failure in bulk execution:
[0]: index [my_rollup_1h_2021.08.17], type [_doc], id [my_rollup_1h$2b4OsNouZ94vhB47tII2ww], message [MapperParsingException[failed to parse]; nested: ClassCastException[org.elasticsearch.index.mapper.DateFieldMapper incompatible with org.elasticsearch.index.mapper.ObjectMapper];]
[1]: index [my_rollup_1h_2021.08.17], type [_doc], id [my_rollup_1h$AU3POF2c3SwXnjzgga3d0g], message [MapperParsingException[failed to parse]; nested: ClassCastException[org.elasticsearch.index.mapper.DateFieldMapper incompatible with org.elasticsearch.index.mapper.ObjectMapper];]
[2]: index [my_rollup_1h_2021.08.17], type [_doc], id [my_rollup_1h$pQOaK-fGV4vl35xgviD4Og], message [MapperParsingException[failed to parse]; nested: ClassCastException[org.elasticsearch.index.mapper.DateFieldMapper incompatible with org.elasticsearch.index.mapper.ObjectMapper];]
[3]: index [my_rollup_1h_2021.08.17], type [_doc], id [my_rollup_1h$72Ci96Cz6XIVIoTHg3z-Rw], message [MapperParsingException[failed to parse]; nested: ClassCastException[org.elasticsearch.index.mapper.DateFieldMapper incompatible with org.elasticsearch.index.mapper.ObjectMapper];]

Seems like mapping exception when trying to index the processed data. Is there any thing else to be done when configuring the job in this scenario?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.