# Getting Doc Counts using a Rollup query

**URL:** https://discuss.elastic.co/t/getting-doc-counts-using-a-rollup-query/206110
**Category:** Elasticsearch
**Created:** [November 1, 2019, 1:26am UTC](https://discuss.elastic.co/t/getting-doc-counts-using-a-rollup-query/206110 "2019-11-01T01:26:18Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![rvshchwl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rvshchwl/32/56346_2.png) [@rvshchwl](https://discuss.elastic.co/u/rvshchwl)
#### Post date: [November 1, 2019, 1:26am UTC](https://discuss.elastic.co/t/getting-doc-counts-using-a-rollup-query/206110/1 "2019-11-01T01:26:18Z")

</div>

I am currently using an Aggregation query to get Document Counts from a Date Histogram of 1 Minute samples. This is my current query:

```
scroll_query = {
"query" : {
"range": {
  "Timestamp": {
    "gte": "now-30d/d",
    "lte": "now"
  }
}
  },
  "size": 20, 
  "aggs" : {
    "resample" : {
      "date_histogram": {
        "field": "Timestamp",
        "interval": "minute"
     }
  }
  }
}

```

And I am able to get Document Counts for each 1 minute time window:

```
{'resample': {'buckets': [{'key_as_string': '2019-09-20T11:30:00.0000000Z',
    'key': 1568979000000,
    'doc_count': 677},
   {'key_as_string': '2019-09-20T11:31:00.0000000Z',
    'key': 1568979060000,
    'doc_count': 648},
   {'key_as_string': '2019-09-20T11:32:00.0000000Z',
    'key': 1568979120000,
    'doc_count': 1873}

```

I am trying to convert this query into a "Rollup", where I just need document counts for bucketting done on `Timestamp`.

I submitted this Rollup job:

```
rollup_payload = {
    "index_pattern": "cn_index",
    "rollup_index": "cn_rollup",
    "cron": "*/30 * * * * ?",
    "page_size" :1000,
    "groups" : {
      "date_histogram": {
        "field": "Timestamp",
        "interval": "minute"
      }
    }
}

```

When I run a query on this rollup, I get errors:

```
GET cn_rollup/_rollup_search
{
    "size" : 0
}

```

I tried this query using different parameters, such as changing `size` to 1000, but that throws a 400 Error. Is it possible to get the same results using Rollup?

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [November 2, 2019, 6:53pm UTC](https://discuss.elastic.co/t/getting-doc-counts-using-a-rollup-query/206110/2 "2019-11-02T18:53:29Z")

</div>

Hi,

your rollup configuration misses a metric, if you use value\_count on the Timestamp field you basically get the doc count you are looking for:

```auto
    "metrics": [
        {
            "field": "Timestamp",
            "metrics": ["value_count"]
        }
    ]

```

When using rollup\_search you need to specify an aggregation: [https://www.elastic.co/guide/en/elasticsearch/reference/7.x/rollup-search.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/rollup-search.html)

However, your usecase is not the typical rollup usecase, actually you do not need rollup\_search, as you already rollup your data in the way you need it. You can actually simply search on the index you created: "cn\_rollup", you do not even need the metric I suggested above, because there is a field for the doc count: "Timestamp.date\_histogram.\_count"

LBNL I want to mention another alternative. You might be interested in transform: [https://www.elastic.co/guide/en/elasticsearch/reference/7.4/put-transform.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/put-transform.html)

I think this is more suited to your usecase: It seems you create some sort of feature index, transform is made for that usecase.

An example:

```auto
PUT _data_frame/transforms/cn_transform
{
  "source": {
    "index": [
      "cn_index"
    ]
  },
  "pivot": {
    "group_by": {
      "time_bucket": {
        "date_histogram": {
          "field": "Timestamp",
          "fixed_interval": "1m"
        }
      }
    },
    "aggregations": {
      "count": {
        "value_count": {
          "field": "Timestamp"
        }
      }
    }
  },
  "dest": {
    "index": "cn_transform"
  },
  "sync": {
    "time": {
      "field": "Timestamp",
      "delay": "60s"
    }
  }
}

POST _data_frame/transforms/cn_transform/_start

```

This creates a transform and starts it. The transform continuously transforms the data from source index by pivoting it according to the configuration. The results are written to the specified destination index. Apart from the count you might want to create more features. Please have a look at transform, the documentation/examples should help you.

I hope this helps!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 30, 2019, 6:53pm UTC](https://discuss.elastic.co/t/getting-doc-counts-using-a-rollup-query/206110/3 "2019-11-30T18:53:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
