# Rookie Question: Record a count in a new Index

**URL:** <https://discuss.elastic.co/t/rookie-question-record-a-count-in-a-new-index/221716>\
**Category:** Elasticsearch\
**Created:** [March 2, 2020, 3:30pm UTC](https://discuss.elastic.co/t/rookie-question-record-a-count-in-a-new-index/221716 "2020-03-02T15:30:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![BlueTeamNinja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/blueteamninja/32/63639_2.png) [@BlueTeamNinja](https://discuss.elastic.co/u/BlueTeamNinja)\
**Post date:** [March 2, 2020, 3:30pm UTC](https://discuss.elastic.co/t/rookie-question-record-a-count-in-a-new-index/221716/1 "2020-03-02T15:30:37Z")

</div>

Hey all,

I'm brand new to my Elastic journey. I am collecting some firewall logs, which accumulate really quickly. What would I like to do is create a new index based on the counts of some of my tags.

A daily record that says 10,000,000 'Standard - Denied by policy' takes up exponentially less space and is a lot easier to chart behaviour over a year, than storing 10,000,000 documents simply to count them.

My first thought is to run a cron/scheduled task on an outside server that simple runs the query then puts it into its own index, but I feel there must be a native way and I'm simply missing the terminology.

How can I run a daily a journal based on counts of my tags? Or at least correct my terminology so I can find what I'm looking for in the amazing documentation.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [March 2, 2020, 3:34pm UTC](https://discuss.elastic.co/t/rookie-question-record-a-count-in-a-new-index/221716/2 "2020-03-02T15:34:54Z")

</div>

I think you might want to check Index lifecycle policy  
where I saw thing called "maximum document" that means it will roll over and create new index when it will reach that count.

---

<div class="post-metadata">

**Author:** ![BlueTeamNinja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/blueteamninja/32/63639_2.png) [@BlueTeamNinja](https://discuss.elastic.co/u/BlueTeamNinja)\
**Post date:** [March 2, 2020, 3:47pm UTC](https://discuss.elastic.co/t/rookie-question-record-a-count-in-a-new-index/221716/3 "2020-03-02T15:47:21Z")

</div>

I think I'm understanding the Lifecycle Management portion (although I'm sure it could be optimized as I learn more), what I am trying to do is demonstrate Firewall and network perimeter statistics, month over month and year over year.

I don't need the full firewall logs other than for a very short period for troubleshooting / investigations. 10,000,000 \*365 = a LOT.

I _think_ what I'm after is a workflow kind of like this:

- Create an hourly/daily schedule
- Create a Watcher that:
  - Uses the created schedule
  - Runs an Aggregation for my Tags
  - Has an Index Action to Put into `/CyberStats` or whatever boardroom / powerpointy index I create

Then I have a daily journal that I can show trending reports without maintaining a huge inventory of useless logs. Getting the syntax right looks like I'll need a cup of coffee.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2020, 3:47pm UTC](https://discuss.elastic.co/t/rookie-question-record-a-count-in-a-new-index/221716/4 "2020-03-30T15:47:24Z")

</div>

Closed on March 30, 2020, 3:47pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.