# Marvel Document Creation Concern
**URL:** https://discuss.elastic.co/t/marvel-document-creation-concern/15571
**Category:** Elasticsearch
**Created:** [February 3, 2014, 8:11pm UTC](https://discuss.elastic.co/t/marvel-document-creation-concern/15571 "2014-02-03T20:11:50Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@David\_Harrigan](https://discuss.elastic.co/u/David_Harrigan)
#### Post date: [February 3, 2014, 8:11pm UTC](https://discuss.elastic.co/t/marvel-document-creation-concern/15571/1 "2014-02-03T20:11:50Z")
Hi,
Firstly, thanks for an awesome plugin. I'm trying it out and so far very
impressed.
However, I'm very concerned about something. I have a small dataset of
about 30,000 documents which only grows a few thousand every few days.
Since installing Marvel (on launch day), my total number of documents has
shot up to over 2 million and growing! It seems that Marvel is writing
documents every few milliseconds into it's indices. I'm watching the
Document Count just going up and up! This leads me on to my first question.
Is this normal? Secondly, if I don't want this number of documents what do
I do? Can I get Marvel to delete data over X days old?
Any guidance and advice on this would be greatly appreciated.
Thank you kindly,
-=david=-
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/786c0e7e-c71a-4439-a14f-094dcd0883e8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/786c0e7e-c71a-4439-a14f-094dcd0883e8%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Joseph\_Lombardo](https://discuss.elastic.co/u/Joseph_Lombardo)
#### Post date: [February 3, 2014, 8:16pm UTC](https://discuss.elastic.co/t/marvel-document-creation-concern/15571/2 "2014-02-03T20:16:29Z")
Same issue here!
1GB of total data with very little traffic and Marvel is generating a 4.5GB
index every day.
On Monday, February 3, 2014 3:11:50 PM UTC-5, David Harrigan wrote:
> Hi,
>
> Firstly, thanks for an awesome plugin. I'm trying it out and so far very
> impressed.
>
> However, I'm very concerned about something. I have a small dataset of
> about 30,000 documents which only grows a few thousand every few days.
> Since installing Marvel (on launch day), my total number of documents has
> shot up to over 2 million and growing! It seems that Marvel is writing
> documents every few milliseconds into it's indices. I'm watching the
> Document Count just going up and up! This leads me on to my first question.
> Is this normal? Secondly, if I don't want this number of documents what do
> I do? Can I get Marvel to delete data over X days old?
>
> Any guidance and advice on this would be greatly appreciated.
>
> Thank you kindly,
>
> -=david=-
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a3b193cb-0f32-454d-9910-1bec27fddec3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a3b193cb-0f32-454d-9910-1bec27fddec3%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [February 4, 2014, 10:13am UTC](https://discuss.elastic.co/t/marvel-document-creation-concern/15571/3 "2014-02-04T10:13:31Z")
Hi David, Joseph,
Marvel generates (by default) reports every 5 seconds. Depending on the
number of nodes/shards & indices this can result in different amount of
data. All of marvel's data is stored (again by default) in daily indices
named .marvel-YYYY.MM.dd (replace with date). You can safely delete older
data based on your retention needs.
This little tool can be very handy
there: [GitHub - elastic/curator: Curator: Tending your Elasticsearch indices](https://github.com/elasticsearch/curator)
Cheers,
Boaz
On Monday, February 3, 2014 9:16:29 PM UTC+1, Joseph Lombardo wrote:
> Same issue here!
>
> 1GB of total data with very little traffic and Marvel is generating a
> 4.5GB index every day.
>
> On Monday, February 3, 2014 3:11:50 PM UTC-5, David Harrigan wrote:
>
> > Hi,
> >
> > Firstly, thanks for an awesome plugin. I'm trying it out and so far very
> > impressed.
> >
> > However, I'm very concerned about something. I have a small dataset of
> > about 30,000 documents which only grows a few thousand every few days.
> > Since installing Marvel (on launch day), my total number of documents has
> > shot up to over 2 million and growing! It seems that Marvel is writing
> > documents every few milliseconds into it's indices. I'm watching the
> > Document Count just going up and up! This leads me on to my first question.
> > Is this normal? Secondly, if I don't want this number of documents what do
> > I do? Can I get Marvel to delete data over X days old?
> >
> > Any guidance and advice on this would be greatly appreciated.
> >
> > Thank you kindly,
> >
> > -=david=-
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eb316604-8bcc-41b2-ad89-f61a5803571d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eb316604-8bcc-41b2-ad89-f61a5803571d%40googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:52am UTC](https://discuss.elastic.co/t/marvel-document-creation-concern/15571/4 "2017-07-06T01:52:51Z")