Marvel Indices taking lot of space ? Can we specify automatic delete of marvel indice?

Elastic Stack Elasticsearch
1

Is there a way to set marvel to delete the marvel indices after 7 days. It looks like Marvel is generating around 2 GB of data everyday. Our disk got full 2 times because of Marvel data. Is there a way to reduce the amount of data generated by marvel ?

Also is there any plan to add alert mechanisms in Marvel.
For example if the Marvel status goto red it will be good to get an email for a specified user. I see Marvel status as red. But it doesnt show why it is causing red. It will be good to get some alerts with the details when the cluster status goes red.

2

I do not use Marvel, but another monitoring system built on top of
Elasticsearch. I use the Elasticsearch Curactor to delete old indices:

I have a cron entry to run the curator once per day. Perhaps something
already exists in Marvel, not sure since I am not a user.

Cheers,

Ivan

On Mon, May 12, 2014 at 7:39 AM, deepakas deepak.subhramanian@gmail.comwrote:

Is there a way to set marvel to delete the marvel indices after 7 days. It
looks like Marvel is generating around 2 GB of data everyday. Our disk got
full 2 times because of Marvel data. Is there a way to reduce the amount of
data generated by marvel ?

Also is there any plan to add alert mechanisms in Marvel.
For example if the Marvel status goto red it will be good to get an email
for a specified user. I see Marvel status as red. But it doesnt show why it
is causing red. It will be good to get some alerts with the details when
the cluster status goes red.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1399905548133-4055729.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAq93AUa76%2BShoYdzRMVSq-m5Qzi%2B3utcdyaErRhA0FkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

1 Like
3

Hi,

Ivan's advice is correct. Currently, curator is the way to go. There is
also talk about building an index level TTL in elasticsearch but this is
not reality yet.

As to reducing data - first question is whether your using version 1.1.X of
marvel - this one has greatly reduced the data signature. If not, upgrading
would be the easiest option. O.w. you can also look at the interval
settings in the agent. You can get it up to 30s if you want to.

At the moment there is no alerting built into Marvel. You can use the
marvel data & easily write a nagios script (or similar) to add it to your
monitoring data.

Regarding RED status explanations - working on it - watch for the upcoming
release :slight_smile:

Cheers,
Boaz

On Monday, May 12, 2014 10:30:00 PM UTC+2, Ivan Brusic wrote:

I do not use Marvel, but another monitoring system built on top of
Elasticsearch. I use the Elasticsearch Curactor to delete old indices:
GitHub - elastic/curator: Curator: Tending your Elasticsearch indices

I have a cron entry to run the curator once per day. Perhaps something
already exists in Marvel, not sure since I am not a user.

Cheers,

Ivan

On Mon, May 12, 2014 at 7:39 AM, deepakas deepak.subhramanian@gmail.comwrote:

Is there a way to set marvel to delete the marvel indices after 7 days. It
looks like Marvel is generating around 2 GB of data everyday. Our disk
got
full 2 times because of Marvel data. Is there a way to reduce the amount
of
data generated by marvel ?

Also is there any plan to add alert mechanisms in Marvel.
For example if the Marvel status goto red it will be good to get an email
for a specified user. I see Marvel status as red. But it doesnt show why
it
is causing red. It will be good to get some alerts with the details when
the cluster status goes red.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1399905548133-4055729.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c446d35c-c3e5-4c5e-91e4-09f39c3e4b42%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Hi Ivan/Boaz,

Thanks for the responses. I will checkout Curator. I already got the latest version of marvel last week using this command.
bin/plugin -i elasticsearch/marvel/latest

Thanks, Deepak

5

Hi Ivan/Boaz,

I am getting some warning when I tried to delete the marvel index using
curator.

github]# /usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p
.marvel
2014-05-14T18:07:44.691 INFO main:333 Job
starting...
2014-05-14T18:07:44.692 INFO _new_conn:180 Starting new
HTTP connection (1): localhost
2014-05-14T18:07:44.694 INFO log_request_success:49 GET
http://localhost:9200/ [status:200 request:0.003s]
2014-05-14T18:07:44.695 INFO main:359 Deleting
indices older than 3 days...
2014-05-14T18:07:44.698 INFO log_request_success:49 GET
http://localhost:9200/.marvel*/_settings?expand_wildcards=closed[status:200
request:0.002s]
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.07
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.08
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.09
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.10
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.11
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.12
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.13
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.14
2014-05-14T18:07:44.723 INFO index_loop:309 DELETE index
operations completed.
2014-05-14T18:07:44.723 INFO main:379 Done in
0:00:00.037963.
[root@node05 github]# ls -ltr
/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel*
/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.10:
total 8
drwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 10 01:00 _state
drwxr-xr-x. 5 elasticsearch elasticsearch 4096 May 10 01:00 0

/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.11:

Any thoughts?

On Tue, May 13, 2014 at 12:26 PM, deepakas [via Elasticsearch Users] <
ml-node+s115913n4055784h39@n3.nabble.com> wrote:

Hi Ivan/Boaz,

Thanks for the responses. I will checkout Curator. I already got the
latest version of marvel last week using this command.
bin/plugin -i elasticsearch/marvel/latest

Thanks, Deepak

If you reply to this email, your message will be added to the discussion
below:

http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4055784.html
To unsubscribe from Marvel Indices taking lot of space ? Can we specify
automatic delete of marvel indice ?, click herehttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4055729&code=ZGVlcGFrLnN1YmhyYW1hbmlhbkBnbWFpbC5jb218NDA1NTcyOXwtNjg4NjI0OTI4
.
NAMLhttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers!nabble%3Aemail.naml-instant_emails!nabble%3Aemail.naml-send_instant_email!nabble%3Aemail.naml

--
Deepak Subhramanian

6

Deepak,

Can you try the command with a hyphen at the end -
/usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p .marvel-
For some reason, curator doesn't match the marvel indices when using
".marvel" as the prefix.

Regards,

On Thursday, May 15, 2014 1:14:14 AM UTC+8, Deepak Subhramanian wrote:

Hi Ivan/Boaz,

I am getting some warning when I tried to delete the marvel index using
curator.

github]# /usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p
.marvel
2014-05-14T18:07:44.691 INFO main:333 Job
starting...
2014-05-14T18:07:44.692 INFO _new_conn:180 Starting new
HTTP connection (1): localhost
2014-05-14T18:07:44.694 INFO log_request_success:49 GET
http://localhost:9200/ [status:200 request:0.003s]
2014-05-14T18:07:44.695 INFO main:359 Deleting
indices older than 3 days...
2014-05-14T18:07:44.698 INFO log_request_success:49 GET
http://localhost:9200/.marvel*/_settings?expand_wildcards=closed[status:200 request:0.002s]
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.07
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.08
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.09
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.10
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.11
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.12
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.13
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.14
2014-05-14T18:07:44.723 INFO index_loop:309 DELETE index
operations completed.
2014-05-14T18:07:44.723 INFO main:379 Done in
0:00:00.037963.
[root@node05 github]# ls -ltr
/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel*
/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.10:
total 8
drwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 10 01:00 _state
drwxr-xr-x. 5 elasticsearch elasticsearch 4096 May 10 01:00 0

/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.11:

Any thoughts?

On Tue, May 13, 2014 at 12:26 PM, deepakas [via Elasticsearch Users] <[hidden
email] http://user/SendEmail.jtp?type=node&node=4055895&i=0> wrote:

Hi Ivan/Boaz,

Thanks for the responses. I will checkout Curator. I already got the
latest version of marvel last week using this command.
bin/plugin -i elasticsearch/marvel/latest

Thanks, Deepak

If you reply to this email, your message will be added to the
discussion below:

http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4055784.html
To unsubscribe from Marvel Indices taking lot of space ? Can we specify
automatic delete of marvel indice ?, click here.
NAMLhttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers!nabble%3Aemail.naml-instant_emails!nabble%3Aemail.naml-send_instant_email!nabble%3Aemail.naml

--
Deepak Subhramanian

View this message in context: Re: Marvel Indices taking lot of space ?
Can we specify automatic delete of marvel indice ?http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4055895.html
Sent from the Elasticsearch Users mailing list archivehttp://elasticsearch-users.115913.n3.nabble.com/at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/36baa5ec-c467-47df-b597-0aee612dd888%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

7

Thanks Gokul. It worked after adding the hyphen.

On Mon, May 19, 2014 at 3:25 AM, Gokul A [via Elasticsearch Users] <
ml-node+s115913n4056054h9@n3.nabble.com> wrote:

Deepak,

Can you try the command with a hyphen at the end -
/usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p .marvel-
For some reason, curator doesn't match the marvel indices when using
".marvel" as the prefix.

Regards,

On Thursday, May 15, 2014 1:14:14 AM UTC+8, Deepak Subhramanian wrote:

Hi Ivan/Boaz,

I am getting some warning when I tried to delete the marvel index using
curator.

github]# /usr/lib/python2.6/site-packages/curator/curator.py -d 3 -p
.marvel
2014-05-14T18:07:44.691 INFO main:333 Job
starting...
2014-05-14T18:07:44.692 INFO _new_conn:180 Starting
new HTTP connection (1): localhost
2014-05-14T18:07:44.694 INFO log_request_success:49 GET
http://localhost:9200/ [status:200 request:0.003s]
2014-05-14T18:07:44.695 INFO main:359 Deleting
indices older than 3 days...
2014-05-14T18:07:44.698 INFO log_request_success:49 GET
http://localhost:9200/.marvel*/_settings?expand_wildcards=closed
[status:200 request:0.002s]
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.07
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.08
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.09
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.10
2014-05-14T18:07:44.722 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.11
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.12
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.13
2014-05-14T18:07:44.723 ERROR find_expired_indices:201 Could not
find a valid timestamp from the index: .marvel-2014.05.14
2014-05-14T18:07:44.723 INFO index_loop:309 DELETE
index operations completed.
2014-05-14T18:07:44.723 INFO main:379 Done in
0:00:00.037963.
[root@node05 github]# ls -ltr /var/lib/elasticsearch/
bigdatadev/nodes/0/indices/.marvel*
/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.10:
total 8
drwxr-xr-x. 2 elasticsearch elasticsearch 4096 May 10 01:00 _state
drwxr-xr-x. 5 elasticsearch elasticsearch 4096 May 10 01:00 0

/var/lib/elasticsearch/bigdatadev/nodes/0/indices/.marvel-2014.05.11:

Any thoughts?

On Tue, May 13, 2014 at 12:26 PM, deepakas [via Elasticsearch Users] <[hidden email]>wrote:

Hi Ivan/Boaz,

Thanks for the responses. I will checkout Curator. I already got the
latest version of marvel last week using this command.
bin/plugin -i elasticsearch/marvel/latest

Thanks, Deepak

If you reply to this email, your message will be added to the
discussion below:

http://elasticsearch-users.115913.n3.nabble.com/Marvel-
Indices-taking-lot-of-space-Can-we-specify-automatic-
delete-of-marvel-indice-tp4055729p4055784.html
To unsubscribe from Marvel Indices taking lot of space ? Can we specify
automatic delete of marvel indice ?, click here.
NAML

--
Deepak Subhramanian

View this message in context: Re: Marvel Indices taking lot of space ? Can we specify automatic
delete of marvel indice ?
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [hidden email]http://user/SendEmail.jtp?type=node&node=4056054&i=0
.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/36baa5ec-c467-47df-b597-0aee612dd888%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/36baa5ec-c467-47df-b597-0aee612dd888%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

If you reply to this email, your message will be added to the discussion
below:

http://elasticsearch-users.115913.n3.nabble.com/Marvel-Indices-taking-lot-of-space-Can-we-specify-automatic-delete-of-marvel-indice-tp4055729p4056054.html
To unsubscribe from Marvel Indices taking lot of space ? Can we specify
automatic delete of marvel indice ?, click herehttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4055729&code=ZGVlcGFrLnN1YmhyYW1hbmlhbkBnbWFpbC5jb218NDA1NTcyOXwtNjg4NjI0OTI4
.
NAMLhttp://elasticsearch-users.115913.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html!nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers!nabble%3Aemail.naml-instant_emails!nabble%3Aemail.naml-send_instant_email!nabble%3Aemail.naml

--
Deepak Subhramanian