# Marvel Shows Documents deleted **URL:** \ **Category:** Elasticsearch\ **Created:** [April 3, 2014, 12:43pm UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787 "2014-04-03T12:43:49Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![BornGenius](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@BornGenius](https://discuss.elastic.co/u/BornGenius)\ **Post date:** [April 3, 2014, 12:43pm UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/1 "2014-04-03T12:43:49Z")
Hi all, Our company has decided to use ElasticSearch for saving some critical data . We are using Marvel for real-time monitoring the cluster. Specifically we have a 9 node cluster with 3 Master Nodes, 2 Search Load Balancers and 4 Data Nodes . Additionally we have a 2 node cluster which has Marvel Indexes and is being used for monitoring the main cluster. We are facing various issues which are as follows : a)Although both the clusters (Production as well as Monitoring) are in the same Subnet still any changes made to the production cluster is not reflected immediately in Marvel Dashboard. Hence we are not able to do a real-time monitoring of the cluster. b)Although we are not deleting a single document still the DELETED DOCUMENTS graph on the node statistics dashboard shows spikes which is again a concern for us. For the reference I am attaching the screenshots please find the same in the attachments. Please can someone help with the above mentioned issues. Thanks AryanJ -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 3, 2014, 9:27pm UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/2 "2014-04-03T21:27:07Z")
FYI you are required to license Marvel with that number of nodes - [http://www.elasticsearch.com/marvel](http://www.elasticsearch.com/marvel) What sort of time differences are you seeing between changes and the display? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 3 April 2014 23:43, AryanJ [aman\_aryan\_jolly@yahoo.com](mailto:aman_aryan_jolly@yahoo.com) wrote: > Hi all, > > Our company has decided to use Elasticsearch for saving some critical data > . We are using Marvel for real-time monitoring the cluster. > > Specifically we have a 9 node cluster with 3 Master Nodes, 2 Search Load > Balancers and 4 Data Nodes . Additionally we have a 2 node cluster which > has Marvel Indexes and is being used for monitoring the main cluster. > We are facing various issues which are as follows : > > a)Although both the clusters (Production as well as Monitoring) are in the > same Subnet still any changes made to the production cluster is not > reflected immediately in Marvel Dashboard. Hence we are not able to do a > real-time monitoring of the cluster. > > b)Although we are not deleting a single document still the DELETED > DOCUMENTS graph on the node statistics dashboard shows spikes which is > again a concern for us. > > For the reference I am attaching the screenshots please find the same in > the attachments. > > Please can someone help with the above mentioned issues. > > Thanks > > AryanJ > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YTo7CwHQV5Ct4n1Qx1Y440GqM7Vr0--7GCBgwovhBxKA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YTo7CwHQV5Ct4n1Qx1Y440GqM7Vr0--7GCBgwovhBxKA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [April 3, 2014, 9:29pm UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/3 "2014-04-03T21:29:31Z")
Are you updating documents? Lucene does not update documents directly, but does a delete/insert. -- Ivan On Thu, Apr 3, 2014 at 2:27 PM, Mark Walkom [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)wrote: > FYI you are required to license Marvel with that number of nodes - > [Monitor and Manage Elasticsearch | Elastic](http://www.elasticsearch.com/marvel) > > What sort of time differences are you seeing between changes and the > display? > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 3 April 2014 23:43, AryanJ [aman\_aryan\_jolly@yahoo.com](mailto:aman_aryan_jolly@yahoo.com) wrote: > > > Hi all, > > > > Our company has decided to use Elasticsearch for saving some critical > > data . We are using Marvel for real-time monitoring the cluster. > > > > Specifically we have a 9 node cluster with 3 Master Nodes, 2 Search Load > > Balancers and 4 Data Nodes . Additionally we have a 2 node cluster which > > has Marvel Indexes and is being used for monitoring the main cluster. > > We are facing various issues which are as follows : > > > > a)Although both the clusters (Production as well as Monitoring) are in > > the same Subnet still any changes made to the production cluster is not > > reflected immediately in Marvel Dashboard. Hence we are not able to do a > > real-time monitoring of the cluster. > > > > b)Although we are not deleting a single document still the DELETED > > DOCUMENTS graph on the node statistics dashboard shows spikes which is > > again a concern for us. > > > > For the reference I am attaching the screenshots please find the same in > > the attachments. > > > > Please can someone help with the above mentioned issues. > > > > Thanks > > > > AryanJ > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/465b5378-0ee2-4720-96d8-f111cba5d7bb%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEM624YTo7CwHQV5Ct4n1Qx1Y440GqM7Vr0--7GCBgwovhBxKA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YTo7CwHQV5Ct4n1Qx1Y440GqM7Vr0--7GCBgwovhBxKA%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAEM624YTo7CwHQV5Ct4n1Qx1Y440GqM7Vr0--7GCBgwovhBxKA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEM624YTo7CwHQV5Ct4n1Qx1Y440GqM7Vr0--7GCBgwovhBxKA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDP60zh3Bhst3k2YdWA1ayS%3DR7k6bVgNdG1\_ss8Mp6r\_A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDP60zh3Bhst3k2YdWA1ayS%3DR7k6bVgNdG1_ss8Mp6r_A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![BornGenius](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@BornGenius](https://discuss.elastic.co/u/BornGenius)\ **Post date:** [April 4, 2014, 4:30am UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/4 "2014-04-04T04:30:45Z")
Hi Mark, Thanks for the reply.I have mentioned in my post that we have DECIDED to use ,actually we have still not started using it in production .What I mentioned is just a prototype that we have decided to use in production and doing the final non-functional testing on the developer side.I read it on the site that we have to purchase license when we start using it commercially right ? And the problem is that while testing if we shut-down any of the data nodes or master's or SLB's ,for around an hour or so, Marvel still shows that those nodes are available as it is visible in the screenshot. As we need a near real-time monitoring so this will not help. Thanks AryanJ ---
**Author:** ![BornGenius](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@BornGenius](https://discuss.elastic.co/u/BornGenius)\ **Post date:** [April 4, 2014, 5:04am UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/5 "2014-04-04T05:04:11Z")
Hi Ivan, Thanks for the reply.We are not deleting any of the documents what we are doing is just inserting records and generating reports by using aggregation API.As I mentioned that the data we are putting in ElasticSearch is critical and we can't afford to loose it ,that's why Marvel showing spikes in the **DELETED DOCUMENTS** graph is of concern. Thanks AryanJ ---
**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\ **Post date:** [April 12, 2014, 6:28pm UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/6 "2014-04-12T18:28:04Z")
Hi Arjan, Are you updating documents? when reindex a document lucene deletes the old copies and indexes new ones. Those deletes count for the chart. After a merge happens they are removed. Regarding shutting down nodes for \>1h. When marvel doesn't hear from a node for more than 1m, the node's data is considered stale and the row will be grayed out with an information icon over which you can hover and see how long the node has not reported. I see that your screenshot. What's interesting is that after 10m those should be removed completed. Was that not the case? what version of Marvel are you using? Cheers, Boaz On Friday, April 4, 2014 6:30:45 AM UTC+2, AryanJ wrote: > Hi Mark, > Thanks for the reply.I have mentioned in my post that we have DECIDED to > use > ,actually we have still not started using it in production .What I > mentioned > is just a prototype that we have decided to use in production and doing > the > final non-functional testing on the developer side.I read it on the site > that we have to purchase license when we start using it commercially right > ? > > And the problem is that while testing if we shut-down any of the data > nodes > or master's or SLB's ,for around an hour or so, Marvel still shows that > those nodes are available as it is visible in the screenshot. As we need a > near real-time monitoring so this will not help. > > Thanks > AryanJ > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Marvel-Shows-Documents-deleted-tp4053389p4053483.html](http://elasticsearch-users.115913.n3.nabble.com/Marvel-Shows-Documents-deleted-tp4053389p4053483.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/182f1197-74c2-4c7e-a52e-18fefb6e2345%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/182f1197-74c2-4c7e-a52e-18fefb6e2345%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![BornGenius](https://avatars.discourse-cdn.com/v4/letter/b/e19b73/32.png) [@BornGenius](https://discuss.elastic.co/u/BornGenius)\ **Post date:** [April 14, 2014, 9:53am UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/7 "2014-04-14T09:53:27Z")
Hi Boaz, Thanks for the reply . We are not deleting or updating any documents . Actually we are able to figure that out .What is happening is that Deleted Documents Graph shows some spike for a particular node whenever we stop that node. We are assuming that when a node goes down the shards are moved around for re-balancing, hence data would be deleted from one node and moved to other this is what shows spikes. It takes some time to update the status of the the nodes.I am not sure if it is exactly 10 min or not. I am not sure about the marvel version ☹ what is know is that it should be the latest one ..as we installed it using bin/plugin -i elasticsearch/marvel/latest Where can i find that ????? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:36am UTC](https://discuss.elastic.co/t/marvel-shows-documents-deleted/16787/8 "2017-07-06T01:36:01Z")