I'm managing a pretty badass 11 node Elasticsearch cluster that is powering
a customer facing dashboard reporting platform. 20 cores per node, 64GB
RAM, SSDs, Dual 10 GbE of awesome. I evaluated Marvel while we were still
in development on the new platform and I found it to be a very valuable
tool. At first Marvel was indexing to the same cluster we were monitoring
and this was okay while we were in development as there were plenty of
extra cycles in the cluster to handle the load but now that we are in
production it doesn't make sense to burden the cluster with this. The
nature of our reporting system requires us to to have an index for each
customer so we're currently at 328 indexes and over 10,000 shards total.
The amount of data indexed by Marvel increases dramatically as the number
of indices increases so once we got over 300 indices in the system the
daily marvel index ended up at around 400 GB replicated and was indexing
around 2,000 documents a second by itself.
What I want to do is have Marvel index to a not as awesome 2 node
Elasticsearch monitoring cluster. 12 cores, 64 GB RAM and spinning disks.
But in practice these 2 nodes are unable to keep up with the load and get
completely bogged down. I'm thinking I can sacrifice redundancy and buy
myself some cycles by not using any replicas on the Marvel index. My other
idea is to set marvel.agent.interval from the default 10s to something like
30s on the assumption that this will cut the amount of data generated by a
third. Does this sound sane or do you have anyone have other ideas on what
I can try to limited the load?
Controls the interval between data samples. Defaults to 10s. Set to -1 to
temporarily disable exporting.
This setting is update-able via the Cluster Update Settings API.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firstname.lastname@example.org.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5884045a-49f7-48d4-a3cb-93a5f70c53cf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.