so i'd like to start visualize some of the data from the _cat apis like indices, volume and replicas, etc.
tried to see if i could scrape the info from .monitoring-es but no dice, is the data from the api somewhere we can pull it from?
hey,
how about using the metricbeat elasticsearch module for this, so you do not have any hassle of parsing things manually?
tried it and it gave me some good stats but it seems like the module doesn't do index and index_summary on 6.2 cluster so i'm gonna have to scrape that from somewhere else.
which internal index may contain the index and store size data im looking for?
You could try the _cat/indices API, it gives you all the indices with store size, primary store size, docs count and docs deleted count.
For instance, this will return all the indices sorted by name with column headers explaining what each column represents:
curl -XGET 'http://localhost:9200/_cat/indices?v&s=index'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .kibana-6 x4RoWEFISwO_5Kwk5Ip8hw 1 1 5 1 106.9kb 53.4kb
green open .kibana_1 EC-zsVoLQBi6dNTwWDuJeg 1 1 4 0 46.5kb 23.2kb
green open .kibana_7 Qci1vY8nTMacksaVpJSgSg 1 1 8 0 57.3kb 28.6kb
green open .monitoring-es-6-2019.08.07 wcvM-3KvR8KDujQV5Y7Wwg 1 1 503096 4048 925.9mb 459.8mb
green open .monitoring-es-6-2019.08.08 Dv-MkmD9QPSr1JW5FVhjkA 1 1 503090 4948 940.5mb 468.5mb
green open .monitoring-es-6-2019.08.09 qBAAVWc4TS-H8v1Ao3ClUg 1 1 502648 4418 925.1mb 467.5mb
green open .monitoring-es-6-2019.08.10 drYDRgmGRp6OqaVTc1cdyw 1 1 502158 4496 929.4mb 461.3mb
It should be straight forward to parse this output either in a script or on command line using grep and awk. The following piped commands return the store.size for the indices created on Aug 14:
curl -XGET 'http://localhost:9200/_cat/indices?v&s=index' -s | grep 2019.08.14 | awk '{print $3" has store.size: "$9}'
.monitoring-es-6-2019.08.14 has store.size: 717.9mb
.monitoring-kibana-6-2019.08.14 has store.size: 3.4mb
right, im aware of the cat api results but having a script to hit the api and then reingest it into the cluster seems redundant, if the api has the data it must be somewhere already which is what im trying to get so i can start doing some visualizations, im mostly looking into displaying a graph of indices sizes so i can see volume peaks and trends.
Sorry, didn't see the title of this thread, just read the posts. From them it seems you should be able to use _cat/indices to obtain store sizes for each index.
Oh, I get your point.
I don't know if there is a single index you can query for this. I have only used the _cat/indices myself. Sorry.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.