Hi,
We've run into a scenario where our data disk space usage has increased drastically in a short period of time. We've determined that the increase is due to data we're indexing (the pattern and trend for indices.store.size_in_bytes by node is inversely correlated with the fs.data.available_in_bytes by node). We'd like to be able to quickly determine the % of growth over time by index type. So, let's say we have these indices per day:
indexA_date1
indexA_date2
indexA_date3
. . .
IndexB_date1
IndexB_date2
IndexB_date3
I'd like to be able to get the sum of the indices.store.size by index type (so would need to be able to specify indexname="IndexA*" (to get for all dates) over time. What would be the best way to do that?
I tested to see if I could do an agg on the _cat API with the below -- not knowing ahead of time if it was even possible -- but got "IndexMissingException[[_cat] missing]"
GET _cat/indices
{
"aggs": {
"biggest_indices": {
"max": {
"field": "store.size"
}
}
}
}
I also tried in Kibana, but do not see a way to make that work. We got close to what we want with the below, but, again, we'd would like it by <index*> instead of by node:
{
"type": "line",
"params": {
"shareYAxis": true,
"addTooltip": true,
"addLegend": true,
"showCircles": true,
"smoothLines": false,
"interpolate": "linear",
"scale": "linear",
"drawLinesBetweenPoints": true,
"radiusRatio": 9,
"times": [],
"addTimeMarker": false,
"defaultYExtents": false,
"setYExtents": false,
"yAxis": {}
},
"aggs": [
{
"id": "1",
"type": "avg",
"schema": "metric",
"params": {
"field": "indices.store.size_in_bytes"
}
},
{
"id": "2",
"type": "date_histogram",
"schema": "segment",
"params": {
"field": "@timestamp",
"interval": "m",
"customInterval": "2h",
"min_doc_count": 1,
"extended_bounds": {}
}
},
{
"id": "3",
"type": "terms",
"schema": "group",
"params": {
"field": "node.name",
"size": 15,
"order": "desc",
"orderBy": "1"
}
}
],
"listeners": {}
}
Any ideas?