# How to build indexes for Object Storage Statistics collection **URL:** \ **Category:** Elasticsearch\ **Created:** [July 3, 2016, 9:27am UTC](https://discuss.elastic.co/t/how-to-build-indexes-for-object-storage-statistics-collection/54594 "2016-07-03T09:27:40Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![ranp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ranp/32/10693_2.png) [@ranp](https://discuss.elastic.co/u/ranp)\ **Post date:** [July 3, 2016, 9:27am UTC](https://discuss.elastic.co/t/how-to-build-indexes-for-object-storage-statistics-collection/54594/1 "2016-07-03T09:27:40Z")
Hi Friends, Newbie to ELK, so patience with me. I have a grid product (Object storage) which is a cluster that is compound of nodes. It is based on nodes (server with disks) grouped into Zones, and the data is stored using object policies with replicas. The cluster has a HTTP API to where I can get its info & stats using HTTP calls & get json replies, which are easy to push into elasticsearch. The problem is that the product doesn't have historical logging, and I want to use elastic-search to store (mainly statistics historically) that data over time & be able to present it using Kibana. The bricks that I can get via the HTTP calls are: 0 - Cluster info 1 - Node info 2 - Zones 3 - Policies info 4 - Statistics summary cluster (sum of all nodes) 5 - Statistics detailed cluster (sum of all nodes) 6 - Statistics detailed per node Most will remain static, (will check updates daily) and what is the most dynamic and I plan to collect every 5min is #6. my plan is to wget/curl the per node statistics every 5min (returns json output, see below) and push it into elasticsearch. Given the above, how would you recommend I build the indexes, so that I can easily use Kibana to build a nice dashboard for historical table/graphic view ? * * * wget -q --user=XXX --password=XXX [http://10.12.11.51:8088/mgmt/statistics\_detail?address=10.12.11.51](http://10.12.11.51:8088/mgmt/statistics_detail?address=10.12.11.51) -O - | /usr/bin/python -mjson.tool \> statistics\_detail\_single { "avgDelLatency": [ 0, "ms" ], "avgGetLatency": [ 0, "ms" ], "avgPutLatency": [ 0, "ms" ], "fileDeletesPerSec": [ 0, "FDPS" ], "fileReadsPerSec": [ 0, "FRPS" ], "fileWritesPerSec": [ 0, "FWPS" ], "getThroughput": [ 0.0, "MB/s" ], "putThroughput": [ 0.0, "MB/s" ], "totalDelFailureCount": [ 0, "requests" ], "totalDelSuccessCount": [ 0, "requests" ], "totalGetFailureCount": [ 0, "requests" ], "totalGetSuccessCount": [ 0, "requests" ], "totalPutFailureCount": [ 0, "requests" ], "totalPutSuccessCount": [ 66761, "requests" ] } ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 3, 2016, 9:49pm UTC](https://discuss.elastic.co/t/how-to-build-indexes-for-object-storage-statistics-collection/54594/2 "2016-07-03T21:49:59Z")
I'd just create daily/weekly indices and put it in as time based data. ---
**Author:** ![ranp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ranp/32/10693_2.png) [@ranp](https://discuss.elastic.co/u/ranp)\ **Post date:** [July 4, 2016, 11:39am UTC](https://discuss.elastic.co/t/how-to-build-indexes-for-object-storage-statistics-collection/54594/3 "2016-07-04T11:39:16Z")
Hi Mark, What do you mean by indices ? Indexs ? Create a Index for a day/week, and than each time-stamp will be an entry ? But Say I have 4 nodes, and for each node I need to collect, so I create 4 x Day indexes ? I thought to make an index for each node, and than each entry will have the statistics, and a parameter in the statistic json would be the date/time, no ? Do you have any sample configuration implementation which I can review & get an idea ? Thanks ! Ran ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 4, 2016, 10:47pm UTC](https://discuss.elastic.co/t/how-to-build-indexes-for-object-storage-statistics-collection/54594/4 "2016-07-04T22:47:16Z")
Indices = more than one index. > [@ranp](#): > > Create a Index for a day/week, and than each time-stamp will be an entry ? Yes. > [@ranp](#): > > But Say I have 4 nodes, and for each node I need to collect, so I create 4 x Day indexes ? Depends, is the datas structure the same for all 4? If so then just one index is fine. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:38pm UTC](https://discuss.elastic.co/t/how-to-build-indexes-for-object-storage-statistics-collection/54594/5 "2017-07-05T22:38:27Z")