How to graph automatically those data

[florentmair (Florent)]

Hi everyone,

I need some advices to graph some data.
Every day at 10:30pm i have an audit of our docbase, the results is the number of documents by type, by format, ...

Those data are stored into ES like that :

{
  "_index": "logstash-dmt",
  "_type": "stateofdocbase",
  "_id": "AVrJTMRHPMa1Ahhz2UYM",
  "_score": null,
  "_source": {
    "IndexName": "logstash-dmt",
    "@timestamp": "2017-03-12T21:30:02.000Z",
    "NB_DOCUMENTS_BY_FORMAT": {
      "bmp": "1",
      "csv": "661",
      "msw6template": "10",
      "java": "4",
      "excel12bbook": "34",
      "excel12mebook": "1361",
      "PDF_StampTest": "4",
      "text": "12232",
      "excel5book": "8",
      "powerpoint": "2",
      "zip": "22678",
      "excel12book": "3798",
      "ppt8_template": "41",
      "ppt12": "3976",
      "excel8book": "821179",
      "ppt8": "26006",
      "pdf": "427521"
    },
    "@version": "1",
    "host": "XXX",
    "IndexRotation": "none",
    "type": "stateofdocbase",
    "IndexType": "dmt",
    "tags": [
      "multiline"
    ]
  },
  "fields": {
    "@timestamp": [
      1489354202000
    ]
  },
  "sort": [
    1489354202000
  ]
} 

is it possible to make a chart with all "NB_DOCUMENTS_BY_FORMAT.*" like a Pie chart.

Thanks

[shaunak (Shaunak Kashyap)]

Kibana charts are generated by performing aggregations on documents in Elasticsearch. Using your example, this means that there would need to be an Elasticsearch document for each NB_DOCUMENTS_BY_FORMAT.* format, so something like this (3 example documents shown):

Document #1:

{
  "format": "bmp",
  "count": 1
}

Document #2:

{
  "format": "csv",
  "count": 661
}

Document #3:

{
  "format": "msw6template",
  "count": 10
}

Then you could create a pie chart showing the various formats and their total counts like this:

[florentmair (Florent)]

Thanks for your response, it seems i need to change my method to integrate values into ES.
I have already asked for a solution here Need advices to ingest this type of data

maybe you will have an idea to improve my method