# How to parse ES json output. Scripting, re-indexing, other suggestion?

**URL:** <https://discuss.elastic.co/t/how-to-parse-es-json-output-scripting-re-indexing-other-suggestion/95953>\
**Category:** Elasticsearch\
**Created:** [August 4, 2017, 8:51pm UTC](https://discuss.elastic.co/t/how-to-parse-es-json-output-scripting-re-indexing-other-suggestion/95953 "2017-08-04T20:51:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![smcooper](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@smcooper](https://discuss.elastic.co/u/smcooper)\
**Post date:** [August 4, 2017, 8:51pm UTC](https://discuss.elastic.co/t/how-to-parse-es-json-output-scripting-re-indexing-other-suggestion/95953/1 "2017-08-04T20:51:38Z")

</div>

Hi,

I am very new to the stack and need some advice.  
I have been trying to make sense out of our logs, imported to ES using logstash, and generating graphs (and queries) from kibana.  
I ran into some difficulties when I started to look for a way to [visualize some success/failure ratio](https://discuss.elastic.co/t/trying-to-vizualize-top-n-ordered-ratios/95747).  
As I can't get a readable visualization in kibana for this problem, I decided to write the Elasticsearch query using the Bucket Script Aggregations needed to get this information.

I now have the information I was looking for, in a nice and very long JSON format, ready to be parsed and sorted.  
Hundreds of blocks like this:

> ```
> ...
> {
> "key": "my_institution_name",
> "doc_count": 153255,
> "Response_success": {
> "doc_count": 116924
> },
> "failure_ratio": {
> "value": 23.706
> }
> },
> ...
> 
> ```

My question is what are my options to use this elasticsearch output and actually make something readable out of it?  
My first idea is to write a script to parse it all and create a sorted table Institutions - Failure ratio.

Do you have some alternatives that would make more sense?  
Should I (/ is there a good way ) to create an index out of the result and to use it into kibana? The original idea was to make a graph, it seems a bit conterproductive to skip kibana, parse the thing by hand and plot the result... :-/

---

<div class="post-metadata">

**Author:** ![Court](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/court/32/6640_2.png) [@Court](https://discuss.elastic.co/u/Court)\
**Post date:** [August 11, 2017, 7:55pm UTC](https://discuss.elastic.co/t/how-to-parse-es-json-output-scripting-re-indexing-other-suggestion/95953/2 "2017-08-11T19:55:49Z")

</div>

It does seem a bit counterproductive to just skip Kibana altogether. Your proposal about indexing the data in a format that can be easily handled with visualizations in Kibana seems like a reasonable workaround until something like [https://github.com/elastic/kibana/issues/4707](https://github.com/elastic/kibana/issues/4707) can be completed. I've seen people do something similar before, so it isn't unprecedented.

Something like [watcher](https://www.elastic.co/guide/en/x-pack/current/watcher-getting-started.html) can be used to calculate updated ratios on an interval.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 8, 2017, 7:55pm UTC](https://discuss.elastic.co/t/how-to-parse-es-json-output-scripting-re-indexing-other-suggestion/95953/3 "2017-09-08T19:55:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.