# PANDAS or Ipython Notebook integration into kibana

**URL:** <https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136>\
**Category:** Kibana\
**Created:** [April 6, 2018, 7:53pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136 "2018-04-06T19:53:18Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mat\_lon](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mat\_lon](https://discuss.elastic.co/u/mat_lon)\
**Post date:** [April 6, 2018, 7:53pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/1 "2018-04-06T19:53:18Z")

</div>

Is there any current thoughts on taking output from a kibana query, and being able to pipe it or take it to an ipython notebook? I am highly interested in this feature, or potentially adding a feature to allow the output of a query be piped to a python function/library for quick stats. primarily, i am itnerested in taking the output of the search and piping it to a pandas dataframe for quick statistics on certain fields.

---

<div class="post-metadata">

**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\
**Post date:** [April 7, 2018, 7:58am UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/2 "2018-04-07T07:58:59Z")

</div>

Python has an elasticsearch library that can be used in jupyter or ipython notebooks. Does that not help in your use case?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 7, 2018, 8:38am UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/3 "2018-04-07T08:38:54Z")

</div>

It's an interesting idea, it'd should be possible as a plugin.

---

<div class="post-metadata">

**Author:** ![mat\_lon](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mat\_lon](https://discuss.elastic.co/u/mat_lon)\
**Post date:** [April 7, 2018, 4:27pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/4 "2018-04-07T16:27:21Z")

</div>

Im more so trying to give a functionality to the search feature of kibana, similar to splunks ability to pass data to a stats function like below:

splunk: search mystuff | stats count by field1, field 2

thoughts for kibana: search mystuff | (pandas df).countby('field1, field2')

It looks as though splunk does something similar because they take the output of their search and pass it to another page (statistics tab), im assuming is just a version of output from a function.

If we could do something similar, we could be able to pass the data to python, then we would have very similar abilities to conduct basic stats, evaluations of new fields, splitting fields, regex support and what not. pandas supports 99% of the use cases i have seen in this capability and i think passing the search output from elk to a pandas dataframe instead of offering some app to do so, would make it inherently faster and more efficient for a majority of the users.

---

<div class="post-metadata">

**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\
**Post date:** [April 7, 2018, 6:05pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/5 "2018-04-07T18:05:00Z")

</div>

That is an interesting take. I believe an app similar to Timelion can be developed specifically for Data Science support in Kibana. This would be an interesting way.

I also use Kibana simply for dash-boarding and I use pandas, numpy, and scipy to do all the analysis, and send the data back to another index in elasticsearch. I think a major area where the ELK stack lacks is the analysis component. I hope Canvas will solve some of the problems in that area.

---

<div class="post-metadata">

**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\
**Post date:** [April 7, 2018, 6:07pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/6 "2018-04-07T18:07:37Z")

</div>

> [@mat\_lon](#):
>
> search mystuff | (pandas df).countby('field1, field2')

I also believe that this would be helpful. Kibana, also gives the data as a dictionary, pandas can easily create a dataframe from the dictionary.

Maybe open an enhancement request on Github?

---

<div class="post-metadata">

**Author:** ![mat\_lon](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mat\_lon](https://discuss.elastic.co/u/mat_lon)\
**Post date:** [April 7, 2018, 6:56pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/7 "2018-04-07T18:56:26Z")

</div>

> <https://github.com/elastic/kibana/issues/17602>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 5, 2018, 6:56pm UTC](https://discuss.elastic.co/t/pandas-or-ipython-notebook-integration-into-kibana/127136/8 "2018-05-05T18:56:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
