I am trying to retrieve a lot of data from kibana using a python script. But I get unsatisfactory query time of execution, so I tried querying from within the dev_tools/console inside kibana. Doing a DSL query from console, it takes at a maximum of 15 secs in all cases to retrieve results.
From the python script I am trying to get the data using /_search?scroll=5m because I've read it is better to use scroll (( Scroll [ The Scroll api is recommended for efficient deep scrolling but scroll contexts are costly and it is not recommended to use it for real time user requests.] )) as I just want to retrieve data, not real time requests. (or am I wrong?)
Within the Python script it takes ~3 mins at least using requests module.
The DSL queries used for each are the same. The difference is in the urls used:
- python using
requestsmodule:kibana_site:some_port_1/some_index/_search?scroll=5m - dev console:
kibana_site:some_port_2/app/kibana#/dev_tools/console?_g=()