Why is such a difference between the time of a query request within the kibana console and the query request from a python script

Elastic Stack Elasticsearch
1

I am trying to retrieve a lot of data from kibana using a python script. But I get unsatisfactory query time of execution, so I tried querying from within the dev_tools/console inside kibana. Doing a DSL query from console, it takes at a maximum of 15 secs in all cases to retrieve results.

From the python script I am trying to get the data using /_search?scroll=5m because I've read it is better to use scroll (( Scroll [ The Scroll api is recommended for efficient deep scrolling but scroll contexts are costly and it is not recommended to use it for real time user requests.] )) as I just want to retrieve data, not real time requests. (or am I wrong?)

Within the Python script it takes ~3 mins at least using requests module.

The DSL queries used for each are the same. The difference is in the urls used:

  • python using requests module: kibana_site:some_port_1/some_index/_search?scroll=5m
  • dev console: kibana_site:some_port_2/app/kibana#/dev_tools/console?_g=()
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.