Kibana crashes during getting all documents from Elasticsearch


(Summer) #1

Dear all,

I would like to know how to get all the documents in Elasticsearch? I know that the default value of getting the elasticsearch was 10 and can still get up to 10,000. It seems that Kibana cannot handle the large number of data whenever I'm querying to get data. Is there any way how to get all the documents without affecting the Kibana or gettin' errors? Thank you so much.

I also found another approach to get all the data from elasticsearch using Python. Now my problem is how would I connect it to Elasticsearch? I tried using Python method and i got these errors. Can anyone help me on this?

GET YOURHOST.us-east-1.es.amazonaws.com [status:404 request:1.197s]
Traceback (most recent call last):
File "<pyshell#19>", line 1, in
es.search(index="my_index-2018.02.06")
File "C:\Users\summer.lively\AppData\Local\Programs\Python\Python37\lib\site-packages\elasticsearch\client\utils.py", line 73, in wrapped
return func(*args, params=params, **kwargs)
File "C:\Users\summer.lively\AppData\Local\Programs\Python\Python37\lib\site-packages\elasticsearch\client_init.py", line 632, in search
doc_type, '_search'), params=params, body=body)
File "C:\Users\summer.lively\AppData\Local\Programs\Python\Python37\lib\site-packages\elasticsearch\transport.py", line 312, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "C:\Users\summer.lively\AppData\Local\Programs\Python\Python37\lib\site-packages\elasticsearch\connection\http_requests.py", line 90, in perform_request
self._raise_error(response.status_code, raw_data)
File "C:\Users\summer.lively\AppData\Local\Programs\Python\Python37\lib\site-packages\elasticsearch\connection\base.py", line 125, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.NotFoundError: TransportError(404, 'index_not_found_exception', 'no such index')

Thank you all.


(Marius Dragomir) #2

For Kibana this is not really feasible, as it's limited by the maximum payload it can get from Elasticsearch and on how it can process all the data. This is why Kibana usually asks for aggregations from Elasticsearch and Elasticsearch does all the heavy workload.

As for the Python issue, that is a question for the Elasticsearch forums, here: https://discuss.elastic.co/c/elasticsearch


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.