Ingesting Data via Python to Elastic

Elastic Stack Elasticsearch
#1

Hello Team
I need help in ingesting the data via python into elastic. I have a cert that I have already uploaded the cert on the browser. After pre-processing is completed, I converted dataframe into dict. Now I want to ingest this into elastic. Need help on the syntax. Usually when cert is not there you ingest directly by entering IP and the auth. Need help when we have cert.
How to edit the below syntax when we have cert and what library to be used apart from elasticsearch

documents = df.to_dict(orient='records')

es = Elasticsearch(hosts=['I have the HOST'],timeout=600, http_auth=('USERID', 'PWD') )

bulk(es, documents, index='index_name',doc_type='doc', raise_on_error=False, ignore=400)

#2

Are you asking how to make the client use HTTPS when connecting to Elasticsearch?

#3

Hello Mark
Yes..Using https...
I used the below code after converting the dataframe in dict

context = create_default_context(capath="Path of the .crt file")
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE

es = Elasticsearch(
['HOST'],
http_auth=('USEDID', 'PASSWORD'),
scheme="https",
port=443,
ssl_context=context
)

But still getting the error
ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPSConnectionPool(host=HOSTNAME', port=443): Read timed out. (read timeout=10))

Elastic Instance is up

#4

Used the below Link
https://elasticsearch-py.readthedocs.io/en/master/

#5

You can do a HTTPS curl against the Elasticsearch instance successfully?

#6

Yes..its sucessful

#7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.