Index keeps getting deleted

Hi,

I am new to elasticsearch.
I need to make 140 million entries in elasticsearch.
For that i have to keep elasticsearch running.
Twice it happened that in the middle of ingesting data, when I am halfway to the number of entries to be ingested, my index gets deleted.

I don't know how its happening, but it is delaying my project.
Even after ingesting the data, i need to keep data in elasticsearch atleast for a month.

There is no such policy of of deleting.
How to stop it from deleting the index ?

Can you see the index being mentioned in the Elasticsearch logs?
Do you have access control enabled?

1 Like

Hi @shivang.ahd Sorry that you are having troubles.

First there is no default policy, action, script etc in Elasticsearch that deletes indices automatically or on some condition like running out of disk space... in fact a common is is HOW to manage and delete indices / data... not the other way around.

To help you will need to provide considerably more detail.

What Version of the Stack are you Running?

How did you install it?

Where and how are are you running it?

Did you secure it?

Did you configure Index Lifecycle Management?

What kind of Host RAM / CPU etc are you running it on?

How are you are you loading the data?

Hi,

Thanks for prompt reply.

Below is the details of elasticsearch version I am using.


{
  "name" : "instance-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "RyMOOdrtTAyav9lHs9vGSg",
  "version" : {
    "number" : "8.4.3",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "42f05b9372a9a4a470db3b52817899b99a76ee73",
    "build_date" : "2022-10-04T07:17:24.662462378Z",
    "build_snapshot" : false,
    "lucene_version" : "9.3.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

I have installed elasticsearch on google cloud VM instance on Ubuntu terminal.
I am ingesting data through python script which is also running on same google cloud VM instance.
I installed elasticsearcch through SSH terminal (Command line terminal of Google Cloud).
It runs on Google Cloud VM instance.

I don't know how to secure it ?
Also I did not configure Index Lifecycle Management ?
Please tell me how to do it as I am totally new to this.

I am using 16 GB RAM on Virtual Machine.
Data is already there in Virtual Machine.
Data is in the form of 140 million small sentences.
The total size of this is 1.5 GB.
I have loaded this data in a python list and from list, one by one, I am sending data to elasticsearch.
Since, RAM is 16 GB, and hard disk is 100 GB, this should not be a problem.

Thanks,

Exactly what method did you use to install elasticsearch a package or tar.gz?

It should have walked you through the secure setup and you should have got a username and password?

Did you not do that?

By default elasticsearch 8.4.3 is secured by default?

Hi,

I am reinstlalling the way i have installed.
Below is the code for the same.

shivang_ahd1234@instance-2:~$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
shivang_ahd1234@instance-2:~$ sudo apt-get install apt-transport-https
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  apt-transport-https
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 160 kB of archives.
After this operation, 166 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 apt-transport-https all 2.2.4 [160 kB]
Fetched 160 kB in 0s (1230 kB/s)            
Selecting previously unselected package apt-transport-https.
(Reading database ... 53843 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_2.2.4_all.deb ...
Unpacking apt-transport-https (2.2.4) ...
Setting up apt-transport-https (2.2.4) ...

shivang_ahd1234@instance-2:~$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
shivang_ahd1234@instance-2:~$ sudo apt-get install apt-transport-https
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  apt-transport-https
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 160 kB of archives.
After this operation, 166 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 apt-transport-https all 2.2.4 [160 kB]
Fetched 160 kB in 0s (1230 kB/s)            
Selecting previously unselected package apt-transport-https.
(Reading database ... 53843 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_2.2.4_all.deb ...
Unpacking apt-transport-https (2.2.4) ...
Setting up apt-transport-https (2.2.4) ...
shivang_ahd1234@instance-2:~$ ^C
shivang_ahd1234@instance-2:~$ echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main
shivang_ahd1234@instance-2:~$ sudo apt-get update && sudo apt-get install elasticsearch
Get:1 https://artifacts.elastic.co/packages/8.x/apt stable InRelease [10.4 kB]
Hit:2 http://deb.debian.org/debian bullseye InRelease               
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://security.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:5 http://deb.debian.org/debian bullseye-backports InRelease [49.0 kB]              
Get:6 http://packages.cloud.google.com/apt cloud-sdk-bullseye InRelease [6781 B]
Get:7 http://packages.cloud.google.com/apt google-cloud-packages-archive-keyring-bullseye InRelease [5557 B]
Get:8 http://packages.cloud.google.com/apt google-compute-engine-bullseye-stable InRelease [5533 B]
Get:9 https://artifacts.elastic.co/packages/8.x/apt stable/main amd64 Packages [34.0 kB]
Get:10 http://deb.debian.org/debian bullseye-updates/main Sources.diff/Index [15.1 kB]
Get:11 http://deb.debian.org/debian bullseye-updates/main amd64 Packages.diff/Index [15.1 kB]
Get:12 http://deb.debian.org/debian bullseye-updates/main Translation-en.diff/Index [6117 B]
Get:13 http://deb.debian.org/debian bullseye-updates/main Sources T-2022-10-31-2015.41-F-2022-09-22-1635.40.pdiff [4009 B]
Get:14 http://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2022-10-31-2015.41-F-2022-09-22-1635.40.pdiff [14.3 kB]
Get:13 http://deb.debian.org/debian bullseye-updates/main Sources T-2022-10-31-2015.41-F-2022-09-22-1635.40.pdiff [4009 B]
Get:14 http://deb.debian.org/debian bullseye-updates/main amd64 Packages T-2022-10-31-2015.41-F-2022-09-22-1635.40.pdiff [14.3 kB]
Get:15 http://deb.debian.org/debian bullseye-updates/main Translation-en T-2022-10-21-2017.32-F-2022-09-22-1635.40.pdiff [6025 B]
Get:15 http://deb.debian.org/debian bullseye-updates/main Translation-en T-2022-10-21-2017.32-F-2022-09-22-1635.40.pdiff [6025 B]
Get:16 http://security.debian.org/debian-security bullseye-security/main Sources [167 kB]
Get:17 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [193 kB]
Get:18 http://security.debian.org/debian-security bullseye-security/main Translation-en [122 kB]
Get:19 http://deb.debian.org/debian bullseye-backports/main Sources.diff/Index [63.3 kB]
Ign:19 http://deb.debian.org/debian bullseye-backports/main Sources.diff/Index         
Get:20 http://deb.debian.org/debian bullseye-backports/main amd64 Packages.diff/Index [63.3 kB]
Ign:20 http://deb.debian.org/debian bullseye-backports/main amd64 Packages.diff/Index
Get:21 http://deb.debian.org/debian bullseye-backports/main Translation-en.diff/Index [63.3 kB]
Get:22 http://deb.debian.org/debian bullseye-backports/main Sources [344 kB]
Get:23 http://deb.debian.org/debian bullseye-backports/main amd64 Packages [356 kB]
Get:24 http://packages.cloud.google.com/apt cloud-sdk-bullseye/main amd64 Packages [196 kB]
Get:25 http://deb.debian.org/debian bullseye-backports/main Translation-en T-2022-10-30-2020.19-F-2022-09-22-1115.32.pdiff [45.0 kB]
Get:25 http://deb.debian.org/debian bullseye-backports/main Translation-en T-2022-10-30-2020.19-F-2022-09-22-1115.32.pdiff [45.0 kB]
Get:26 http://packages.cloud.google.com/apt google-cloud-packages-archive-keyring-bullseye/main amd64 Packages [388 B]
Get:27 http://packages.cloud.google.com/apt google-compute-engine-bullseye-stable/main amd64 Packages [1663 B]
Fetched 1880 kB in 1s (1558 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  elasticsearch
0 upgraded, 1 newly installed, 0 to remove and 18 not upgraded.
Need to get 566 MB of archives.
After this operation, 1170 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/8.x/apt stable/main amd64 elasticsearch amd64 8.4.3 [566 MB]
Fetched 566 MB in 12s (45.4 MB/s)                                                      
Selecting previously unselected package elasticsearch.
(Reading database ... 53847 files and directories currently installed.)
Preparing to unpack .../elasticsearch_8.4.3_amd64.deb ...
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Unpacking elasticsearch (8.4.3) ...
Setting up elasticsearch (8.4.3) ...
--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : -tykjC5C+4H4-=ZHLu_S

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with 
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with 
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with 
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service


shivang_ahd1234@instance-2:~$ sudo nano /etc/elasticsearch/elasticsearch.yml

shivang_ahd1234@instance-2:~$ sudo systemctl restart elasticsearch

shivang_ahd1234@instance-2:~$ curl -X GET "localhost:9200"
{
  "name" : "instance-2",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "dUQiQaafTq-m7GplXwTWwA",
  "version" : {
    "number" : "8.4.3",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "42f05b9372a9a4a470db3b52817899b99a76ee73",
    "build_date" : "2022-10-04T07:17:24.662462378Z",
    "build_snapshot" : false,
    "lucene_version" : "9.3.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

In the instruction -
sudo nano /etc/elasticsearch/elasticsearch.yml

I have edited three thing in YML file ..
1. I have changed the ip adress to 0.0.0.0
2. I have removed ['host1','host2'] and made it [ ] ...blank bracket
3. xpack.security.enabled: false -- changed 'true' to 'false'

After this my elasticsearch is working. There is no .tar file and it does not ask me for any id and password.

Below are the two links from which i have installed.

Install Elasticsearch with Debian Package | Elasticsearch Guide [8.11] | Elastic

[How to Install Elasticsearch 8 on Ubuntu 20.04 LTS]

Thanks,

You explicitily disabled security, this means that if your cluster is acessible to the public internet, everyone has access to it and everyone can create or delete indices in your cluster.

You need to enable security, the version 8 evens help you with it as you can see in the Security autoconfiguration information.

Enable security again and follow that information for the next steps.

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : -tykjC5C+4H4-=ZHLu_S

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with 
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with 
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with 
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.
1 Like

Hi,
Got your point. But will this ensure that the index is not deleted in two days, which was the problem I faced twice ?

Changing to 'true' will only ensure security, but I am still left with the same problem, - Why my index keeps getting deleted and what should i do to prevent it ?

Elasticsearch does not delete any of your indices unless you have configured ILM. If your indices are being deleted and your cluster is not secured it is likely that someone (maybe a bot) has accessed your cluster. If you secure the cluster properly it may very well stop your indices from being deleted. Have a look at this thread for a recent example.

1 Like

As already said, there is nothing in Elasticsearch that will automatically delete your index, while you may configure a delete phase using a lifecycle policy, there is nothing like that enabled by default.

To delete an index you would need to explicitily use a DELETE request, and this is probably what happened, someone else deleted the data in your cluster since it is exposed to the internet without any security enabled.

If you had any sensitive data in your cluster you may as well assume that it was copied by someone else.

2 Likes

Hi,

I have enabled security to true.
but i am not able to start the elasticsearch, I am getting the following error....

shivang_ahd1234@instance-1:~$ sudo systemctl restart elasticsearch

shivang_ahd1234@instance-1:~$ curl -X GET "localhost:9200"
curl: (52) Empty reply from server
shivang_ahd1234@instance-1:~$ curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200 
Enter host password for user 'elastic':
curl: (77) error setting certificate verify locations:  CAfile: /etc/elasticsearch/certs/http_ca.crt CApath: /etc/ssl/certs
shivang_ahd1234@instance-1:~$ sudo systemctl daemon-reload
shivang_ahd1234@instance-1:~$ sudo systemctl enable elasticsearch.service
Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /lib/systemd/system/elasticsearch.service.
shivang_ahd1234@instance-1:~$  sudo systemctl start elasticsearch.servic
Failed to start elasticsearch.servic.service: Unit elasticsearch.servic.service not found.
shivang_ahd1234@instance-1:~$ 

Thanks,

You used the wrong service.

Check if elasticsearch is running with sudo systemctl status elasticsearch.

Or use a curl curl -XGET "https://localhost:9200" -u elastic -k

1 Like

Thanks... Its running.. it asked me for password and then it was running...

shivang_ahd1234@instance-1:~$ sudo systemctl restart elasticsearch
shivang_ahd1234@instance-1:~$ sudo systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset:>
     Active: active (running) since Tue 2022-11-01 16:42:45 UTC; 4s ago
       Docs: https://www.elastic.co
   Main PID: 769 (java)
      Tasks: 79 (limit: 19186)
     Memory: 8.5G
        CPU: 1min 4.463s
     CGroup: /system.slice/elasticsearch.service
             ├─769 /usr/share/elasticsearch/jdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialG>
             ├─829 /usr/share/elasticsearch/jdk/bin/java -Des.networkaddress.cache.ttl=>
             └─853 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin>

Nov 01 16:42:25 instance-1 systemd[1]: Starting Elasticsearch...
Nov 01 16:42:45 instance-1 systemd[1]: Started Elasticsearch.

shivang_ahd1234@instance-1:~$ curl -XGET "https://localhost:9200" -u elastic -k
Enter host password for user 'elastic':
{
  "name" : "instance-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AP_dtXGYQl24x1mkhwD5_w",
  "version" : {
    "number" : "8.5.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "c94b4700cda13820dad5aa74fae6db185ca5c304",
    "build_date" : "2022-10-24T16:54:16.433628434Z",
    "build_snapshot" : false,
    "lucene_version" : "9.4.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
shivang_ahd1234@instance-1:~$ 

One last thing... while connecting to elasticsearch through python script ... do i need to give password or without password i will be able to connect ?

You will need to pass both username and password, it will always asked for the password.

Hi,

Trying with this code :

# connect to ES on localhost on port 9200
es = Elasticsearch([{'host': '34.133.230.90', 'port': 9200,'scheme':'http'}],basic_auth=('username', 'password'))
if es.ping():
	print('Connected to ES!')
else:
	print('Could not connect!')
	# sys.exit()

print("*********************************************************************************");

but got the following error:-

WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode([http://34.133.230.90:9200](http://34.133.230.90:9200/))> has failed for 1 times in a row, putting on 1 second timeout WARNING:elastic_transport.transport:Retrying request after failure (attempt 0 of 3) Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_transport.py", line 334, in perform_request request_timeout=request_timeout, File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request raise err from None elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode([http://34.133.230.90:9200](http://34.133.230.90:9200/))> has failed for 2 times in a row, putting on 2 second timeout WARNING:elastic_transport.transport:Retrying request after failure (attempt 1 of 3) Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_transport.py", line 334, in perform_request request_timeout=request_timeout, File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request raise err from None elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode([http://34.133.230.90:9200](http://34.133.230.90:9200/))> has failed for 3 times in a row, putting on 4 second timeout WARNING:elastic_transport.transport:Retrying request after failure (attempt 2 of 3) Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_transport.py", line 334, in perform_request request_timeout=request_timeout, File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request raise err from None elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode([http://34.133.230.90:9200](http://34.133.230.90:9200/))> has failed for 4 times in a row, putting on 8 second timeout

Could not connect! *********************************************************************************

Thanks...

You need to use https, not http.

Hi,

I have changed to "https", but still getting the error.

# connect to ES on localhost on port 9200
es = Elasticsearch([{'host': '34.133.230.90', 'port': 9200,'scheme':'https'}],basic_auth=('username', 'password'))
if es.ping():
	print('Connected to ES!')
else:
	print('Could not connect!')
	# sys.exit()

print("*********************************************************************************");
WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode(https://34.133.230.90:9200)> has failed for 1 times in a row, putting on 1 second timeout
WARNING:elastic_transport.transport:Retrying request after failure (attempt 0 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_transport.py", line 334, in perform_request
    request_timeout=request_timeout,
  File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.TlsError: TLS error caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091))
WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode(https://34.133.230.90:9200)> has failed for 2 times in a row, putting on 2 second timeout
WARNING:elastic_transport.transport:Retrying request after failure (attempt 1 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_transport.py", line 334, in perform_request
    request_timeout=request_timeout,
  File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.TlsError: TLS error caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091))
WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode(https://34.133.230.90:9200)> has failed for 3 times in a row, putting on 4 second timeout
WARNING:elastic_transport.transport:Retrying request after failure (attempt 2 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_transport.py", line 334, in perform_request
    request_timeout=request_timeout,
  File "/usr/local/lib/python3.7/dist-packages/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
    raise err from None
elastic_transport.TlsError: TLS error caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091))
WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode(https://34.133.230.90:9200)> has failed for 4 times in a row, putting on 8 second timeout
Could not connect!
*********************************************************************************

Check the error and the documentation, you are using a self signed certificate you will need to change your code to pass this certificate or tell your code to not verify the certificate.

Please check this documentation the part about using SSL.

You basically need to pass the path to your CA.

es = Elasticsearch(
    ['localhost:443', 'other_host:443'],
    # turn on SSL
    use_ssl=True,
    # make sure we verify SSL certificates
    verify_certs=True,
    # provide a path to CA certs on disk
    ca_certs='/path/to/CA_certs'
)

Or tell it to ignore e don't validate.

es = Elasticsearch(
    ['localhost:443', 'other_host:443'],
    # turn on SSL
    use_ssl=True,
    # no verify SSL certificates
    verify_certs=False,
    # don't show warnings about ssl certs verification
    ssl_show_warn=False
)

I tried the following code:-

# connect to ES on localhost on port 9200
from ssl import create_default_context
import ssl
# from elasticsearch.connection import create_ssl_context

es = Elasticsearch([{'host': '34.133.230.90', 'port': 9200,'scheme':'https'}],basic_auth=('username', 'password'),
                    verify_certs=False, ssl_show_warn=True)

if es.ping():
	print('Connected to ES!')
else:
	print('Could not connect!')
	# sys.exit()

print("*********************************************************************************");

Output is:-

Could not connect!
*********************************************************************************

Another Code ---

# connect to ES on localhost on port 9200
from ssl import create_default_context
import ssl
# from elasticsearch.connection import create_ssl_context

es = Elasticsearch([{'host': '34.133.230.90', 'port': 9200,'scheme':'https'}],basic_auth=('username', 'password'),
                    use_ssl=True,verify_certs=False, ssl_show_warn=True)

if es.ping():
	print('Connected to ES!')
else:
	print('Could not connect!')
	# sys.exit()

print("*********************************************************************************");

Output -

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-28-2ef570288f0b> in <module>
      5 
      6 es = Elasticsearch([{'host': '34.133.230.90', 'port': 9200,'scheme':'https'}],basic_auth=('username', 'password'),
----> 7                     use_ssl=True,verify_certs=False, ssl_show_warn=True)
      8 
      9 if es.ping():

TypeError: __init__() got an unexpected keyword argument 'use_ssl'

Adding CA_path"-

# connect to ES on localhost on port 9200
from ssl import create_default_context
import ssl
# from elasticsearch.connection import create_ssl_context

es = Elasticsearch([{'host': '34.133.230.90', 'port': 9200,'scheme':'https'}],basic_auth=('username', 'password'),
                    verify_certs=False, ssl_show_warn=True,ca_certs='/path/to/CA_certs')

if es.ping():
	print('Connected to ES!')
else:
	print('Could not connect!')
	# sys.exit()

print("*********************************************************************************");

output:-

Could not connect!
*********************************************************************************

Hi
I ran the following code :

# connect to ES on localhost on port 9200
from ssl import create_default_context
import ssl
# from elasticsearch.connection import create_ssl_context

es = Elasticsearch([{'host': 'localhost', 'port': 9200,'scheme':'https'}],basic_auth=('username', 'password'), verify_certs=False)

if es.ping():
	print('Connected to ES!')
else:
	print('Could not connect!')
	# sys.exit()

print("*********************************************************************************");

And got the following error:-

AuthenticationException                   Traceback (most recent call last)
Cell In [11], line 1
----> 1 print(es.info())

File ~/miniconda3/lib/python3.9/site-packages/elasticsearch/_sync/client/utils.py:414, in _rewrite_parameters.<locals>.wrapper.<locals>.wrapped(*args, **kwargs)
    411         except KeyError:
    412             pass
--> 414 return api(*args, **kwargs)

File ~/miniconda3/lib/python3.9/site-packages/elasticsearch/_sync/client/__init__.py:2296, in Elasticsearch.info(self, error_trace, filter_path, human, pretty)
   2294     __query["pretty"] = pretty
   2295 __headers = {"accept": "application/json"}
-> 2296 return self.perform_request(  # type: ignore[return-value]
   2297     "GET", __path, params=__query, headers=__headers
   2298 )

File ~/miniconda3/lib/python3.9/site-packages/elasticsearch/_sync/client/_base.py:321, in BaseClient.perform_request(self, method, path, params, headers, body)
    318         except (ValueError, KeyError, TypeError):
    319             pass
--> 321     raise HTTP_EXCEPTIONS.get(meta.status, ApiError)(
    322         message=message, meta=meta, body=resp_body
    323     )
    325 # 'X-Elastic-Product: Elasticsearch' should be on every 2XX response.
    326 if not self._verified_elasticsearch:
    327     # If the header is set we mark the server as verified.

AuthenticationException: AuthenticationException(401, 'security_exception', 'unable to authenticate user [username] for REST request [/]')

username i am using is ==> 'elastic'.
What doe the error mean...is there any other username ?