Databricks + ElasticSearch setup

First and foremost, apologies for the very simple question.

Today is my first day using Elasticsearch. After doing some research I found I can use the tool via a Databricks cluster, which I'm moderately familiar with. In the documentation on databricks here:


### Elasticsearch

<img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt5ebe80fb665aef6b/5ea8c8f26b62d4563b6ecec2/brand-elasticsearch-220x130.svg" width="300">

1. Launch a cluster in your workspace or choose an existing cluster.
2. Once the new cluster is running, go to the "Libraries" tab of that cluster, and click "Install new" -> choose "Maven" -> enter the maven coordinates `org.elasticsearch:elasticsearch-spark-30_2.12:7.17.6` -> click "Install".
3. Once the installation has finished, attach this notebook to the cluster, and run write and/or read operations against your Elasticsearch cluster

**Important**: In the following cells, replace `<ip-address>`, `<port>`, `<ssl>`, `<hostname>` and `<index>` with your Elasticsearch configuration 

%sh 
nc -vz ip-address port

My confusion is how to find/create the ip-address, port, ssl, hostname, and index.
This seems like it should be easy, but I can't find anywhere that it's actually spelled out for the slow folks, aka me. Any advice or documentation is greatly appreciated. Thanks in advance

Welcome to our community! :smiley:

Where do you have Elasticsearch installed?

Thank you! I have both downloaded Elasticsearch locally and have enabled Elasticsearch (Elastic Cloud) on the Azure subscription and resource group that Databricks is using

If you go into your Elastic Cloud cluster management page you should see;
Screen Shot 2022-10-14 at 09.40.23

Just use the Copy endpoing for Elasticsearch and that's all you should need :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.