Databricks + ElasticSearch setup

Elastic Stack Elasticsearch
1

First and foremost, apologies for the very simple question.

Today is my first day using Elasticsearch. After doing some research I found I can use the tool via a Databricks cluster, which I'm moderately familiar with. In the documentation on databricks here:


### Elasticsearch

<img src="https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt5ebe80fb665aef6b/5ea8c8f26b62d4563b6ecec2/brand-elasticsearch-220x130.svg" width="300">

1. Launch a cluster in your workspace or choose an existing cluster.
2. Once the new cluster is running, go to the "Libraries" tab of that cluster, and click "Install new" -> choose "Maven" -> enter the maven coordinates `org.elasticsearch:elasticsearch-spark-30_2.12:7.17.6` -> click "Install".
3. Once the installation has finished, attach this notebook to the cluster, and run write and/or read operations against your Elasticsearch cluster

**Important**: In the following cells, replace `<ip-address>`, `<port>`, `<ssl>`, `<hostname>` and `<index>` with your Elasticsearch configuration 

%sh 
nc -vz ip-address port

My confusion is how to find/create the ip-address, port, ssl, hostname, and index.
This seems like it should be easy, but I can't find anywhere that it's actually spelled out for the slow folks, aka me. Any advice or documentation is greatly appreciated. Thanks in advance

2

Welcome to our community! :smiley:

Where do you have Elasticsearch installed?

3

Thank you! I have both downloaded Elasticsearch locally and have enabled Elasticsearch (Elastic Cloud) on the Azure subscription and resource group that Databricks is using

4

If you go into your Elastic Cloud cluster management page you should see;
Screen Shot 2022-10-14 at 09.40.23

Just use the Copy endpoing for Elasticsearch and that's all you should need :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.