# ETL tool for ElasticSearch

**URL:** <https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803>\
**Category:** Elasticsearch\
**Created:** [January 2, 2018, 3:25pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803 "2018-01-02T15:25:51Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![faoc](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@faoc](https://discuss.elastic.co/u/faoc)\
**Post date:** [January 2, 2018, 3:25pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/1 "2018-01-02T15:25:51Z")

</div>

Hi Guys,  
do you know any ETL tool where is possible to have a connector to ElasticSearch and get all data that i need?  
I was trying talend studio (big data) and oracle data integrator, but without results.  
if with this tools i cannot find a way to extract the data, Do you think that is better extract data and write it to a file and after use the ETL tool to load data from file?

thank you

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [January 2, 2018, 3:37pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/2 "2018-01-02T15:37:37Z")

</div>

If I read the question correctly, you want to copy documents from Elasticsearch to another system. Correct?

Elastic has the [Logstash product](https://www.elastic.co/guide/en/logstash/current/introduction.html) and there is an Elasticsearch input that can process the results of a ES query.

What is the destination of the docs?

---

<div class="post-metadata">

**Author:** ![jbackshire](https://avatars.discourse-cdn.com/v4/letter/j/85f322/32.png) [@jbackshire](https://discuss.elastic.co/u/jbackshire)\
**Post date:** [January 2, 2018, 4:22pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/3 "2018-01-02T16:22:08Z")

</div>

Are you trying to Move documents from ES to another DB or are you trying to move documents into ES?

Please clarify.

Thanks!

---

<div class="post-metadata">

**Author:** ![faoc](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@faoc](https://discuss.elastic.co/u/faoc)\
**Post date:** [January 2, 2018, 11:31pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/4 "2018-01-02T23:31:24Z")

</div>

I have data in elasticsearch and i want to create dashboards in powerBI with some data saved on ElasticSearch. So, my intention is create a ETL process that load data from ES, treat the data and insert the new data on Data Warehouse (im using MySQL).  
My problem, is that i didnt find any connector for elasticsearch, on Talend Big Data Studion (free edition) neither on ODI oracle.  
Do i need to export the data to files and after use the ETL tool or do you know some other way to do that?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 2, 2018, 11:41pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/5 "2018-01-02T23:41:21Z")

</div>

Why not use Kibana?

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [January 3, 2018, 6:42am UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/6 "2018-01-03T06:42:10Z")

</div>

I'm curious to learn what PowerBI can do for you that Kibana can't 😉

Have you tried Dremio yet?

> **[Power BI on Elasticsearch](https://www.dremio.com/solutions/power-bi-on-modern-data/power-bi-on-elasticsearch)**
>
> Connect Power BI to Elasticsearch. Start running interactive SQL in minutes

---

<div class="post-metadata">

**Author:** ![faoc](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@faoc](https://discuss.elastic.co/u/faoc)\
**Post date:** [January 3, 2018, 9:24am UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/7 "2018-01-03T09:24:53Z")

</div>

Thank you, i didn't try yet.  
PowerBI you can create more "professional" dashboards and create reports for customers. I use kibana but just inside team.

---

<div class="post-metadata">

**Author:** ![val](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/val/32/138203_2.png) [@val](https://discuss.elastic.co/u/val)\
**Post date:** [January 3, 2018, 9:42am UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/8 "2018-01-03T09:42:42Z")

</div>

> [@faoc](#):
>
> more "professional" dashboards

Well, only until [Canvas](https://www.elastic.co/blog/canvas-tech-preview) comes out, that is 😉

And to be honest, Kibana 6.1 with dashboard-only role, dashboard input controls and more leaves nothing to be desired

---

<div class="post-metadata">

**Author:** ![faoc](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@faoc](https://discuss.elastic.co/u/faoc)\
**Post date:** [January 3, 2018, 4:17pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/9 "2018-01-03T16:17:16Z")

</div>

I want import data from ES and my output would be mysql. Can be possible do that with logstash?

---

<div class="post-metadata">

**Author:** ![afrozl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/afrozl/32/26197_2.png) [@afrozl](https://discuss.elastic.co/u/afrozl)\
**Post date:** [January 3, 2018, 4:27pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/10 "2018-01-03T16:27:17Z")

</div>

Yes. logstash would work perfectly for this use case - you can read the data from elasticsearch, apply any transformations needed and then load to mysql directly or output to a file and load from that.

---

<div class="post-metadata">

**Author:** ![faoc](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@faoc](https://discuss.elastic.co/u/faoc)\
**Post date:** [January 3, 2018, 5:09pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/11 "2018-01-03T17:09:27Z")

</div>

Perfect, i will try this :). thank you

---

<div class="post-metadata">

**Author:** ![jbackshire](https://avatars.discourse-cdn.com/v4/letter/j/85f322/32.png) [@jbackshire](https://discuss.elastic.co/u/jbackshire)\
**Post date:** [January 3, 2018, 6:18pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/12 "2018-01-03T18:18:42Z")

</div>

An alternative approach, if you don't want to use Logstash, you can use Pentaho with a few lines of simple custom python code that you could customize to do what you want.

---

<div class="post-metadata">

**Author:** ![faoc](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@faoc](https://discuss.elastic.co/u/faoc)\
**Post date:** [January 4, 2018, 4:04pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/13 "2018-01-04T16:04:38Z")

</div>

thank you for your answer.  
Im trying logstash but i'm getting some errors to add jdbc output plugin...

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 4, 2018, 9:44pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/14 "2018-01-04T21:44:13Z")

</div>

It'd be worth creating a new thread in #logstash with your problems so we can help 🙂

---

<div class="post-metadata">

**Author:** ![ericmad](https://avatars.discourse-cdn.com/v4/letter/e/48db29/32.png) [@ericmad](https://discuss.elastic.co/u/ericmad)\
**Post date:** [January 9, 2018, 10:09pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/15 "2018-01-09T22:09:45Z")

</div>

The CData Elasticsearch Drivers ([https://www.cdata.com/drivers/elasticsearch/](https://www.cdata.com/drivers/elasticsearch/)) should be able to plug-in to just about any ETL tool and help you accomplish what you are trying to do. The driver interfaces are supported by every modern data movement application.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 9, 2018, 10:13pm UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/16 "2018-01-09T22:13:57Z")

</div>

@ericmad please be clear that you are an employee of this organisation and you are promoting a paid product as an employee.

---

<div class="post-metadata">

**Author:** ![n.maire](https://avatars.discourse-cdn.com/v4/letter/n/f07891/32.png) [@n.maire](https://discuss.elastic.co/u/n.maire)\
**Post date:** [January 11, 2018, 9:35am UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/17 "2018-01-11T09:35:33Z")

</div>

You might wanna take a look at [Apache NiFi](https://nifi.apache.org/) which has processors (connectors) for a ton of stuff, including Elasticsearch and SQL.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2018, 9:35am UTC](https://discuss.elastic.co/t/etl-tool-for-elasticsearch/113803/18 "2018-02-08T09:35:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
