[API Logs] - Pulling Latest Logs to Power BI

[aisyaharifin (Siti Nuraisyah)]

Hi Elastic,

I want to ask is there a way we can continuously pulling the Elasticsearch data to Power BI Cloud for dashboard purposes?

The team currently using Azure Data Factory to pull the data from our Elastic.

Currently they pull the data using the URL and API Key to access the data.

Thank you.

[carly.richmond]

Hi @aisyaharifin,

Not sure if this is relevant to Power BI Cloud, but are you able to use the Elasticsearch ODBC driver similar to the desk guide in the documentation?

[aisyaharifin (Siti Nuraisyah)]

Hi @carly.richmond,

Thank you for your reply.

Apparently the documentation is not applicable for my situation.

If so how do I pull the latest data continuously from Elasticsearch?

Let's say I would like to pull the latest APM logs data continuously from Elasticsearch, how do I do that?

[carly.richmond]

Hi @aisyaharifin ,

@aisyaharifin

Apparently the documentation is not applicable for my situation.

Can you explain a bit more about why it's not applicable? Do you not have access to the ODBC connector in Power BI Cloud?

Having a search around I can think of a few ways of getting the information into Power BI:

1. Using Azure Data Factory as you currently. I see from this blog that you can create a live connection using the CData Connect Cloud Virtual SQL Server API. I'm not sure if this is what you are currently doing?
2. Using the Power BI Web connector, although looking at this blog I worry this will require polling.
3. Using Logstash with an Elasticsearch input plugin and a http output plugin to call the Power BI REST API to send the data.

Do any of those work with your use case? If not do let us know a bit more about what you're trying to do.