How to connect my api directly to kibana?

[OAJ]

Hi, there I'm new to Kibana.

I was wondering if I have an API link such as api.sampleapis.com/switch/games how can I connect it to Kibana directly so that I may use it in my dashboard.

The only way I found is to upload it to Kibana is by converting the json into bulk data format for Elasticsearch then upload it using PowerShell, which works. But this means that every time there's a change in API I need to do same steps over and over again.

I want it to be automatic and instantly read the API from source.

Here is the code I've been using to upload locally from my laptop:

1. run json file

const axios = require("axios");
const fs = require("fs");

// Function to convert data into Elasticsearch bulk format
function convertToElasticsearchBulkFormat(data, indexName) {
  const bulkData = [];

  data.forEach((item) => {
    const docId = item.id;
    delete item.id;
    bulkData.push({ index: { _index: indexName, _id: docId } });
    bulkData.push(item);
  });

  return bulkData;
}

// Function to fetch data from API and convert it
async function fetchDataAndConvert(apiUrl, indexName) {
  try {
    const response = await axios.get(apiUrl);
    const data = response.data;

    const bulkFormatData = convertToElasticsearchBulkFormat(data, indexName);

    fs.writeFileSync(
      "bulk_data.json",
      bulkFormatData.map((line) => JSON.stringify(line)).join("\n")
    );
    console.log("Data successfully converted and saved to bulk_data.json");
  } catch (error) {
    console.error("Error fetching or converting data:", error);
  }
}

const apiUrl = "https://api.sampleapis.com/switch/games";
const indexName = "gametest";
fetchDataAndConvert(apiUrl, indexName);

2. Run PS commands for upload

$username = "elastic"

$password = "enter-your-pass"

$securePassword = ConvertTo-SecureString $password -AsPlainText -Force

$credential = New-Object System.Management.Automation.PSCredential ($username, $securePassword)

Invoke-RestMethod -Uri "https://localhost:9200/gametest/_bulk?pretty" -Method Post -ContentType 'application/x-ndjson' -InFile "bulk_data.json" -Credential $credential

I'm using version 8.15

Thanks in advance

[carly.richmond]

Hi @OAJ,

Welcome! To confirm, you're asking if you can access data from another API within a Kibana dashboard? Are you looking to visualize the data in a component in your dashboard?

[OAJ]

Hi @carly.richmond,
Yes I would like to connect data from a given API to Kibana hence visualize it in dashboard

[carly.richmond]

Ok, is there a particular reason why you don't want to ingest the data into Elasticsearch to visualize the data, potentially using an approach such as the Logstash HTTP plugin or one of the Elasticsearch language clients? That is probably what I would recommend to make visualizing different cuts of the data, along with constraints such as performance, enrichment and archiving.

Another option could be to write a Kibana plugin,. As you would be connecting to an API outside of Elastic I would be mindful of the security and performance impact to Kibana in your code as covered in the general and security best practices.

Hope that helps!