Automatically visualize data on kibana from input?

I'm wondering if there's a way to automatically visualize data on KIBANA. ? My logstash will receive frequent new inputs, and i need to know if there's way to automatically create dashboards ? may be a way to define in templates ?

There is no fully automated way to do so - however you could build something like this using a script and the saved object api of Kibana: Saved objects APIs | Kibana Guide [7.12] | Elastic

A bit like this:

It's not an out-of-the-box solution, but you would have a lot of control this way.

Thank you @flash1293 , i will go through these documents and get back incase of questions :slight_smile:

Cheers!

i was able to view the fields in my index via GET method . Not sure if this is correct , please correct me if this is wrong

GET log_sample/_field_caps?fields=*

{
  "indices" : [
    "log_sample"
  ],
  "fields" : {
    "CustID" : {
      "text" : {
        "type" : "text",
        "searchable" : true,
        "aggregatable" : false
      }
    },
    "Forward_hostname" : {
      "text" : {
        "type" : "text",
        "searchable" : true,
        "aggregatable" : false
      }
    },
    "Req_HTTP_Status" : {
      "text" : {
        "type" : "text",
        "searchable" : true,
        "aggregatable" : false
      }
    },
    "Req_Transfer_time.keyword" : {
      "keyword" : {
        "type" : "keyword",
        "searchable" : true,
        "aggregatable" : true
      }
    },
    "RequestID" : {
      "text" : {
        "type" : "text",
        "searchable" : true,
        "aggregatable" : false

How would one be able to turn these fields into visualization / dashboard saved objects ?

It depends on how exactly you want them visualized - as you know Kibana offers a lot of ways to do so.

Check out how your existing Visualizations are structured (by exporting them). In most cases you probably want to replace the field name mentioned in the export by the new field names.

Thank you @flash1293 , in my case , the field names would remain the same . Only the data would keep changing , as new log files are taken in as input.

I will try exporting them and get back in case of doubts . Thanks

Wait, your fields will stay the same? Maybe I misunderstood your question then.

My logstash will receive frequent new inputs, and i need to know if there's way to automatically create dashboards

What do you mean by that? If there's just new data coming in, you can simply change the time range in the top right to visualize your new data in your existing dashboards. Dashboards and visualuzations are just a piece of configuration, the data is always loaded from Elasticsearch directly. It's not saving the actual data points, just a description how to fetch the data and how to visualize it.

Yes, the fields will stay the same . But the Data that will be displayed will change .

The idea is to treat each input file as different when a user places a log file,but display data in the same kind of visualizations

Then it's way easier than that. Index everything into the same index, then put the source of the data into each document (it's common to have a dataset field), then simply add a filter like dataset: thatFile or dataset:thisFile to your dashboard.

This allows you to use the same dashboard and switch between different data sources.

Related: Logstash: how to add file name as a field? - Stack Overflow

Thank you , i will try this .