Kibana intergration with all types of AWS Logs like (VPC flow Logs, Instances Logs, Cloudwatch logs, Cloud trail logs , systemlogs )


(Shashank Gupta) #1

how to do Kibana intergration with all types of AWS Logs like (VPC flow Logs, Instances Logs, Cloudwatch logs, Cloud trail logs , systemlogs ).

Thanks
Shashank


(Joe Fleming) #2

You'd first need to index those logs in Elasticsearch. Are you already doing that?


(Shashank Gupta) #3

Hello

Yes i am ready for that i want to integrated the vpc flowlogs, and cloudtrail logs, S3 bucket logs, and operating system logs and Application log and want to see the kibana dashboard.

Please provide me steps or document.

Regards

Shashank


(Joe Fleming) #4

That's a complex ask. The gist is:

  • You'd want to take all that log data, index it in Elasticsearch
  • Build Kibana visualizations to see what you want to see in that Elasticsearch data
  • Add those visualizations to Dashboards to build an overview

The devil's in the details here though. For example, do you need to enrich or otherwise modify this data as you index it (for example, to geo-code IP address, or re-shaping the log structure to something more searchable)? Should each type of log be in its own index (Kibana visualizations are tied to a single index, so if you need to compare different logs in a single visualization, you need to put them in the same index and structure the data in the same way for all of them)? What you want to see in your data will drive the kinds of visualizations you need to build, do you know that already?

It sounds like you are pretty new with the Elastic stack, maybe finding some overview or getting started resources will help you here. There's a couple of getting started videos you can find links to here: https://www.elastic.co/start. The docs on our site can also be helpful, and there's not shortage of other overview and getting started resources that you can find with a web search.


(system) closed #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.