Getting sample apache logs into Kibana

Elastic Stack Kibana
1

Hello everyone

I'm currently building an ELK stack and what I want to do is use a previous Apache log file to ingest into the ELK server for demo purposes.

One idea I had was to make the log locally available and then parse through Logstash by adding an apache conf file in the Logstash conf.d directory and restart Logstash.

In short I'm trying to find a quick way to add some "known" data to be able to demo to my colleagues the benefits of using Kibana dashboards for identifing security events.

Thanks

Paul

2

The kibana team, and i think a good portion of elastic uses this awesome tool called makelogs here http://github.com/spalger/makelogs, it's a great way to just pop in some really generic data into a cluster to analyze and show off what you can do.

Peace,
Khalah

3

Thanks for the info Khalah, I could find a use case for that however what I'm trying to do is use an existing log that contains, let's say "activities of interest" to view in Kibana. So it would be champion if I could use that.

Paul

4

I think you're idea is fine. I found this pretty detailed step-by-step guide that gets Apache logs into Logstash and into Elasticsearch;

Let us know if you get stuck on some part of that.

Regards,
Lee

5

Thank you Lee, i did give that a go but fell at the logstash apache config

3output {
    elasticsearch { protocol => "http" }
}

with this inplace logstash fails reporting a protocol error! Can you please advise?

UPDATE: As David Bowie sung "Ch Ch Ch Changes" new format config noted in dropping the use of protocol within the conf file. https://www.elastic.co/guide/en/logstash/2.0/advanced-pipeline.html

UPDATE UPDATE: Data still absent from ELK!