ELK Architecture Questions

Elastic Stack Elasticsearch
1

Hi, I'm relatively new to ELK and have some specific questions about the overall architecture.

In a typical ELK stack by the time log data is in ES does it need to be in a specific schema so that it can be read by Kibana?

The reason I ask is I have a very specific architecture where I want to insert log data from log4j directly into ElasticSearch without LogStash. I also want the data once it is in ES to be viewed by Kibana OR by a custom application that would read from ES directly. What I haven't been able to understand is whether or not there is in essence a required schema in ES.

Any tips to help me clarify these questions are appreciated.

2

Preferably yes, but KB will still work.

Look into templates - Index templates | Elasticsearch Guide [8.11] | Elastic

3

Thanks, are there pre-defined templates for use with Kibana or generally when using ES to store log data?

4

You can look at the Logstash one - https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch/elasticsearch-template.json