I'm testing ELK in a virtual environment (WinServer AD + DNS, Ubuntu Server 22.04, Ubuntu Client 22.04 and Win 10 Client)
I've installed ELK stack on an Ubuntu Server 22.04 (I've been helped by a youtube video)
In this video, the person doesn't configure logstash so I didn't configure it and my ELK stack (Rather EK in my case) works well.
I'm not using any cloud and my cluster has just 1 node
It all depends on your use case. In the Elastic Stack (also known as ELK Stack), there are several components: Elasticsearch, Logstash, Kibana, and Beats or Elastic Agent. Elastic Agent is a first-class citizen in the Elastic ecosystem, and it is responsible for collecting and shipping data to Elasticsearch or Logstash.
Logstash is often used for data enrichment, processing, and distribution. If your use case doesn't require these operations, you can have Elastic Agent write directly to Elasticsearch, bypassing Logstash. However, if you need to transform, filter, or enrich the data before indexing it in Elasticsearch, using Logstash would be beneficial.
In summary, whether you need to use Logstash or not depends on your specific requirements and data processing needs. You can write directly from Elastic Agent to Elasticsearch if Logstash isn't necessary for your situation.
Filebeat sends messages (aka documents) to Logstash
Logstash does some parsing (grokking) of the documents and extracts fields & values
Logstash sends the documents to Elasticsearch for indexing and storage.
Users log in to Kibana to view their log messages
It is entirely possible to skip steps 4-5 and have Filebeat send the documents directly to Elasticsearch. You can even skip steps 2-3 and have the Application send directly to Elasticsearch (if it knows how to talk to Elasticsearch).
If your setup is for monitoring systems data, not application messages, then yes, you can do a setup with just 3 components:
ElasticAgent sends monitoring data to Elasticsearch
Elasticsearch indexes and stores the data
You view the monitoring data from Kibana (and control your 'fleet' of ElasticAgents from Kibana)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.