# Is logstash necessarily

**URL:** <https://discuss.elastic.co/t/is-logstash-necessarily/328833>\
**Category:** Logstash\
**Created:** [March 29, 2023, 1:24pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833 "2023-03-29T13:24:14Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![LilBaloche](https://avatars.discourse-cdn.com/v4/letter/l/e8c25b/32.png) [@LilBaloche](https://discuss.elastic.co/u/LilBaloche)\
**Post date:** [March 29, 2023, 1:24pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/1 "2023-03-29T13:24:14Z")

</div>

Hi everyone

I'm testing ELK in a virtual environment (WinServer AD + DNS, Ubuntu Server 22.04, Ubuntu Client 22.04 and Win 10 Client)  
I've installed ELK stack on an Ubuntu Server 22.04 (I've been helped by a youtube video)

In this video, the person doesn't configure logstash so I didn't configure it and my ELK stack (Rather EK in my case) works well.  
I'm not using any cloud and my cluster has just 1 node

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [March 29, 2023, 2:00pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/2 "2023-03-29T14:00:29Z")

</div>

LogStash will then use default params, so should work.  
There are configurations:

- logstash.yml - extra settings for LS
- pipelines.yml - if you need extra pipelines for more advance configurations
- \*.conf - a configuration for data processing, the most important. Input, filter, and output define how to process data and where to send

---

<div class="post-metadata">

**Author:** ![LilBaloche](https://avatars.discourse-cdn.com/v4/letter/l/e8c25b/32.png) [@LilBaloche](https://discuss.elastic.co/u/LilBaloche)\
**Post date:** [March 29, 2023, 2:10pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/3 "2023-03-29T14:10:28Z")

</div>

Logstash is not even started and I set up security on Kibana and Elasticsearch by changing port, xpack, certificates...

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [March 29, 2023, 2:17pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/4 "2023-03-29T14:17:12Z")

</div>

Can you test LS with simple.conf

```auto
input {
  generator {
       message => "Info 11 Jan 2023 07:39:50.527 [Thread-A] - TaskScheduler, State=NORMAL"
       count => 1
  }

} # input

filter {
}

output {

   file { path => "/somepath/test_%{+YYYY-MM-dd}.txt" } 

   stdout { codec => rubydebug{} }
}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 29, 2023, 2:17pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/5 "2023-03-29T14:17:14Z")

</div>

> [@LilBaloche](#):
>
> In this video, the person doesn't configure logstash so I didn't configure it and my ELK stack (Rather EK in my case) works well.

It depends in your use case, what you want to collect, what you want to do with your data.

You only need to use Logstash if you want.

---

<div class="post-metadata">

**Author:** ![LilBaloche](https://avatars.discourse-cdn.com/v4/letter/l/e8c25b/32.png) [@LilBaloche](https://discuss.elastic.co/u/LilBaloche)\
**Post date:** [March 29, 2023, 2:26pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/6 "2023-03-29T14:26:00Z")

</div>

I've left my working desktop, I'll try it tomorrow. I'll be back, ty for reply

---

<div class="post-metadata">

**Author:** ![LilBaloche](https://avatars.discourse-cdn.com/v4/letter/l/e8c25b/32.png) [@LilBaloche](https://discuss.elastic.co/u/LilBaloche)\
**Post date:** [March 29, 2023, 2:32pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/7 "2023-03-29T14:32:29Z")

</div>

It would be monitoring the different machines of my infrastructure such as cpu usage, temperature, bandwidth etc..

---

<div class="post-metadata">

**Author:** ![Sunile\_Manjee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunile_manjee/32/111461_2.png) [@Sunile\_Manjee](https://discuss.elastic.co/u/Sunile_Manjee)\
**Post date:** [March 30, 2023, 5:25pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/8 "2023-03-30T17:25:36Z")

</div>

It all depends on your use case. In the Elastic Stack (also known as ELK Stack), there are several components: Elasticsearch, Logstash, Kibana, and Beats or Elastic Agent. Elastic Agent is a first-class citizen in the Elastic ecosystem, and it is responsible for collecting and shipping data to Elasticsearch or Logstash.

Logstash is often used for data enrichment, processing, and distribution. If your use case doesn't require these operations, you can have Elastic Agent write directly to Elasticsearch, bypassing Logstash. However, if you need to transform, filter, or enrich the data before indexing it in Elasticsearch, using Logstash would be beneficial.

In summary, whether you need to use Logstash or not depends on your specific requirements and data processing needs. You can write directly from Elastic Agent to Elasticsearch if Logstash isn't necessary for your situation.

---

<div class="post-metadata">

**Author:** ![jba](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jba/32/118482_2.png) [@jba](https://discuss.elastic.co/u/jba)\
**Post date:** [March 30, 2023, 8:30pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/9 "2023-03-30T20:30:36Z")

</div>

At my employer we use a setup like this:

1. Application writes log messages to disk-file
2. Filebeat reads log messages from from disk-file
3. Filebeat sends messages (aka documents) to Logstash
4. Logstash does some parsing (grokking) of the documents and extracts fields & values
5. Logstash sends the documents to Elasticsearch for indexing and storage.
6. Users log in to Kibana to view their log messages

It is entirely possible to skip steps 4-5 and have Filebeat send the documents directly to Elasticsearch. You can even skip steps 2-3 and have the Application send directly to Elasticsearch (if it knows how to talk to Elasticsearch).

If your setup is for monitoring systems data, not application messages, then yes, you can do a setup with just 3 components:

1. ElasticAgent sends monitoring data to Elasticsearch
2. Elasticsearch indexes and stores the data
3. You view the monitoring data from Kibana (and control your 'fleet' of ElasticAgents from Kibana)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 27, 2023, 8:31pm UTC](https://discuss.elastic.co/t/is-logstash-necessarily/328833/10 "2023-04-27T20:31:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
