Duplicate data Logstash

Elastic Stack Logstash
1

Hi, guy's
i need your help, please with this...

I have a problem with the indexing of data from logstash to elastic, I have two pipelines and when I run the service I duplicate the data from one pipeline to the other, and does not let me create an individualized one, what I need is to create an index with the correct data and not pass me the other data from the other pipeline, thanks, it should be noted that the first pipeline I have with snmp and the other, which shows me the data from snmp is syslog.

2

Logstash by default concatenates config files and do not handle them separately. This means that data from all inputs go to all outputs unless you implement control through conditionals or specify the configs as distinct pipelines using pipelines.yml.

3

good, thanks, but now I have a problem with the configuration of multiple pipelines, and is that I do not work the configuration in the pipeline.yml, the pipeline_id remains the same by default, how can I configure this correctly use the command: /usr/share/logstash/bin/logstash -f efecty --config.reload.automatic , but according to the documentation tells me that with the -f ignores the pipeline.yml, but here is the problem if I remove the -f error comes out what can I do?

captue
captue1477×692 287 KB

Capture
Capture957×806 34.1 KB

4

What is the error? You are running logstash in the command line with the parameter -f pointing to a config file, it will ignore the pipelines.yml file and the pipeline_id will be called main, this is what is happening in your first screenshot.

If you want to run two different pipelines you need to configure the pipeline.yml file and point to the config files.

Based in your screenshots, you should use this pipelines.yml

- pipeline.id: efecty
  path.config: "/etc/logstash/conf.d/efecty.conf"
  pipeline.workers: 3

- pipeline.id: syslog
  path.config: "/etc/logstash/conf.d/syslog.conf"
  queue.type: persisted

Try to use this config and run logstash without the -f parameter.

Besides that, please avoid posting screenshots, sometimes it is not possible to read, share your code or logs using the </> button.

5

ok, thanks I am going to try it , ok i get it

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.