Requirement to build a Multiple Input system in Logstash as given below, Can I build something like this?

Elastic Stack Logstash
1

I need to implement a UDP system in between the input and output patterns as mentioned below.

inputs : beats, tcp
output: elastic-search

meanwhile i need to send all the data from the inputs like beats, tcp to an UDPOutput.
and receive the same from UDPOutput to UDPInput. and finally from UDPInput to elasticsearch.

input -- pipeline output
                        \
                         pipeline input -- udp output
                                                     \
                                                      udp input -- output that stop [SE]

And the main purpose of bringing in UDP system in between is to ensure outputs are not blocked.

 logstash.conf1: |
    input {
      beats {
        port => 5044
      }
     tcp {
        id => "json_txt"
        port => 5025
        codec => json
        type => "json-tcp"
      }
    }
    output {
      udp {
        id => "my_plugin_id"
        port => 5014
        type => syslog
      }
    }    

  logstash.conf2: |
   input {
     udp {
       id => "my_plugin_id"
     }
   }
  output {
    elasticsearch {
      hosts => ["localhost:9200"]
      index => "logs-%{+yyyy.MM.dd}"
    }
  }

How can i achieve this in one logstash system?

2

Yes, look at using Pipeline-to-pipeline communication | Logstash Reference [8.11] | Elastic

You're happy with data loss then?

3

Yes Warkolm, i am good few gets lost.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.