Logstash won't pick up logs over port 514

Elastic Stack Logstash
1

Hello All,

So my logstash installation (full elastic stack running on one machine, ubuntu 19) is configured to accept syslogs from our firewall over UDP on port 514. I can use tcpdump to see that there is indeed traffic coming in on port 514. However, Logstash is picking up nothing. I turned on debug-level logging and here is logstash's logs:

[2019-06-12T11:49:54,677][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.8.0"}
[2019-06-12T11:50:05,081][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-06-12T11:50:05,555][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-06-12T11:50:07,117][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2019-06-12T11:50:07,764][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-06-12T11:50:07,766][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2019-06-12T11:50:07,787][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2019-06-12T11:50:07,796][INFO ][logstash.outputs.elasticsearch] Using default mapping template
[2019-06-12T11:50:07,814][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-06-12T11:50:07,837][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2019-06-12T11:50:07,850][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-06-12T11:50:07,851][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2019-06-12T11:50:07,855][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2019-06-12T11:50:07,941][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dy$
[2019-06-12T11:50:09,267][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-5.0.3-java/vendor/GeoLite2-City.mmdb"}
[2019-06-12T11:50:15,102][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2019-06-12T11:50:15,129][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2c3d05cc run>"}
[2019-06-12T11:50:15,528][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"192.168.50.225:514"}
[2019-06-12T11:50:15,673][INFO ][org.logstash.beats.Server] Starting server on port: 5044
[2019-06-12T11:50:15,992][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-06-12T11:50:16,396][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"192.168.50.225:514", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2019-06-12T11:50:17,428][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

I have looked at many other topics to try to find other solutions but I have found none that work. I used setcap to allow binding to port 514. Here is my logstash config file:

input {
  udp {
    host => "192.168.50.225"
    port => 514
    type => "syslog"
  }
}

filter { }

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    manage_template => false
    index => "syslog-%{+YYYY.MM.dd}"
  }
}

If I need to provide more information just let me know. Please help.

2

This indicates that UDP is specifically bound to a single IP; does the traffic you see in tcpdump target this IP? It is possible to bind the UDP input to all IPs provided by all interfaces by specifying 0.0.0.0.

3

Oh, it was my understanding that the host described the localhost in this instance (it gave errors along the lines of "unable to bind" when set to the source of the traffic), but I removed the line it in a later attempt to no avail.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.