Does Topbeat Also Send to UDP 25826 By Default?

Elastic Stack Beats
1

I'm learning ELK and had a setup for Topbeat (server metrics) and Filebeat going into a Logstash config over port 5000. That's working fine and I was able to load the Topbeat Dashboards, etc.

I wanted to use CollectD to monitor my Redis cluster (aside from the Topbeat metrics). So I created a new logstash conf (collectd.conf) and my config is below:

input {
  udp {
    port => 25826
    buffer_size => 1452
    codec => collectd { }
    type => "collectd"
  }
}
output {
  elasticsearch {
    hosts => ["xx.xx.xx.xx:9200"]
    sniffing => true
    manage_template => false
    index => "collectd-%{+YYYY.MM.dd}"
    document_type => "collectd"
  }
}

I started getting CollectD metrics (I had recognized them from when I was testing out collectd before topbeat). My ELK was collecting data from 6 nodes and all 6 were sending data to the collectd-* indexes. None of these nodes were running collectd and once I stopped topbeat, it stopped collecting data. For reference, here is my beats.conf

input {
  beats {
    host => "xx.xx.xx.xx"
    port => 5000
    codec => "json"
  }
}
output {
  if [type] == "system" or [type] == "filesystem" or [type] == "process" {
    elasticsearch {
      hosts => ["xx.xx.xx.xx:9200"]
      sniffing => true
      manage_template => false
      index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
      document_type => "%{[@metadata][type]}"
    }
  }
  if [type] == "nginx-access" {
    elasticsearch {
      hosts => ["xx.xx.xx.xx:9200"]
      sniffing => true
      manage_template => false
      index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
      document_type => "%{[@metadata][type]}"
    }
  }
}

When I was using filebeat/topbeat, these were getting properly indexed. Once I configured the collectd.conf, I started receiving data already before collectd was running on my test node.

2

I started debugging the issue by running /opt/logstash/bin/logstash -f /etc/logstash/conf.d/{logstash_name}.conf -v --debug against each configuration and stopped/started topbeat. Nothing comes in to the collectd index

However, when I run the logstash service and both .conf's are loaded, the topbeats data is sent to collectd index. Please note that I'm not even running collectd anywhere in my tests. Just one node sending data via topbeat.

3

Topbeat does not support collectd protocol and is not sending any data to any uncofigured port by magic.

You run logstash with both config files enabled at the same time? In this case you might want to guard the elasticsearch config for collectd output by checking the type for example. Config files in logstash are not separate entities, but all configs (when put into directory) are merged into one big config. That is, all events will be indexed due to collectd config not having a conditional guarding the output from unwanted events.

4

Makes sense. I will add a conditional to it.

Thanks!