Ganglia Packets as Input in Logstash

(Christos Markou) #1

Hello guys,

i would like to take ganglia packets in Logstash as input. From Logstash documention ( i make that i have to configure my Logstash Server to listen on a port (8649) and an address to listen on (my Logstash Server IP?). Right?

But when it comes to Ganglia's side, how i can configure Ganglia to send the packets to my Logstash Server?

I know that this question is not mainly about Logstash itself, but if someone has done sth similar or have an idea about this please share.! :wink:


Ganglia into logstash
(Joshua Rich) #2

Probably the easiest solution would be to adjust the gmond.conf file on each node to be monitored and set up a new udp_send_channel configuration directive that points at your Logstash server:

Something like the following in your gmond.conf should work:

 udp_send_channel {
   host = my.logstash.server
   port = 8649
   ttl = 1

(Christos Markou) #3

Thanx for the quick reply!

I will check this and leave feedback soon.

(Christos Markou) #4

Hi again,

we have configured gmond.conf without trouble as mentionde above, but in my LogstashServer side nothing seems to work. When i start logstash service everything is fine but it seems that logstash doesn't listen on port 8649.
My configuration file is:

input {

tcp {
type => "apache"
port => 3333

ganglia {
port => 8649
type => "ganglia"

lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"

And netstat gives:
ubuntu@logstashserver:~$ netstat -nat | grep LISTEN
tcp 0 0* LISTEN
tcp 0 0* LISTEN
tcp 0 0* LISTEN
tcp6 0 0 :::22 :::* LISTEN
tcp6 0 0 :::3333 :::* LISTEN
tcp6 0 0 :::5000 :::* LISTEN

->Port 8649 doesn't appear...

logstash.log doesn't give any error or warning.
Any ideas???

(Christos Markou) #5


i am refresing the topic hoping for an answer. So if anyone has done something similar (connecting ganglia and logstash) and has the experience plz share because i am stack.

(Joshua Rich) #6

Hey @ChrisMark,

Ganglia uses UDP by default and the LS plugin follows along. So to check it's listening, change the -t in your netstat command to -u, i.e., try netstat -nau to see Logstash listening on UDP port 8649.

Hope this helps!

(Christos Markou) #7


i followed the suggestion and i see:

ubuntu@logstashganglia:~$ netstat -nau
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0*
udp 0 0*
udp6 0 0 :::8649 :::*
udp6 0 0 :::54328 :::*
udp6 0 0 :::10124 :::*

So i make that connection is not ESTABLISED ???

(Joshua Rich) #8

You probably won't generally see a constant ESTABLISHED connection in netstat for Ganglia. UDP is connectionless, so clients will just be sending a data stream to the Logstash server when needed. They won't leave the connection open like you might see with some programs using TCP.

(Christos Markou) #9

Ok got it, but then how i will figure out if packets are coming?
The fact that i have not configured a filter for gagnlia's type logs may affect the whole thing?

(Joshua Rich) #10

Even without filters in place, Logstash is still processing any Ganglia events that are sent. What outputs do you have configured? Try adding a simple stdout output like the following and then watch the /var/log/logstash/logstash.stdout file for any Ganglia events:

output {
  if [type] == "ganglia" {
    stdout {
      codec => "rubydebug"

(Christos Markou) #11

Τhat's a good idea, but i think i have located the problem. As i posted before my Logstash listens ubuntu@logstashganglia:~$ netstat -nau
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
udp 0 0*
udp6 0 0 :::8649 :::*

This means that 8649 udp is on my ipV6, rigth? So i have to configure Ganglia's side to send the packets on my ipv6 not to ipv4? Right?

(Joshua Rich) #12

If you aren't using IPv6, it will probably be best to configure Logstash to listen on IPv4 only. You can do this by editing either /etc/default/logstash (on Debian/Ubuntu) or /etc/sysconfig/logstash (on RedHat/CentOS) and adding to the LS_JAVA_OPTS setting. After making this change, restart Logstash to have it listen just on an IPv4 address.

(Saggarsunil) #13

Hi Chris:

I am also trying to configure ganglia as input to logstash. As of now, i am running logstash and ganglia on the same machine.

Ganglia command line:
bin/logstash -e 'input { ganglia { host=>"" port=> 8686 } } output { stdout {} }'

As you can see, i am using a different port for logstash and it starts fine BUT i don't see any ganglia messages ..

All i see is messages like this:
2015-08-23T14:11:41.571Z %{message}
2015-08-23T14:11:41.572Z %{message}
2015-08-23T14:11:41.572Z %{message}
2015-08-23T14:11:41.573Z %{message}
2015-08-23T14:12:01.446Z %{message}

On the ganglia conf file, i am using 2 udp send channels.
udp_send_channel {
#bind_hostname = yes # Highly recommended, soon to be default.
# This option tells gmond to use a source address
# that resolves to the machine's hostname. Without
# this, the metrics may appear to come from any
# interface and the DNS names associated with
# those IPs will be used to create the RRDs.
mcast_join =
port = 8649
ttl = 1

udp_send_channel {
host =
port = 8686

I am trying to debug further but any help at this point will really help in expedite the problem resolution.


(Magnus Bäck) #14

@saggarsunil – Since your question is unrelated to the original question (you're receiving messages just fine) please start a new topic.

This is clearly a case of poor documentation and I've filed issue #8 on your behalf.


Hi magnus

Following up form the earlier message of saggarsunil
I also am in same situation, I see %{message} on the stdout

However when i use the below option..'input {ganglia {port=>8649 type=>"ganglia" }} output { stdout { codec => rubydebug } file { path => "/tmp/gmond-log.txt"}}'

i see continuous steram of following data (with some value changiing)

@version" => "1",
"@timestamp" => "2015-08-28T11:00:55.541Z",
"log_host" => "E8a7--13",
"dmax" => 0,
"tmax" => 180,
"slope" => "both",
"type" => "float",
"units" => "KB",
"host" => ""

It seems not all data is comming from gmond but only static information
Howevedr when i run telnet localhost 8649, I see all of the data (below is the snippet for it)

METRIC NAME="swap_free" VAL="4194296" TYPE="float" UNITS="KB" TN="26" TMAX="180" DMAX="0" SLOPE="both
Why logstash is not outputting the value of the all the caputured metrics ( as seen in telnet command)

How do i get all the metrics data printed through logstash so that it can be caputerd by elasticsearch

I am stuck and dont know the way forward !!

P.S I was facing this issue where logstash was not outputing anything from ganglia
It was only after i ran iptables --flush, i was able to get above shown output through logstash

(Magnus Bäck) #16

The issue below seems relevant to your problem.

(Josh Star Long) #18

Hi @magnusbaeck,

I was able to send some metrics of ganglia to logstash. Here is my input configuration
input {
lumberjack {
port => 5043
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
udp {
port => 8649
codec => json_lines
This configuration is working and I able to visaulized my ganglia metric the awesome kibana dashbroad. But, logstash is unable to understand the message of the metric. That is the message look like
"message" => "\u0000\u0000\u0000\x86\u0000\u0000\u0000\u0010ip-172-31-37-235\u0000\u0000\u0000\fload_fifteen\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0004%.2f=L\xCC\xCD\u0000\u0000\u0000\x84\u0000\u0000\u0000\u0010ip-172-31-37-235\u0000\u0000\u0000\theartbeat\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0002%u\u0000\u0000V\xAC\xDFF\u0000\u0000\u0000\x84\u0000\u0000\u0000\u0010ip-172-31-37-235\u0000\u0000\u0000\theartbeat\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0002%u\u0000\u0000V\xAC\xDFF\u0000\u0000\u0000\x86\u0000\u0000\u0000\u0010ip-172-31-37-235\u0000\u0000\u0000\bmem_free\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0004%.0fH\xEBK\u0000\u0000\u0000\u0000\x86\u0000\u0000\u0000\u0010ip-172-31-37-235\u0000\u0000\u0000"
I have attach a screenshot.
Now my question is how can I configure logstash to understand the messages sent by ganglia.
Thanks for the concern.

(Magnus Bäck) #19

@jstar—please start a new thread for your question.

(system) #20