New Plugin logstash-input-proc Linux /proc parser

Announcements Community Ecosystem
1

Hi everyone, I wanted to let you know about a new plugin. I wanted to start pulling statistics from the operating system and could not figure out how to get logstash/grock to parse some of the file formats. So, to avoid making Exec calls to ps, lsof, netstat, iostat and other tools. I wrote a plugin that would gather the raw data directly from the /proc directory

Currently I am parsing - hoping to add more
/proc/meminfo, vmstat, diskinfom, loadavg, mounts
/proc/net/dev
/proc//cmdline,exec,environ,stats,status
/proc//fd/*
/proc//task/*

I am right now considering:

  • breaking this in to multiple plugins as some data does not change often while others do.
  • changing the PID output to be multiple outputs rather then just 1 large doc
  • adding more files
  • Creating a sample Kibana dashboard to demo the data

I would love to hear some feed back about what you think of the plugin.

2

Very nice idea!

3

Thanks, I hope everyone likes it,

Now I am working on some Kibana Dashboards to display some of the data. It seems like it is starting to come together.

I think next week I will split it in to 2 or 3 different plugins.

  • Process monitoring and statistics

  • Static system information (Cpuinfo, PCI bus, and other information related to the hardware

  • Highly Volatile data like MEMINFO,VMSTATS, NET/DEV, ....

I found that with all the files enabled it takes ~1 second to grab every file, not bad when sar collects by default once every 10 minutes.

4

The plugin is now available via RubyGems

https://rubygems.org/gems/logstash-input-proc
http://eperry.github.io/logstash-input-proc/

##install
${LS_HOME}/bin/plugins install logstash-inputs-proc

use

Example Config all features enabled

input {
    proc {
        interval=>60
        vmstats =>{ }
        loadavg =>{ }
        meminfo =>{ }
        pidstats =>{ 
            user => "root"
        }
        
    }
}

output { 
    stdout{ 
        codec=>"rubydebug"
    }
}

##Example Minimal

input {
    proc {
        interval=>60
        meminfo =>{ }
    }
}

output { 
    stdout{ 
        codec=>"rubydebug"
    }
}
5

Just saw this... Very cool...
I process Sosreports and a couple other log bundles that do a basic proc dump. Mind if I fork and setup a config option to pull from a non standard path?

6

Go a head that is the wonders of github feel free to fork it.

7

We've also implemented a similar thing via Topbeat https://www.elastic.co/guide/en/beats/topbeat/current/index.html

8

I might go look at how your collecting the stats. Though I like the idea of only having to run logstash on a server. One of the biggest issue I am starting to see is all the different agents I have to run on servers

Satellite runs gofred
Nagios and Ganglia want an agent
Puppet wants an agent (Or Salt)
NewRelic wants an agent
Oracle Monitoring or Websphere/Weblogic Node Agent
Plus all the systems background processes

Then to add Logstash, packetbeat and topbeat on top of this. I seem to add up to 1 or 2 gigs just for monitoring and management.

Fortunately I run systems with +100GB memory so not that big of issue but this is starting to make me think about how much of what is running might be consolidated in to one or two tools.

Eh, guess this is the next hurdle to think about in IT management.

9

Hi, I newly installed ES 2.0 with Logstash 2.0.0 and cannot download your plugin, maybe because of
the Runtime dependency: logstash-core < 2.0.0, >= 1.4.0
on https://rubygems.org/gems/logstash-input-proc/versions/0.3.1.

May you change this or is there a hidden option to force download?

10

Let me look at it, I have not tested it with LS 2.0 nor looked at the new API but lets see

1 Like
12

Sounds so cool. Is there any plans to upgrade this to work with the latest LS releases?

13

you know I did not think about it, it might work as is. It is not doing more then a standard search. I will look at it
you may want to check out this project though