If Else in Logstash

Elastic Stack Logstash
1

Hi ,
i am trying to create an index when the condition is if [fs.mount_point] == "C:" and [fs.used] == "87264018432"
i am trying to push data from topbeat to logstash. and then i am trying to give the if condition. But i am not getting the desired output.

This is my configuration
input {
beats {
port => 5044
}
}

filter {
grok {
match =>["message", "%{GREEDYDATA:Data}"]}
}

output {

if [fs.mount_point] == "C:" and [fs.used] == "87264018432"
{
stdout { codec => rubydebug }

elasticsearch { hosts => ["localhost:9200"] index => "topbeat_test_2"
template => "./apache_template.json"
template_name => "*"
template_overwrite => true }
}
}

and the output which i am getting is:
D:\LogAnalyzer-3.0\logstash-2.4.0\bin>logstash agent -f eyeshare1.config
←[31mfetched an invalid config {:config=>"input { \n beats {\n port => 5044 \n }\n }\n\n\nfilter { \n grok {\n match =>["message", "%{GREEDYDATA:Data}"]}\n}\n\n\noutput {\n\nif [fs.m
ount_point] == "C:\" and [fs.used] == "87264018432"\n{\n\tstdout { codec => rubydebug }\n\t\nelasticsearch { hosts => ["localhost:9200"] index => "topbeat_test_2"\ntemplate => "./apache_template.
json"\ntemplate_name => "*"\ntemplate_overwrite => true } \n}\n}\n", :reason=>"Expected one of #, and, or, xor, nand, { at line 16, column 48 (byte 184) after output {\n\nif [fs.mount_point] == "C:\"
and [fs.used] == "", :level=>:error}←[0m
The signal HUP is in use by the JVM and will not work correctly on this platform

i think the way i am writing the config file is wrong. Kindly guide me in the right direction. Thanks

2

Try "C:\\" instead of "C:\".

Secondly, if mount_point is a subfield of fs you need to say [fs][mount_point] and not fs.mount_point. See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references. Same thing for fs.used of course.

3

@magnusbaeck thanks for your prompt response.
I made the changes as suggested by you, but still i am getting the same error.

Logstash configuration:

input {
beats {
port => 5044
}
}

filter {
grok {
match =>["message", "%{GREEDYDATA:Data}"]}
}

output {

if [fs][mount_point] == "C:\" and [fs][used] == "87264018432"
{
stdout { codec => rubydebug }

elasticsearch { hosts => ["localhost:9200"] index => "topbeat_test_2"
template => "./apache_template.json"
template_name => "*"
template_overwrite => true }
}
}

output:

D:\LogAnalyzer-3.0\logstash-2.4.0\bin>logstash agent -f eyeshare1.config
←[31mfetched an invalid config {:config=>"input {\nbeats {\nport => 5044 \n}\n}\n\nfilter { \ngrok {\nmatch =>["message", "%{GREEDYDATA:Data}"]}\n}\n\noutput {\n\nif [fs][mount_point] == "C:\\" and
[fs][used] == "87264018432"\n{\nstdout { codec => rubydebug }\n\nelasticsearch { hosts => ["localhost:9200"] index => "topbeat_test_2"\ntemplate => "./apache_template.json"\ntemplate_name => "*"
\ntemplate_overwrite => true } \n}\n}\n", :reason=>"Expected one of #, and, or, xor, nand, { at line 14, column 51 (byte 159) after output {\n\nif [fs][mount_point] == "C:\\" and [fs][used] == "", :le
vel=>:error}←[0m
The signal HUP is in use by the JVM and will not work correctly on this platform

Is it because the logstash is not able to recognize the fields [fs.mount_point] and [fs.used]?
If this is the case how to get these two particular fields from topbeat?

4

From what I read it fails on the " what happens if you remove the " around the number?

5

Okay, it seems Logstash's awful handling of escape sequences in strings is at fault. This will at least pass the syntax check:

if [fs][mount_point] =~ /^C:\\$/ and [fs][used] == "87264018432"
6

ya, its passing the syntax check, but i am still not getting the desired output. The if condition is not working.

Logstash Config:
input {
beats {
port => 5044
}
}

filter {
grok {
match =>["message", "%{GREEDYDATA:Data}"]}
}

output {

if [fs][mount_point] =~ /^C:\$/ and [fs][used] == 87264018432
{
elasticsearch { hosts => ["localhost:9200"] index => "topbeat_test_2"
template => "./apache_template.json"
template_name => "*"
template_overwrite => true }
stdout { codec => rubydebug }
}
}

output:

D:\LogAnalyzer-3.0\logstash-2.4.0\bin>logstash agent -f Topbeat.config
Settings: Default pipeline workers: 4
log4j:WARN No appenders could be found for logger (io.netty.util.internal.logging.InternalLoggerFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See Apache log4j 1.2 - Frequently Asked Technical Questions for more info.
Pipeline main started

The index is also not created.

7

@pjanzen i tried it but still not getting the desired output.

8

Put the stdout output outside the conditional and post the results here formatted as preformatted text using the </> toolbar button so that we can see exactly what the event looks like.

9 
  {
    "@timestamp" => "2017-02-07T08:52:59.877Z",
          "beat" => {
        "hostname" => "BLRWFD1538",
            "name" => "BLRWFD1538"
    },
         "count" => 1,
            "fs" => {
        "device_name" => "D:\\",
              "total" => 250053914624,
               "used" => 97300754432,
             "used_p" => 0.39,
               "free" => 152753160192,
              "avail" => 152753160192,
              "files" => 0,
         "free_files" => 0,
        "mount_point" => "D:\\"
    },
          "type" => "filesystem",
      "@version" => "1",
          "host" => "BLRWFD1538",
          "tags" => [
        [0] "beats_input_raw_event"
    ]
}
{
    "@timestamp" => "2017-02-07T08:52:59.877Z",
          "beat" => {
        "hostname" => "BLRWFD1538",
            "name" => "BLRWFD1538"
    },
         "count" => 1,
          "proc" => {
         "cmdline" => "\"LogonUI.exe\" /flags:0x0",
             "cpu" => {
                  "user" => 109,
                "user_p" => 0,
                "system" => 156,
                 "total" => 265,
            "start_time" => "Jul20"
        },
             "mem" => {
             "size" => 8462336,
              "rss" => 13672448,
            "rss_p" => 0,
            "share" => 0
        },
            "name" => "LogonUI.exe",
             "pid" => 10800,
            "ppid" => 0,
           "state" => "running",
        "username" => "NT AUTHORITY\\SYSTEM"
    },
          "type" => "process",
      "@version" => "1",
          "host" => "BLRWFD1538",
          "tags" => [
        [0] "beats_input_raw_event"
    ]
}
{
    "@timestamp" => "2017-02-07T08:53:09.634Z",
          "beat" => {
        "hostname" => "BLRWFD1538",
            "name" => "BLRWFD1538"
    },
         "count" => 1,
           "cpu" => {
            "user" => 214977942,
          "user_p" => 0.0651,
            "nice" => 0,
          "system" => 49926248,
        "system_p" => 0.025,
            "idle" => 1399433256,
          "iowait" => 0,
             "irq" => 0,
         "softirq" => 0,
           "steal" => 0
    },
          "load" => {
         "load1" => 0,
         "load5" => 0,
        "load15" => 0
    },
           "mem" => {
                "total" => 3644895232,
                 "used" => 3176570880,
                 "free" => 468324352,
               "used_p" => 0.87,
          "actual_used" => 890712064,
          "actual_free" => 1256640512,
        "actual_used_p" => 0.24
    },
          "swap" => {
         "total" => 0,
          "used" => 0,
          "free" => 0,
        "used_p" => 0
    },
          "type" => "system",
      "@version" => "1",
          "host" => "BLRWFD1538",
          "tags" => [
        [0] "beats_input_raw_event"
    ]
}

this is the output i am getting.

10

Right, although those events obviously don't match your condition. Apart from that it looks like your conditional would work.

Do you really want the condition to include [fs][used] == 87264018432? What's the point of sending data to ES if the amount of free space is exactly 87264018432 bytes?

11

yes i want to include [fs][used] , actually i have a use case wherein i have to trigger an email if the used space on disk is 50 GB.
So , i was trying to test it out using [fs][used] == 87264018432, as it is what is available now.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.