Unexpected error with Beats input plugin

Elastic Stack Logstash
1

Previously i used JSON data from a file (that were previously taken by beats) to test my logstash conf and filtering. I had no problems, but when i try to read straight from auditbeat with the Beats input plugin, Logstash prints few errors and also the output doesn't appear.

here is sample of the error form logstash while receiving auditbeat events:


[2021-08-12T19:46:31,639][WARN ][logstash.filters.json    ][main][699e35f42d506ad9de208d3e678ec089b2419d6ee7373c8b174f7f900352ed56] Error parsing json {:source=>"message", :raw=>"Process containerd-shim (PID: 5305) by user root is RUNNING", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'Process': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"Process containerd-shim (PID: 5305) by user root is RUNNING"; line: 1, column: 9]>}
[2021-08-12T19:46:31,640][WARN ][logstash.filters.json    ][main][699e35f42d506ad9de208d3e678ec089b2419d6ee7373c8b174f7f900352ed56] Error parsing json {:source=>"message", :raw=>"ERROR for PID 5323: failed to hash executable /pause for PID 5323: failed to stat file /pause: stat /pause: no such file or directory", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'ERROR': was expecting ('true', 'false' or 'null')

and another sample:

[2021-08-12T19:51:56,036][WARN ][logstash.filters.json    ][main][699e35f42d506ad9de208d3e678ec089b2419d6ee7373c8b174f7f900352ed56] Error parsing json {:source=>"message", :raw=>"Process sleep (PID: 26212) by user root STOPPED", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'Process': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"Process sleep (PID: 26212) by user root STOPPED"; line: 1, column: 9]>}
[2021-08-12T19:52:06,041][WARN ][logstash.filters.json    ][main][699e35f42d506ad9de208d3e678ec089b2419d6ee7373c8b174f7f900352ed56] Error parsing json {:source=>"message", :raw=>"Process sleep (PID: 26625) by user root STARTED", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'Process': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"Process sleep (PID: 26625) by user root STARTED"; line: 1, column: 9]>}

Here is the logstash conf:

input{
    # file{
    #     path => "/usr/share/logstash/config/auditbeat.json"
    #     start_position => "beginning"
    #     ecs_compatibility => "v1"
    #     # codec => "json"
    # }
    beats {
        port => 5044
        ecs_compatibility => "v1"
    }
}

filter{

    json {
        source => "message"
        target => "[@metadata][message]"
        remove_field => "message"
    }
    if [@metadata][message][destination][ip] {
        mutate{
            add_field => { "[dst_ip]" => "%{[@metadata][message][destination][ip]}"}
        }
    }

    if [@metadata][message][source][ip] {
        mutate{
            add_field => { "[src_ip]" => "%{[@metadata][message][source][ip]}"}
        }
    }
    if ![src_ip] and ![dst_ip]{
        drop {}
    }

    mutate{
        remove_field => [ "[@metadata][message]" ]
    }

.
.
.
.


output{
    # stdout {
    #    codec => rubydebug
    #    { metadata => true }
    # }
    file{
        path => "/usr/share/logstash/config/enriched-auditbeat.json"
        codec => "json"
    }
}
2

What i want to achieve is to take specific fields from the beats input in order to check them later in the filter. So i don't want all the data from the input. Just specific JSON fields.

3

"Process containerd-shim (PID: 5305) by user root is RUNNING"

That is not JSON, so your json filter warns you about that. Not sure what else you would expect. It does not look like auditbeat data either.

4

It is Auditbeat and it IS JSON. But that JSON has also a message field I guess.
Tomorrow I will post you what I get as pure input without the filter which causes the errors.

5

So here is the raw input i receive from auditbeat

{
       "process" => {
        "working_directory" => "/home/user",
                      "pid" => 27377
    },
         "agent" => {
            "hostname" => "lent",
                "type" => "auditbeat",
             "version" => "7.13.2"
    },
     "@metadata" => {
          "input" => {
            "beats" => {
                "host" => {
                    "ip" => "20.86.153.197"
                }
            }
        },
           "type" => "_doc",
           "beat" => "auditbeat",
        "version" => "7.13.2"
    },
        "auditd" => {
              "result" => "success",
             "summary" => {
             "actor" => {
                "secondary" => "azureuser",
                  "primary" => "azureuser"
            },
            "object" => {
                   "type" => "process",
                "primary" => "systemctl stop auditbeat.service"
            }
        },
            "sequence" => 163,
                "data" => {
            "terminal" => "pts/0",
                 "cmd" => "systemctl stop auditbeat.service"
        },
             "session" => "1",
        "message_type" => "user_cmd"
    },
          "tags" => [
        [0] "beats_input_raw_event"
    ],
         "cloud" => {
        "instance" => {
            "name" => "lent",
 
        },
        "provider" => "azure",
         "machine" => {
            "type" => "Standard_B2s"
        },
         "service" => {
            "name" => "Virtual Machines"
        },
          "region" => "europe",
         "account" => {}
    },
    "@timestamp" => 2021-08-13T06:37:14.359Z,
           "ecs" => {
        "version" => "1.9.0"
    },
       "service" => {
        "type" => "auditd"
    },
      "@version" => "1",
          "host" => {
             "hostname" => "lent",
                   "os" => {
            
            "codename" => "bionic",
                "name" => "Ubuntu",
                "type" => "linux",
            "platform" => "ubuntu"
        },
        "containerized" => false,
                   "ip" => [
            [ 0] "10.0.1.5",
    
        ],
                 "name" => "lent",
                  "mac" => [
              [ 1] "ee:ee:ee:ee:ee:ee",
            
        ],
         "architecture" => "x86_64"
    },
         "event" => {
            "kind" => "event",
          "module" => "auditd",
          "action" => "ran-command",
            "type" => [
            [0] "start"
        ],
        "category" => [
            [0] "process"
        ],
         "outcome" => "success"
    },
          "user" => {
        "audit" => {
            "name" => "user",
              "id" => "100"
        },
         "name" => "user",
           "id" => "100"
    },
    "message": "Process sleep (PID: 26212) by user root STOPPED"
}

Look at the end of this JSON. There is a message field! Maybe that is causing the damage. Do you know how can i deal with it?

As a stated earlier what i want is just to get few specific fields from that Json an "build" my own event via my pipeline.

What should i do? What i had in mind is that when an event comes ALL this JSON data would be inside a MESSAGE filed. That's why in my conf in my Json Filter i do source => "message".

6

You could use mutate+add_field to copy all the fields you actually do want, then

prune { blacklist_names => [ "agent", "auditd", "cloud", "ecs", "event", "host", "message", "process", "service", "user" ] }

Or whitelist the fields you want to keep after adding them.

1 Like
7

Ok, got it.
Two questions then, so when reading from a file i get all the data into the "message" field while when i get input from beats, data comes straight without being into a "message" field right ?

Second, about the prune filter, when i whitelist a filed which has many "son" fields, i hope it keeps also the son fields right?

8

Yes a file input puts the line from the file into the message field. beats data is usually much more structured. For other inputs it varies between the two extremes.

prune only operates on top level fields. If a top-level field is blacklisted anything inside it is also removed, if whitelisted the contents stay.

1 Like
9

So the solution to my problem was that BEATS input plugin DOES NOT put the data into a "message" field as FILE input does.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.