"message" is empty or non-existent

Elastic Stack Logstash
1

using this input:

input {
  udp {
    port  => 2055
    codec => netflow
  }
}

and stdout:

"@timestamp" => 2019-12-10T11:48:02.921Z,
"netflow" => {
"input_snmp" => 0,
"ipv4_dst_addr" => "77.12.190.94",
"output_snmp" => 0,
"dst_mask" => 30,
"engine_id" => 0,
"ipv4_src_addr" => "10.154.20.12",
"tcp_flags" => 0,
"protocol" => 6,
"l4_src_port" => 9010,
"in_pkts" => 316,
"version" => 5,
"last_switched" => "2019-12-10T11:48:02.708Z",
"flow_seq_num" => 10,
"src_tos" => 0,
"dst_as" => 29576,
"first_switched" => "2019-12-10T11:48:02.315Z",
"l4_dst_port" => 3306,
"flow_records" => 16,
"src_mask" => 23,
"sampling_algorithm" => 0,
"sampling_interval" => 0,
"ipv4_next_hop" => "150.20.145.1",
"engine_type" => 1,
"src_as" => 55858,
"in_bytes" => 228
},
"host" => "172.17.0.2",
"@version" => "1"
}

Now, when I try to use fields to access this content, it doesn't work, and when I try to use filters to access or alter the content, it doesn't work. I am under the impression that all content is imported and stored in the "message" variable, however, when I try to access this variable it is empty. Clearly the content is there or stdout wouldn't ouput anything, so... what is or how do I determine the variable where the content is actually stored so I can access it?

P.S. Apparently code and quote blocks are both not working on this post, so sorry for the janky format on the code.

2

That is not true. Many inputs do put data into a field called [message], but, for example, if you use a json codec you do not get a field called [message], you just get the structured data that was in the JSON. I cannot test it but it may well be that the netflow codec works the same way.

3

That's great to hear, but how exactly do I access the data fields that are delivered to logstash? I'm sure it's something simple that i'm missing but i've been all over this forum and the internet and the documentation and have no answer.

Filters require a variable so they know what they're editing. There doesn't seem to be any documentation on which variable contains the netflow data once it has come into logstash.

4

There is a [netflow] object that contains a bunch of fields like [netflow][ipv4_dst_addr]

5

Negative, I have spent 2 days trying every variation of this including the example you gave and [netflow] contains nothing.

6

I've found the solution...

It seems there is a bug in both the netflow codec and the syslog plugin that created the perfect storm.

the default variable for netflow is "netflow" but it doesn't work. You have to manually define the variable

codec => netflow { target => variable_name}

Then you can use

%{variable_name}

to access the data

Then with syslog, the default message is %{message} and there's no way to change it. So I had to mutate:

add_field {"message" => "%{variable_name}}

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.