Syslog Output only transferring the "message" field

Elastic Stack Logstash
1

Hi, I am using the Syslog Output plugin to transfer data to another host:

syslog {
   host => "IP"
   port => Port
}

However, the target host only receives the content of the message fields.
My input looks like this:

input {
    beats {
      port => Port
   }
}

How can I use Syslog to transmit not only the message field of each event, but the entire Winlog event data?

2

To ensure that Logstash transmits the entire Winlog event data via Syslog, ensure the following:

  1. Input Configuration: Set up the Beats input plugin to receive Winlog event data.
  2. Filtering: Optionally, apply filters if needed to parse or modify the incoming Winlog event data.
  3. Output Configuration: Use the syslog output plugin with the appropriate host and port settings. Set the output format to include the entire event using %{message} or the appropriate field containing the Winlog event data.
input {
  beats {
    port => Port
  }
}

filter {
  # Add any necessary filters here to parse or manipulate the incoming data
}

output {
  syslog {
    host => "IP"
    port => Port
    # Include the entire event by referencing the 'message' field which typically contains the JSON representation of the event
    format => "%{message}"
  }
}
3

Thank you, but that is exactly my problem, that the message field does not contain all the event data. Additionally, there is no format field accoring to the syslog output plugin documentation Syslog output plugin | Logstash Reference [8.12] | Elastic

4

I notice the same for some event without LS, WLB doesn't send full event log content. I would say, you are looking on the wrong place. From some reason WLB removes some part.

5

I also send the WLB output to a text file and see a lot more information there like the event ID or the event provider name. I just need a way to send the entire event data and not just the message field.

6

You can also include debug log or set output in console:

output.console:
  pretty: true

Maybe this is an issue: bulk_max_size

The maximum number of events to buffer internally during publishing. The default is 2048.

7

That is not the problem in this case. WLB works perfectly fine. I can see all the necessary details via the stdout output and also fully parsed data in Elasticsearch. The only problem is the syslog output. I either need to rename the current message field to something else and paste the entire event data into a newly created message field or need a solution to send the entire event data via syslog instead of just the message field.

8

You could use a Ruby filter to parse the entire event as json (or key=value) into a field and then use the message field on the syslog output to use the json field you created as the content. On the receiving side you'd parse the json back into a message.

See Logstash-output-syslog full json in message? - #6 by Badger

9

Nice, thats a good idea. I am using a ruby filter to create a new field named "combined_data" using code => ' event.set( "combined_data", event.to_json ) '

However, my syslog output still only transmits the content of the message field. My configuration looks like this:

output {
   syslog {
      host => ...
      port => ...
      message => "%{combined_data}"
}
1 Like
10

I believe there is a bug with the syslog output plugin and the message field and there is a workaround provided in this issue on GitHub "message" configuration parameter ignored in Logstash 7.2.0 and up · Issue #51 · logstash-plugins/logstash-output-syslog · GitHub

codec => plain { message => "%{combined_data}" }

11

This results in Unknown setting 'message' for plain. The same applies to codec => json. Using Logstash version 7.17.18.

12

Can you try format instead of message?

13

Does not result in any errors, but also does not change the syslog message. I have tried the following versions:

output {
   syslog {
      ...
      message => "test"
      codec => plain { format => "%{message}" }
   }
}

and

output {
   syslog {
      ...
      message => "%{combined_data}"
      codec => plain { format => "%{combined_data}" }
   }
}
14

The syslog output is actually a community plugin so we have limited ability to support troubleshooting.

Luckily syslog is basically just a udp message with a timestamp and the hostname in it.

You could try switching to the udp output via

udp {
  host => "localhost"
  port => 3001
  codec => plain {
    format => "%{message}"
  }
}
1 Like
15

Well, I'll have to convert the message to the syslog format myself then, but it works great! Thank you!

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.