Logstash - Syslog Output - Custom message

vasek · July 13, 2020, 1:44pm

Hi,
I'm working with Logstash - Syslog Output and I've found problem with custom field message. I'm using Elasticstack 7.8.0.

I've installed logstash syslog-output plugin.

/usr/share/logstash/bin/logstash-plugin install logstash-output-syslog

Logstash configuration:

 output {
    syslog {
        host => "localhost"
        sourcehost => "logstash-other-as1"
        port => 10514
        protocol => "tcp"
        message => "testing message"
        #message => "%{custom-message}"
        appname => "apptest"
      }
}

I'm storing all received data to Elasticsearch DB to review them. Data (Document in Kibana) looks like:

<13>Jul 13 13:41:11 logstash-other-as1 apptest[-]: 2020-07-13T13:41:11.000Z %{host} %{message}

I was using the same configuration on ES stack 6.3.2 and it worked well.
Does anyone know how to resolve this issue?

vasek · July 20, 2020, 2:55pm

Is it bug?

vasek · July 29, 2020, 8:36am

I've done a workaround with logstash plugins split and prune.

kelk · July 29, 2020, 8:08pm

Out of curiosity, why are you sending as "syslog" from "logstash to elasticsearch"? Can it be native output from logstash?
Or do you mean you accept "syslog" into logstash and then sent to elasticsearch? if that's the case, then it should be "input" at logstash

vasek · July 30, 2020, 5:20am

I'm receiving events from Redis queue.
The messages are:

stored to ES for internal purposes (+ parsing, + enrichment of documents)
original (RAW) message (no parsing, no transformation) is sent to external syslog server to customer which requires this way

kelk · July 30, 2020, 8:24am

ah.ok. So what you are saying is

Redis queue -> logstash -> (output for) elasticsearch (is Working?)
Redis queue -> logstash -> (output for) external syslog server (NOT working?)

vasek · July 30, 2020, 8:38am

Both of them are working, but in second case (Redis - Logstash - Remote Syslog Server) was problem with defining custom message and host in logstash-output-syslog plugin which can be easily fixed by split and prune in filter section.

Logstash-output-syslog plugin must be installed using:

/usr/share/logstash/bin/logstash-plugin install logstash-output-syslog

kelk · July 30, 2020, 2:37pm

cheers for the clarification.

for the custom message is it by any chance STRUCTRED_MSG?
What's ur template for collecting the syslog at syslog-server?

system · August 27, 2020, 2:37pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.