Wrong filebeat hostname in logs

Elastic Stack Beats
1

I have 100 systems that are sending rsyslog messages to a syslog server.
I have installed the ELK stack and filebeats on that server.

Filebeats is configured to monitor the logs that are being populated by syslog and send to the ELK stack.
Here is the filebeat.input section:

#=========================== Filebeat inputs =============================

filebeat.inputs:

Each - is an input. Most options can be set at the input level, so

you can use different inputs for various configurations.

Below are the input specific configurations.

  • type: log

    Change to true to enable this input configuration.

    enabled: true

    Paths that should be crawled and fetched. Glob based paths.

    paths:

    • /var/log/*.log
    • /var/secure
    • /var/log/audit/audit.log

When looking at logs in kibana, all logs, regardless of what client sent them, have the same host.name .
In this case chiops

host.name chiops

chiops is the name of the syslog server and its also the ELK stack server.
The 'message' part of the log entry in Kibana shows the actual client IP address

For example:
message Aug 12 14:58:50 192.168.2.1 miniupnpd[778]: HTTP Connection from 192.168.2.248 closed unexpectedly

I would like to see the IP address of the actual client that sent the log and not the host servers name. as the host.name and not the server.

Can someone advise on how to configure this?

Here is my logstash configuration :

input {
beats {
port => 5044
}
}

filter {
mutate {
add_field => { "remote_ip" => "%{[@metadata][ip_address]}" }
}
if [fileset][module] == "system" {
if [fileset][name] == "auth" {
grok {
match => { "message" => ["%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system$
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostnam$
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostnam$
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostnam$
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostnam$
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostnam$
"%{SYSLOGTIMESTAMP:[system][auth][timestamp]} %{SYSLOGHOST:[system][auth][hostnam$
pattern_definitions => {
"GREEDYMULTILINE"=> "(.|\n)"
}
remove_field => "message"
}
date {
match => [ "[system][auth][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
geoip {
source => "[system][auth][ssh][ip]"
target => "[system][auth][ssh][geoip]"
}
}
else if [fileset][name] == "syslog" {
grok {
match => { "message" => ["%{SYSLOGTIMESTAMP:[system][syslog][timestamp]} %{SYSLOGHOST:[syst$
pattern_definitions => { "GREEDYMULTILINE" => "(.|\n)" }
remove_field => "message"
}
date {
match => [ "[system][syslog][timestamp]", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
}

output {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}

2

Please use the </> button in the editor window to format logs and configs.

Filebeat does not parse the syslog messages, but forwards lines found in the files as is. Filebeat adds some metadata to events, like the host name the event was collected from. As you send your events via logstash and all parsing happens within logstash, you might want to adapt your logstash filters to overwrite host.name.

3

Thanks! That was very helpful.
Do you have a sample or example logstash filter configuration that shows how to have logstash overwrite host.name?

4

Thanks! That was very helpful.
Does anyone have a sample or example logstash filter configuration that shows how to have logstash overwrite host.name?

5

grok_pattern
grok_pattern.png1861×1032 55.4 KB

Please test your message and grok pattern : http://grokdebug.herokuapp.com

Then you can use it in Filebeat input->processors
(https://www.elastic.co/guide/en/beats/filebeat/current/defining-processors.html)

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.