Logfiles seem to be combining!

Elastic Stack Kibana
1

Hello,

I have 2 inputs for log files with 2 separate types for example Internal on one and External on another.

For some reason when the data reaches Kibana it is joining them together. For example
"85.833, 20.233, 85.833, 20.233" all in one field. Any ideas it is driving me potty!

cheers
Pete.

2

I was having that same issue - and I think it was due to using multiline in both Filebeat and Logstash. Once I removed my multiline plugins in Logstash I stopped getting those errors.

Of course, you may also be getting that error if you are accidentally adding to that field multiple times in your Logstash config.

3

Hi Michelle,

Are you able to explain a bit more about the filebeat?

Here is my config

input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "crt"
ssl_key => "key"
}
}
input {
file {
path => "to logfile"
type => "snort_external" # a type to identify those logs (will need this later)
start_position => "beginning"
}
}
#Snort
filter{
grok {
match => { "message" => "%{SYSLOG5424SD} ((%{DATA:snort_rule_mod} ))?(%{DATA:snort_rule}) %{GREEDYDATA:snort_msg_v} [Classification: %{DATA:snort_classification}] [Priority: %{INT:snort_priority}] {%{WORD:protocol}} %{IP:src_ip}:%{INT:src_port} -> %{IP:dst_ip}:%{INT:dst_port}" }
add_field => [ "Priority", "%{snort_priority}" ]
add_field => [ "Classification", "%{snort_classification}" ]
}
}

4

For which field are you seeing this result? What was the input line in that case?

Are you able to explain a bit more about the filebeat?

You're not using Filebeat so that's a distraction.

add_field => [ "Priority", "%{snort_priority}" ]

Why not capture the priority into the Priority field in the first place instead of capturing it into snort_priority and copying that value into Priority?