Context: I have 4 folders, one contains logstash, the other 3 each contain filebeat, the first acts as if an application launched log, the second with 2 applications that send logs for which filebeat has 2 entries, and the last as if I had 3 applications so the filebeat contains 3 entries. THE PROBLEM APPEARS WHEN MY LOGS ARE CENTRALIZED IN LOGSTASH WHEN THE FATALS OF 2 OR MORE APPS ARE SENT AT THE SAME TIME. They are all sending different logs but I realized that if I send fatal from app1 and another fatal from app3 I get this error:
From the screen, you have issue with the XML parsing - no closing tags.
Have you add tags or fields in order to distinguish sources?
I added this to identify from which app the logs are being sent. I had been told that this was correct.
I also have a general processors that shows the company and the state of development.
And how does Logstash filter part? Does have IFs based on [fields][V_APP_NAME]?
I already corrected the error, the configuration was incomplete, I added the multiline in each input, I'm not sure if it's correct but it works.
like here is my logstash, if I can improve something I would like to know.
input {
beats {
port => 5044
}
}
filter {
mutate{
gsub => [
'message', '><log4j:message', 'xmlns:log4j="http://jakarta.apache.org/log4j/"><log4j:message'
]
}
xml {
source => "message"
target => "doc"
}
if [doc][level] != "" {
mutate{
copy => { "[doc][level]" => "V_EVENT_TYPE" }
}
}
date {
match => [ "[doc][timestamp]", "UNIX", "UNIX_MS" ]
target => "date_create"
}
}
output {
elasticsearch {
hosts => ["tamcis403.ten.tech.net:9200"]
sniffing => true
manage_template => false
index => "filebeat-change13-%{+YYYY.MM.dd}"
action => "index"
document_type => "%{[@metadata][type]}"
}
}
filebeat:
- type: filestream
id: detection-app5
enabled: true
paths:
- C:\Users\60085367\Desktop\ServerTest3\3_Log_Generados2\*
processors:
- add_fields:
fields:
V_APP_NAME: Detection.Log.Tester2
parsers:
- multiline:
type: pattern
pattern: '^<log4j:event.*'
negate: true
match: after
- type: filestream
id: powerapps-app6
enabled: true
paths:
- C:\Users\60085367\Desktop\ServerTest3\3_Log_Generados3\*
processors:
- add_fields:
fields:
V_APP_NAME: PowerApps.Log.Tester3
parsers:
- multiline:
type: pattern
pattern: '^<log4j:event.*'
negate: true
match: after
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3
setup.kibana:
host: "tamca403.ten.techt.net:5601"
output.logstash:
hosts: ["0.0.0.0:5044"]
processors:
- add_host_metadata:
netinfo.enabled: true
- add_fields:
fields:
V_CODE_ENV: PRD
V_CODE_REGION: TAM
fields_under_root: true
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.