I have a log file with 1 root of xml as below. I have created a logstash parser and tested it out in manual mode and it seems to work fine. But while ingesting into ES and get the error.
Input file:
<?xml version="1.0" encoding="utf-8"?>
<trouble_shooter_log xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="TroubleShooterLog.xsd"
version="1.0" >
<event date="2020-07-21" time="03:17:36" line="814" text="Diameter peer connection up"/>
<event date="2020-07-21" time="03:17:39" line="301" text="tcpproxy;"/>
<event date="2020-07-21" time="03:23:53" line="253" text="Http Client with 7 worker threads started"/>
</trouble_shooter_log>
Logstash configuration:
input {
file
{
path => "/root/h.log"
start_position => "beginning"
codec => multiline {
pattern => "^<event "
negate => "true"
what => previous
auto_flush_interval => 1
}
}
}
filter {
if "<event " in [message]{
xml {
namespaces => {
"xsi" => "http://www.w3.org/2001/XMLSchema-instance"
}
store_xml => true
source => "message"
target => "parsed"
}
}
}
output {
stdout{}
}
Error wen trying to ingest into ES:
[ERROR] 2020-09-07 14:38:58.833 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
log file without ingestion shows multiline tag for the first xml line
{
"host" => "elk01.novalocal",
"@timestamp" => 2020-09-07T19:41:42.413Z,
"@version" => "1",
"message" => "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<trouble_shooter_log xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n\txsi:noNamespaceSchemaLocation=\"TroubleShooterLog.xsd\"\n\tversion=\"1.0\" >",
**"tags" => [**
** [0] "multiline"**
],
"path" => "/root/h.log"
}
{
"host" => "elk01.novalocal",
"parsed" => {
"line" => "814",
"time" => "03:17:36",
"date" => "2020-07-21",
"text" => "Diameter peer connection up"
},
"@timestamp" => 2020-09-07T19:41:42.454Z,
"@version" => "1",
"message" => "<event date=\"2020-07-21\" time=\"03:17:36\" line=\"814\" text=\"Diameter peer connection up\"/>",
"path" => "/root/h.log"
}