We're experienceing difficulties pulling events from xml input
The xml input looks like this
<?xml version="1.0" encoding="utf-8"?>
<Tasks xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://domain.com/schemas/tsaks/v1.xsd">
<Task>
<TaskId>ServerTasks-5017</TaskId>
<TaskState>Success</TaskState>
<Created>2015-12-22T08:20:03</Created>
<QueueTime>2015-12-22T08:20:03</QueueTime>
<StartTime>2015-12-22T08:20:06</StartTime>
<CompletedTime>2015-12-22T08:21:11</CompletedTime>
<DurationSeconds>68</DurationSeconds>
</Task>
<Task>
<TaskId>ServerTasks-5018</TaskId>
<TaskState>Success</TaskState>
<Created>2015-12-22T09:20:03</Created>
<QueueTime>2015-12-22T09:20:03</QueueTime>
<StartTime>2015-12-22T09:20:06</StartTime>
<CompletedTime>2015-12-22T09:21:11</CompletedTime>
<DurationSeconds>68</DurationSeconds>
</Task>
<Tasks>
Our multiline codec looks like this
codec =>
multiline {
pattern => "<Task>"
negate => "true"
what => "next"
}
In the filter plugin we want to process the information within a task and output those to elasticsearch afterwards as distinct documents .
First event received
Contains
<?xml version="1.0" encoding="utf-8"?>
<Tasks xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://domain.com/schemas/tsaks/v1.xsd">
and since it is of little value it is dropped in the beginning of the filter.
Then the tasks are received as separate events which workes as expected, until the last Task element which is not received untill we poll the file again. Then the
The last task is received in the filter with the ending tasks element and leading elements from pulling the file again like this:
<Task>
<TaskId>ServerTasks-5018</TaskId>
<TaskState>Success</TaskState>
<Created>2015-12-22T09:20:03</Created>
<QueueTime>2015-12-22T09:20:03</QueueTime>
<StartTime>2015-12-22T09:20:06</StartTime>
<CompletedTime>2015-12-22T09:21:11</CompletedTime>
<DurationSeconds>68</DurationSeconds>
</Task>
<Tasks>
<?xml version="1.0" encoding="utf-8"?>
<Tasks xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://domain.com/schemas/tsaks/v1.xsd">
What is the best practice and approach to solve this?
preferably in the multiline codec