Enrich message

Elastic Stack Logstash
1

I have a java application that is writing out message as follows for example:
00:00:13,950 INFO [com.companyurl.application] (bq-thread-pool-threads - 881) ABC response by ApplicationManager - received <?xml version="1.0" encoding="UTF-8"?> message 1 message unstructure message3 message4 message5

I need to extract the tags and post the whole message as is , for example

Tag1 : message
Tag2 : message
Tag3 : message3
msg : Whole message as-is

I would grateful if someone can point me to an example of doing this.

Many thanks in advance...

2

Here is a practical introduction to working with Logstash. If you want more detailed guidance you need to show real examples of what your logs look like and what the expected output is.

3

Many thanks Christian

I have actually been using your blog to learn more about logtash - it is VERY helpful. Many thanks.

Here is a sample of log that I am capturing using Filebeat and planning to send to logtash.

00:00:13,950 INFO [com.companyurl.application] (bq-thread-pool-threads - 881) ABC response by ApplicationManager - received <?xml version="1.0" encoding="UTF-8"?>
<sr:Response schemaVersion="3.0" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:ns3="http://www.companyurl/ResponseAndAlert" xmlns:ns4="http://www.companyurl/S1SP" xmlns:ns5="http://www.companyurl/sm" xmlns:sr="http://www.companyurl/ServiceUserGateway">00:00:13,949 INFO [com.companyurl] (bq-thread-pool-threads - 881) COMP response by ServiceResponseController - received <?xml version="1.0" encoding="UTF-8"?>
sr:Header
sr:ResponseID11-22-33-44</sr:ResponseID>
sr:ResponseCodeE62</sr:ResponseCode>
sr:ResponseDateTime2019-08-31T09:57:04.98Z</sr:ResponseDateTime>
</sr:Header>
sr:Body
sr:COMPAlertMessage
sr:COMPAlertCodeN55</sr:COMPAlertCode>
sr:COMPAlert
sr:S1SPAlertDSP
<S1SPAlert schemaVersion="3.0" xmlns="http://www.companyurl/ServiceUserGateway11-22-33-44S1CE20042019-08-31T09:57:03<CanonicalizationMethod
sr:ScheduleID38060</sr:ScheduleID>
</sr:S1SPAlert>
</sr:COMPAlert>
</sr:COMPAlertMessage>
</sr:Body>
</sr:Response>

I would like to extract for example ScheduleID value and post to ES the whole message from filebeast as-is plus the ScheduleID as a key (index??) and 38060 as the value of that key.

I look forward to hearing from you soon.

Many thanks for you help in advance...

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.