Hi,
I am using ELK stack+filebeats in our project and the application log is integration with filebeats. Filebeat is reading the application log. The log is successfully read by logstash. My question is how to parse the XML inside the log file. The log is inserted in following format below. Please help me to parse the XML from the log and read the elements from the XML using logstash.
Blockquote
2021-08-23 18:48:22.711 INFO [bwEngThread:In-Memory Process Worker-1] com.tibco.bw.palette.generalactivities.Log.Framework.sharedmodule.Log - MSGCODE: <?xml version="1.0" encoding="UTF-8"?>
<tns:Logger_Request xmlns:tns="http://www.ericsson.com/tibco/schema/Logger" xmlns:tib="http://www.tibco.com/bw/xslt/custom-functions">
tns:conversationIdCONVbw0a101j9t</tns:conversationId>
tns:correlationIdCORIDbw0a101j9t</tns:correlationId>
tns:logTimestamp2021-08-23T18:48:22.703+05:30</tns:logTimestamp>
tns:typeSTART</tns:type>
tns:businessReferenceIdABC</tns:businessReferenceId>
tns:systemConsumerABC</tns:systemConsumer>
tns:serviceNameABC</tns:serviceName>
tns:operationNameABC</tns:operationName>
tns:payload<?xml version="1.0" encoding="UTF-8"?>ns1:customerIdMTX12345672</ns1:customerId>ns2:collectionAgentEmailcollectionAgentEmail@string.com</ns2:collectionAgentEmail>ns2:collectionLeaderEmailcollectionLeaderEmail@string.com</ns2:collectionLeaderEmail>ns2:maidenNameOfMothermaidenNameOfMother</ns2:maidenNameOfMother>ns2:businessLinebusinessLine</ns2:businessLine>ns2:lineOfBusinessBanking</ns2:lineOfBusiness>ns2:subBusinessLineAgency and Bureau</ns2:subBusinessLine>ns2:hideDetailPriceFlagtrue</ns2:hideDetailPriceFlag>ns2:showContractNumbertrue</ns2:showContractNumber>ns2:contractNumberInformationcontractNumberInformation</ns2:contractNumberInformation>ns2:showContractTitletrue</ns2:showContractTitle>ns2:contractTitleInformationcontractTitleInformation</ns2:contractTitleInformation>ns2:showEmailFlagtrue</ns2:showEmailFlag>ns2:showPO1Flagtrue</ns2:showPO1Flag>ns2:po1Informationpo1Information</ns2:po1Information>ns2:showPO2Flagtrue</ns2:showPO2Flag>ns2:po2Informationpo2Information</ns2:po2Information>ns2:holdBillFlagtrue</ns2:holdBillFlag>ns2:contactTypeIndosat Reference</ns2:contactType>ns2:invoicingCompanyinvoicingCompany</ns2:invoicingCompany>ns2:accountClassSpecial Account</ns2:accountClass>ns2:IndividualNamens1:accountNameFeryanto</ns1:accountName>ns1:formattedNameDoddy Feryanto</ns1:formattedName>ns1:contactNameDoddy</ns1:contactName>ns1:deliveryNamedeliveryName</ns1:deliveryName></ns2:IndividualName>ns2:dateofBirth2021-06-10</ns2:dateofBirth>ns2:genderM</ns2:gender>ns2:maritalStatusMAR</ns2:maritalStatus>ns2:jobDescriptionjobDescription</ns2:jobDescription>ns2:nationalityIDN</ns2:nationality>ns2:hobbyhobby</ns2:hobby>ns2:religionHindu</ns2:religion>ns2:educationeducation</ns2:education>ns2:employerNameemployerName</ns2:employerName>ns3:IDValue098766543</ns3:IDValue>ns3:IDValue112233445</ns3:IDValue>ns3:IDValue123456</ns3:IDValue>ns3:IDTypecode7</ns3:IDTypecode>ns3:IDValueIDValue</ns3:IDValue>29/05/2021ns4:seqNumber0</ns4:seqNumber>ns4:addressRoleCodeS</ns4:addressRoleCode>ns4:addressLine1Street No 2</ns4:addressLine1>ns4:addressLine2Menara Sel.,22,Jl.H.R</ns4:addressLine2>ns4:addressLine3address line 3</ns4:addressLine3>ns4:contactAddresscontactAddress</ns4:contactAddress>ns4:cityJakarta</ns4:city>ns4:stateMakassar</ns4:state>ns4:zip12940</ns4:zip>ns4:seqNumber5</ns4:seqNumber>ns4:addressRoleCodeB</ns4:addressRoleCode>ns4:addressLine1Street No 2</ns4:addressLine1>ns4:addressLine2Menara Sel.,22,Jl.H.R</ns4:addressLine2>ns4:addressLine3address line 3</ns4:addressLine3>ns4:contactAddresscontactAddress</ns4:contactAddress>ns4:cityJakarta</ns4:city>ns4:stateMakassar</ns4:state>ns4:zip12940</ns4:zip>ns4:areaCode+62</ns4:areaCode>ns4:number9118256710</ns4:number>ns4:number9118256711</ns4:number>ns4:number9118256711</ns4:number>ns4:number9118256712</ns4:number>ns4:number+6289077562411</ns4:number>ns4:eMailAddresseMailAddress@string.com</ns4:eMailAddress>ns4:eMailAddress1eMailAddress1@string.com</ns4:eMailAddress1>ns4:eMailAddress2eMailAddress2@string.com</ns4:eMailAddress2></tns:payload>
tns:Log-LevelAUDIT</tns:Log-Level>
tns:appSpaceABC</tns:appSpace>
tns:appNodeABC</tns:appNode>
tns:engineABC</tns:engine>
tns:appModuleABC</tns:appModule>
</tns:Logger_Request>. JobId [bw0a101j9u], ProcessInstanceId [bw0a101j9u], Activity [Log], Process [framework.sharedmodule.LogProcess], Module [Framework.sharedmodule:1.0.0.20210823170120], Application [LogPrj:1.0].
2021-08-23 18:48:22.803 INFO [Thread-43] com.tibco.thor.frwk.Application - TIBCO-THOR-FRWK-300006: Started BW Application [LogPrj:1.0]
Blockquote
After parsing the XML i want to display the record in kibana like.
