Issue while Parsing XML data

Elastic Stack Logstash
1

I have log data which is mixed with xml content in single line.. When i used xml filter to parse the data , i am not seeing any events added.

My input log:
INFO | jvm 1 | 2019/08/27 12:47:51.895 |<tns:FileTransferOperationRequest xmlns:tns="http://www.talend.org/service/"><filename>pmresult_1526726662_15_201908271215_201908271230_201908271247.csv.gz</filename><fromUrl>/talendprojects/fileServices/workingFolder</fromUrl><toUrl>/mnt/gis/Input/NEMs/AvailabilityHuawei</toUrl><deleteOriginal>False</deleteOriginal><protocol>SFTP</protocol><ftpDetails><sftp>True</sftp><serverAddress>81.17.75.83</serverAddress><port>22</port><username>eeprdsftp</username><password>test</password></ftpDetails><removePrefix>false</removePrefix></tns:FileTransferOperationRequest>

In the above log need to grep the datetime, filename, fromurl & tourl details.

my config file:
input
{
stdin{}
}

filter
{

    if "<removePrefix>false</removePrefix>" not  in  [message]
    {
        drop {}

    }
    xml
    {
            source => "message"
            store_xml => "true"
            remove_namespaces => "true"
            xpath => [ "/FileTransferOperationRequest/fromUrl/text()", "fromurl",
                       "/FileTransferOperationRequest/toUrl/text()", "tourl",
                       "/FileTransferOperationRequest/filename/text()", "filename"]
    }

}

2

Please edit your post, select the log message and click on </> in the toolbar above the edit pane. Make sure that the appearence of the log message changes to be

like this
3

Thank you.. Can you please help on this

4

If you are using store_xml => true then the xml filter does not care about random text prefixing the XML. However, the xpath option most certainly does. Try

    mutate { gsub => [ "message", "^[^<]+<", "<" ] }
    xml {
        source => "message"
        store_xml => false
        force_array => false
        remove_namespaces => true
        xpath => {
            "/FileTransferOperationRequest/fromUrl/text()" => "fromurl"
            "/FileTransferOperationRequest/toUrl/text()" => "tourl"
            "/FileTransferOperationRequest/filename/text()" => "filename"
        }
    }

which gets me

   "fromurl" => "/talendprojects/fileServices/workingFolder",
     "tourl" => "/mnt/gis/Input/NEMs/AvailabilityHuawei",
  "filename" => "pmresult_1526726662_15_201908271215_201908271230_201908271247.csv.gz",
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.