Issue while Parsing XML data

I have log data which is mixed with xml content in single line.. When i used xml filter to parse the data , i am not seeing any events added.

My input log:
INFO | jvm 1 | 2019/08/27 12:47:51.895 |<tns:FileTransferOperationRequest xmlns:tns="http://www.talend.org/service/"><filename>pmresult_1526726662_15_201908271215_201908271230_201908271247.csv.gz</filename><fromUrl>/talendprojects/fileServices/workingFolder</fromUrl><toUrl>/mnt/gis/Input/NEMs/AvailabilityHuawei</toUrl><deleteOriginal>False</deleteOriginal><protocol>SFTP</protocol><ftpDetails><sftp>True</sftp><serverAddress>81.17.75.83</serverAddress><port>22</port><username>eeprdsftp</username><password>test</password></ftpDetails><removePrefix>false</removePrefix></tns:FileTransferOperationRequest>

In the above log need to grep the datetime, filename, fromurl & tourl details.

my config file:
input
{
stdin{}
}

filter
{

    if "<removePrefix>false</removePrefix>" not  in  [message]
    {
        drop {}

    }
    xml
    {
            source => "message"
            store_xml => "true"
            remove_namespaces => "true"
            xpath => [ "/FileTransferOperationRequest/fromUrl/text()", "fromurl",
                       "/FileTransferOperationRequest/toUrl/text()", "tourl",
                       "/FileTransferOperationRequest/filename/text()", "filename"]
    }

}

Please edit your post, select the log message and click on </> in the toolbar above the edit pane. Make sure that the appearence of the log message changes to be

like this

Thank you.. Can you please help on this

If you are using store_xml => true then the xml filter does not care about random text prefixing the XML. However, the xpath option most certainly does. Try

    mutate { gsub => [ "message", "^[^<]+<", "<" ] }
    xml {
        source => "message"
        store_xml => false
        force_array => false
        remove_namespaces => true
        xpath => {
            "/FileTransferOperationRequest/fromUrl/text()" => "fromurl"
            "/FileTransferOperationRequest/toUrl/text()" => "tourl"
            "/FileTransferOperationRequest/filename/text()" => "filename"
        }
    }

which gets me

   "fromurl" => "/talendprojects/fileServices/workingFolder",
     "tourl" => "/mnt/gis/Input/NEMs/AvailabilityHuawei",
  "filename" => "pmresult_1526726662_15_201908271215_201908271230_201908271247.csv.gz",

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.