Logstash Error: agent failed to execute action

Hello Dears,

XML sample:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
   <soap:Header>
      <Header xmlns="https://kjkjj.com/kkkk" xmlns:ns2="https://sih.sadad.com/common">
         <ns2:RequestID>1a7f81ac-355f-11ec-bc47-3b3678cb106a</ns2:RequestID>
      </Header>
   </soap:Header>
   <soap:Body>
      <PaymentValidationResponse xmlns="https://######.com" xmlns:ns2="https://s/###/##">
         <StatusCode>0</StatusCode>
         <StatusDescription>Success</StatusDescription>
         <PaymentValidations>
            <PaymentValidation>
               <InvoiceCode>175810975800</InvoiceCode>
               <SupplierID>649</SupplierID>
               <SupplierName>MED</SupplierName>
               <Amount>2552.23</Amount>
               <StatusCode>0</StatusCode>
            </PaymentValidation>
         </PaymentValidations>
      </PaymentValidationResponse>
   </soap:Body>
</soap:Envelope>

Filter plugin conf:

filter {
     
     multiline {
         pattern =>  "<soap:Body>"
         what => "previous"
         negate => "true"
     }
     xml {
        source => "message"
        store_xml => false
        remove_namespaces => true
     
        xpath => [
            "/soap:Envelope/soap:Header/Header/RequestID/text()", "RequestID",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/StatusDescription/text()", "StatusDescription",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/InvoiceCode/text()", "InvoiceCode",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierID/text()", "SupplierID",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierName/text()", "SupplierName",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/Amount/text()", "Amount",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/StatusCode/text()", "StatusCode",
            "/soap:Envelope/soap:Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/ErrorMessage/text()", "ErrorMessage"
        ]
     }
     
     mutate {
              add_field => ["StatusDescription", "%{StatusDescription}"]
              #add_field => ["NameIndexed", "%{Name}"]
            }
    
    
}

I need to solve this issue.

Thanks.

That is going to result in two events, one containing the header, one containing the body. The body does not have the Envelope.

You need to tell the filter what the namespaces are, even if you want it to remove them. Try

    xml {
        source => "message"
        store_xml => false
        remove_namespaces => true

        namespaces => {
            "soap" => "http://schemas.xmlsoap.org/soap/envelope/"
            "ns1" => "https://kjkjj.com/kkkk"
            "ns2" => "https://sih.sadad.com/common"
        }
        xpath => [
            "/Envelope/Header/Header/RequestID/text()", "RequestID",
            "/Body/PaymentValidationResponse/StatusDescription/text()", "StatusDescription",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/InvoiceCode/text()", "InvoiceCode",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierID/text()", "SupplierID",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierName/text()", "SupplierName",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/Amount/text()", "Amount",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/StatusCode/text()", "StatusCode",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/ErrorMessage/text()", "ErrorMessage"
        ]
     }

Note that with xpath everything is an array.

@Badger thanks for your response, keep multiline block or delete it from filter?

You definitely need a multiline codec.

After updating, I still have the same error!!

filter {
     
     multiline {
         pattern =>  "<soap:Body>"
          what => "previous"
          negate => "true"
     }
     namespaces => {
            "soap" => "http://schemas.xmlsoap.org/soap/envelope/"
            "ns1" => "https://sih.sadad.com/PaymentValidation/defs"
            "ns2" => "https://sih.sadad.com/common"
        }
     xml {
        source => "message"
        store_xml => false
        remove_namespaces => true
     
        xpath => [
            "/Envelope/Header/Header/RequestID/text()", "RequestID",
            "/Body/PaymentValidationResponse/StatusDescription/text()", "StatusDescription",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/InvoiceCode/text()", "InvoiceCode",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierID/text()", "SupplierID",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierName/text()", "SupplierName",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/Amount/text()", "Amount",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/StatusCode/text()", "StatusCode",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/ErrorMessage/text()", "ErrorMessage"
        ]
     }
     
     mutate {
              add_field => ["StatusDescription", "%{StatusDescription}"]
              #add_field => ["NameIndexed", "%{Name}"]
            }
    
}

Error in "namespaces" lines!!

Expected one of [ \t\r\n], \”#\”, \”=>\” at line 10, column 17

This is a configuration error, what do you have in line 10 in your pipeline configuration? Share your full pipeline, not just the filter parts.

Also, there is no multiline filter, multiline should be done in the input using codec, this is probably the cause of your error.

Thanks @leandrojmp for your response, this is my full pipeline:

input {
    file {
       path => "/etc/logstash/logsample.txt"
       start_position = > "beginning"
    }
}


filter {
     
     multiline {
         pattern =>  "<soap:Body>"
          what => "previous"
          negate => "true"
     }
     namespaces => {
            "soap" => "http://schemas.xmlsoap.org/soap/envelope/"
            "ns1" => "https://sih.sadad.com/PaymentValidation/defs"
            "ns2" => "https://sih.sadad.com/common"
        }
     xml {
        source => "message"
        store_xml => false
        remove_namespaces => true
     
        xpath => [
            "/Envelope/Header/Header/RequestID/text()", "RequestID",
            "/Body/PaymentValidationResponse/StatusDescription/text()", "StatusDescription",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/InvoiceCode/text()", "InvoiceCode",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierID/text()", "SupplierID",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/SupplierName/text()", "SupplierName",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/Amount/text()", "Amount",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/StatusCode/text()", "StatusCode",
            "/Body/PaymentValidationResponse/PaymentValidations/PaymentValidation/ErrorMessage/text()", "ErrorMessage"
        ]
     }
     
     mutate {
              add_field => ["StatusDescription", "%{StatusDescription}"]
              #add_field => ["NameIndexed", "%{Name}"]
            }
    
}

output {
        stdout { codec => rubydebug }       
}

@leandrojmp any update!

@Badger

The multiline filter has been deprecated for years now, use a multiline codec.

The namespaces option should be inside the xml filter.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.