Inconsistent results with http_poller & XML parsing

Elastic Stack Logstash
1

Looking for some advice here on an issue i'm facing with http_poller & xml parsing.

I'm polling 2 urls every 60 secs that return XML & getting some very inconsistent results.
Sometimes i'm getting perfect output (3 distinct records per URL per min with the xml parsed perfectly)
Other times 1 or 2 fail parsing as the message is not as expected, other time it would be 4-5 failures each min, i dont understand why its working some of the time but not others

pollerfailures
pollerfailures1868×858 152 KB

Given that the multiline & xml pasting config works 239 times out of 360 ( 6 records per min for an hour) it doenst feel like its an issue with the way multiline & the xml parsing is configured

I've provided an example of the XML source, i'm having to use multiline as the source always contains 3 nested xml responses that i need to parse out as 3 distinct records in elastic

Is there a better way i could be doing this that would give more stable results ?

Logstash Pipeline:

input {
    http_poller {

        urls => {
         
          G04_ISIP_01_TRUNK => "http://xxxxxxx:8088/serverx?sipTrunkStatistics"
          G04_ISIP_02_TRUNK => "http://xxxxxxx:8088/serverx?sipTrunkStatistics"
        
        }
    
    
        request_timeout => 90
        schedule => { cron => "*/1 * * * * UTC"}
        metadata_target => "http_poller_metadata"
        codec => multiline {
            pattern => "^<sipTrunkData"
            negate => "true"
            what => "previous"
            auto_flush_interval => 10
        }
    }
}
filter
{
     
    if [message] !~ /^\<sipTrunkData id='sipTrunkData'>/ {
        drop { } 
    }


    mutate{
        gsub =>["message","</sipTrunkStatistics>",""]
        gsub =>["message","<sipTrunkStatistics id='sipCapacityGroupTable'>",""]
        gsub =>["message","\n\n",""]
    
    }
    
    xml {
        remove_namespaces => "false"
        source => "message"
        target => "doc"
        force_array => true
        force_content => true
        
        
    }
    
    date {
            match => [ "logdate", "MMM dd yyyy HH:mm:ss" ]
    }


  
    mutate {
        remove_field => ["message2"]
    }
}
output {
 elasticsearch {
      }
}

Xml page source

<?xml version='1.0'?><sipTrunkStatistics id='sipTrunkStatistics'>
<sipTrunkStatistics id='sipTrunkTable'>
<sipTrunkData id='sipTrunkData'>
<TRUNK id='007FE126-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Trunk">iSIP_SBC</TRUNK>
<CURRENT_CALLS id='007FE144-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALL_RATE id='007FE158-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<PEAK_CALLS id='007FE162-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALL_RATE id='007FE16C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<CALL_ATTEMPTS id='007FE16D-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<RELEASED_CALLS id='007FE176-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<SUMMARY_START id='007FE180-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary Start">1668775095765</SUMMARY_START>
<SUMMARY_END id='007FE18A-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary End">1668778695765</SUMMARY_END>
<CAPACITY id='007FE194-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY_GROUP id='007FE19E-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Capacity Group">iSIP_SBC</CAPACITY_GROUP>
<IN_SERVICE id='007FE1A8-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
</sipTrunkData>
<sipTrunkData id='sipTrunkData'>
<TRUNK id='008002F0-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Trunk">PCI_MAN</TRUNK>
<CURRENT_CALLS id='00800304-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALL_RATE id='00800318-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<PEAK_CALLS id='00800322-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALL_RATE id='00800323-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<CALL_ATTEMPTS id='0080032C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<RELEASED_CALLS id='00800336-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<SUMMARY_START id='00800340-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary Start">1668775095765</SUMMARY_START>
<SUMMARY_END id='0080034A-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary End">1668778695765</SUMMARY_END>
<CAPACITY id='00800354-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY_GROUP id='0080035E-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Capacity Group">PCI_MAN</CAPACITY_GROUP>
<IN_SERVICE id='00800368-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
</sipTrunkData>
<sipTrunkData id='sipTrunkData'>
<TRUNK id='00800660-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Trunk">PCI_NEW</TRUNK>
<CURRENT_CALLS id='00800674-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALL_RATE id='0080067E-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<PEAK_CALLS id='00800688-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALL_RATE id='00800692-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<CALL_ATTEMPTS id='0080069C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<RELEASED_CALLS id='008006A6-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<SUMMARY_START id='008006B0-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary Start">1668775095765</SUMMARY_START>
<SUMMARY_END id='008006B1-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary End">1668778695765</SUMMARY_END>
<CAPACITY id='008006BA-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY_GROUP id='008006C4-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Capacity Group">PCI_NEW</CAPACITY_GROUP>
<IN_SERVICE id='008006CE-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
</sipTrunkData>
</sipTrunkStatistics>
<sipTrunkStatistics id='sipCapacityGroupTable'>
</sipTrunkStatistics>
</sipTrunkStatistics>
2

One the unexpected behaviours is its merging 2 sections together in the message

input from webpage

<sipTrunkData id='sipTrunkData'>
<TRUNK id='007FE126-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Trunk">iSIP_SBC</TRUNK>
<CURRENT_CALLS id='007FE144-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALL_RATE id='007FE158-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<PEAK_CALLS id='007FE162-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALL_RATE id='007FE16C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<CALL_ATTEMPTS id='007FE16D-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<RELEASED_CALLS id='007FE176-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<SUMMARY_START id='007FE180-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary Start">1668775095765</SUMMARY_START>
<SUMMARY_END id='007FE18A-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary End">1668778695765</SUMMARY_END>
<CAPACITY id='007FE194-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY_GROUP id='007FE19E-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Capacity Group">iSIP_SBC</CAPACITY_GROUP>
<IN_SERVICE id='007FE1A8-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
</sipTrunkData>
<sipTrunkData id='sipTrunkData'>
<TRUNK id='008002F0-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Trunk">PCI_MAN</TRUNK>
<CURRENT_CALLS id='00800304-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALL_RATE id='00800318-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<PEAK_CALLS id='00800322-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALL_RATE id='00800323-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<CALL_ATTEMPTS id='0080032C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<RELEASED_CALLS id='00800336-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<SUMMARY_START id='00800340-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary Start">1668775095765</SUMMARY_START>
<SUMMARY_END id='0080034A-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary End">1668778695765</SUMMARY_END>
<CAPACITY id='00800354-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY_GROUP id='0080035E-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Capacity Group">PCI_MAN</CAPACITY_GROUP>
<IN_SERVICE id='00800368-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
</sipTrunkData>

Output to elastic

<sipTrunkData id='sipTrunkData'>
<sipTrunkData id='sipTrunkData'>
<TRUNK id='007FE126-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Trunk">iSIP_SBC</TRUNK>
<TRUNK id='00300D40-8B1D-129D-9691-CD324A0AAA77' type='7' rem="Trunk">iSIP_SBC</TRUNK>
<CURRENT_CALLS id='007FE144-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALLS id='00300D7C-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Calls">0</CURRENT_CALLS>
<CURRENT_CALL_RATE id='007FE158-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<CURRENT_CALL_RATE id='00300D90-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Call Rate">0</CURRENT_CALL_RATE>
<PEAK_CALLS id='007FE162-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALLS id='00300D9A-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Peak Calls">0</PEAK_CALLS>
<PEAK_CALL_RATE id='007FE16C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<PEAK_CALL_RATE id='00300DA4-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Peak Call Rate">0</PEAK_CALL_RATE>
<CALL_ATTEMPTS id='007FE16D-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<CALL_ATTEMPTS id='00300DAE-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Call Attempts">0</CALL_ATTEMPTS>
<RELEASED_CALLS id='007FE176-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<RELEASED_CALLS id='00300DB8-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Released Calls">0</RELEASED_CALLS>
<SUMMARY_START id='007FE180-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary Start">1668785897139</SUMMARY_START>
<SUMMARY_START id='00300DC2-8B1D-129D-9691-CD324A0AAA77' type='5' rem="Summary Start">1668786116672</SUMMARY_START>
<SUMMARY_END id='007FE18A-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem="Summary End">1668789497139</SUMMARY_END>
<SUMMARY_END id='00300DCC-8B1D-129D-9691-CD324A0AAA77' type='5' rem="Summary End">1668789716672</SUMMARY_END>
<CAPACITY id='007FE194-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY id='00300DEA-8B1D-129D-9691-CD324A0AAA77' type='4' rem="Capacity">0</CAPACITY>
<CAPACITY_GROUP id='007FE19E-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="Capacity Group">iSIP_SBC</CAPACITY_GROUP>
<CAPACITY_GROUP id='00300DF4-8B1D-129D-9691-CD324A0AAA77' type='7' rem="Capacity Group">iSIP_SBC</CAPACITY_GROUP>
<IN_SERVICE id='007FE1A8-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
<IN_SERVICE id='00300DFE-8B1D-129D-9691-CD324A0AAA77' type='7' rem="In Service">Yes</IN_SERVICE>
</sipTrunkData>
</sipTrunkData>

JSON

"message": "<sipTrunkData id='sipTrunkData'>\n<sipTrunkData id='sipTrunkData'>\n<TRUNK id='007FE126-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem=\"Trunk\">iSIP_SBC</TRUNK>\n<TRUNK id='00300D40-8B1D-129D-9691-CD324A0AAA77' type='7' rem=\"Trunk\">iSIP_SBC</TRUNK>\n<CURRENT_CALLS id='007FE144-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Calls\">0</CURRENT_CALLS>\n<CURRENT_CALLS id='00300D7C-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Calls\">0</CURRENT_CALLS>\n<CURRENT_CALL_RATE id='007FE158-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Call Rate\">0</CURRENT_CALL_RATE>\n<CURRENT_CALL_RATE id='00300D90-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Call Rate\">0</CURRENT_CALL_RATE>\n<PEAK_CALLS id='007FE162-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Peak Calls\">0</PEAK_CALLS>\n<PEAK_CALLS id='00300D9A-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Peak Calls\">0</PEAK_CALLS>\n<PEAK_CALL_RATE id='007FE16C-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Peak Call Rate\">0</PEAK_CALL_RATE>\n<PEAK_CALL_RATE id='00300DA4-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Peak Call Rate\">0</PEAK_CALL_RATE>\n<CALL_ATTEMPTS id='007FE16D-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Call Attempts\">0</CALL_ATTEMPTS>\n<CALL_ATTEMPTS id='00300DAE-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Call Attempts\">0</CALL_ATTEMPTS>\n<RELEASED_CALLS id='007FE176-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Released Calls\">0</RELEASED_CALLS>\n<RELEASED_CALLS id='00300DB8-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Released Calls\">0</RELEASED_CALLS>\n<SUMMARY_START id='007FE180-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem=\"Summary Start\">1668785897139</SUMMARY_START>\n<SUMMARY_START id='00300DC2-8B1D-129D-9691-CD324A0AAA77' type='5' rem=\"Summary Start\">1668786116672</SUMMARY_START>\n<SUMMARY_END id='007FE18A-8B2E-129D-9F7D-D2324A0AAA77' type='5' rem=\"Summary End\">1668789497139</SUMMARY_END>\n<SUMMARY_END id='00300DCC-8B1D-129D-9691-CD324A0AAA77' type='5' rem=\"Summary End\">1668789716672</SUMMARY_END>\n<CAPACITY id='007FE194-8B2E-129D-9F7D-D2324A0AAA77' type='4' rem=\"Capacity\">0</CAPACITY>\n<CAPACITY id='00300DEA-8B1D-129D-9691-CD324A0AAA77' type='4' rem=\"Capacity\">0</CAPACITY>\n<CAPACITY_GROUP id='007FE19E-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem=\"Capacity Group\">iSIP_SBC</CAPACITY_GROUP>\n<CAPACITY_GROUP id='00300DF4-8B1D-129D-9691-CD324A0AAA77' type='7' rem=\"Capacity Group\">iSIP_SBC</CAPACITY_GROUP>\n<IN_SERVICE id='007FE1A8-8B2E-129D-9F7D-D2324A0AAA77' type='7' rem=\"In Service\">Yes</IN_SERVICE>\n<IN_SERVICE id='00300DFE-8B1D-129D-9691-CD324A0AAA77' type='7' rem=\"In Service\">Yes</IN_SERVICE>\n</sipTrunkData>\n</sipTrunkData>"

But its merging them with data from the other of the 2 URLS's

Does the HTTP poller process each URL at a time or does it combine the 2 and process as 1 long input ?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.