Split array with multiple field

Elastic Stack Logstash
1

Hi there,

I have tried by my own, but I'm unable to figure out what I'm doing wrong here. I get a reply from a SQL database as an enrichment of my data. The SQL reply is an array and looks like:

{
"servicename": "Skype-Communication",
"servicedescription": "for Communication",
"application": "Skype"
},
{
"servicename": "SAP-Warehouse",
"servicedescription": "for Warehouse",
"application": "SAP"
}

When I try to split these return by using

split {
field => "[dst_port_info]"
target => "dst_port_info"
}

The split seems only take the first data element and is not adding the information from the second.

Does anyone please have an advise for me what I'm doing wrong here?

Thanks
Stefan

2

The error message indicates that there is no sql_return field. Please show an example event produced by Logstash, preferably the output from a stdout { codec => rubydebug } output plugin.

1 Like
3

Hi, thanks for your reply. I have messed up the error message. This was not related to this error and I have updated my previous post accordingly. I have also attached the requested ruby debug output.

Thanks again,
Stefan

{
"src_network_info" => {
"subnet" => "10.1.1.0",
"descrip" => "src network"
},
"ciscotag" => "ASA-6-302013",
"host" => "10.3.3.3",
"dst_network_info" => {
"subnet" => "10.2.2.0",
"descrip" => "dest network"
},
"protocol" => "TCP",
"dst_port" => "10000",
"src_hostname" => "srchost.ms.com",
"dst_port_info" => [
[0] {
"servicedescription" => "for Communication",
"application" => "Skype",
"servicename" => "Skype-Communication"
},
[1] {
"servicedescription" => "for Warehouse",
"application" => "SAP",
"servicename" => "SAP-Warehouse"
}
],
"connection_id" => "951455647",
"dst_ip" => "10.2.2.2",
"dst_interface" => "wan-inside",
"src_ip" => "10.1.1.1",
"src_port" => "80",
"@version" => "1",
"@timestamp" => 2018-03-09T14:14:21.358Z,
"direction" => "outbound",
"action" => "Built",
"syslog_pri" => "190",
"dst_hostname" => "desthost.ms.com",
"timestamp" => "Mar 09 2018 14:14:16",
"src_interface" => "DMZ",
"tags" => [
[0] "cisco",
[1] "asa",
[2] "dst_mapped_port_info_enriched",
[3] "debug"
]
}
4

I'm confused. Is this a single event? Because it has two dist_port_info fields which is impossible.

1 Like
5

Sorry for being not clear, yes it is one event enriched by the jdbc_streaming filter that gets back two rows from the SQL server. I messed copying the output to this post. The field is just one time there. I try to split the SQL reply (field: dst_port_info) which contains one or more elements to the same document.

6

Okay. And what events do you get when applying the split filter?

1 Like
7

I think split is creating two documents for each element in the array, but I'm aiming to have the elements and values joined together in the same document.

Thanks again.

8

Then I don't understand what you're after. Please copy the current event contents from above and modify it to show what you want to get.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.