Logstash kv filter plugin query

[Neeraj_Jain]

Hi,
I am trying to use logstash to read a file containing data as below. I want to push this data as key/value to elastic search.

pid: 2601 start: 0 stacksize: 0 breaksize: 25 command: app1 arguments: APP01
pid: 2393 start: 0 stacksize: 0 breaksize: 21 command: app2 arguments: APP02
pid: 2500 start: 0 stacksize: 0 breaksize: 27 command: app3 arguments: APP03

So i use kv filter plugin. Below is the filter.
filter {
kv {
value_split => ":"
}
}

But i see that due to new-lines, the key/value pair is not formed correctly. Below is the output.

{
"@timestamp": "2019-04-22T19:36:52.349Z",
"@version": "1",
"arguments": [
"APP01\npid:",
"APP02\npid:",
"APP03\n"
],
"breaksize": [
"25",
"21",
"27"
],
"command": [
"app1",
"app2",
"app3"
],
"host": "tb929cscf01",
"message": "pid: 2601 start: 0 stacksize: 0 breaksize: 25 command: scscf arguments: IMS_SCSCF0E\npid: 2393 start: 0 stacksize: 0 breaksize: 21 command: scscf arguments: IMS_SCSCF03\npid: 2500 start: 0 stacksize: 0 breaksize: 27 command: scscf arguments: IMS_SCSCF05\n",
"pid": "2601",
"stacksize": [
"0",
"0",
"0"
],
"start": [
"0",
"0",
"0"
],
"type": "proc_heap"
}

Why is '\n' followed by the first word in next line appended to value. please advice. Thanks

[Badger]

What does your input look like?

[Neeraj_Jain]

Hi Badger,
Your question gave me a hint. my input was exec plugin with command as cat of the file. resulting in such an output. I changed to below, and now it works fine.
input{
file
{
path => "/etc/logstash/temp_data"
start_position => "beginning"
}
}

However, my original problem statement was to run command every interval and capture the output of the command

input{
exec{
command => "my_command"
interval => 10
type => 'proc_heap'
}

output of my command looks like the data in my previous question (as below). So i started debugging by adding into file and checking. So please help me how can i achieve the same output with exec command

pid: 2601 start: 0 stacksize: 0 breaksize: 25 command: app1 arguments: APP01
pid: 2393 start: 0 stacksize: 0 breaksize: 21 command: app2 arguments: APP02
pid: 2500 start: 0 stacksize: 0 breaksize: 27 command: app3 arguments: APP03

[Neeraj_Jain]

Hi,
With below logstash conf, i am now able to get the job done, except that the drop filter isn't working (drop lines which do not contain "APP" string). Can you please help me with this

output of my command:
pid: 32199 start: 0 stacksize: 0 breaksize: 16 command: app1 arguments: APP01
pid: 32258 start: 0 stacksize: 0 breaksize: 0 command: app2 arguments: APP02
pid: 32697 start: 0 stacksize: 0 breaksize: 86 command: java arguments: java
pid: 32699 start: 0 stacksize: 0 breaksize: 140 command: java arguments: java

input{
exec{
command => "/home/app/my_script"
interval => 10
type => 'proc_heap'
}
}
filter {
mutate {
gsub => [
"message", "\n", " "
]
}
if ([message] !~ "APP") {
drop { }
}
kv {
value_split => ":"
}
}
output {
stdout {
codec => json
}
}

[Badger]

If your command outputs multiple lines then the exec input will join them together into a single message. A split filter can be used to separate each line into its own event...

split {}

[Neeraj_Jain]

Hi Badger,
i see some difference when input reads from file and when we use exec command. with file input, each line is read in message and processed through the filter and so my output looks as expected. See below:

{
"pid" => "1335",
"host" => "tb929cscf01",
"@version" => "1",
"path" => "/etc/logstash/full_data",
"breaksize" => "25",
"@timestamp" => 2019-04-23T17:30:41.134Z,
"start" => "0",
"message" => "pid: 1335 start: 0 stacksize: 0 breaksize: 25 command: scscf arguments: IMS_SCSCF0F",
"arguments" => "IMS_SCSCF0F",
"command" => "scscf",
"stacksize" => "0"
}
{
"pid" => "1180",
"host" => "tb929cscf01",
"@version" => "1",
"path" => "/etc/logstash/full_data",
"breaksize" => "25",
"@timestamp" => 2019-04-23T17:30:41.101Z,
"start" => "0",
"message" => "pid: 1180 start: 0 stacksize: 0 breaksize: 25 command: scscf arguments: IMS_SCSCF06",
"arguments" => "IMS_SCSCF06",
"command" => "scscf",
"stacksize" => "0"
}

But when i exec my command, which gives similar lines, the message has all the lines with '\n' character as one string. And so my output isn't as expected (see below). Is there a way i can have exec command output also be read line by line?

{
"message" => "pid: 32199 start: 0 stacksize: 0 breaksize: 16 command: STMgr arguments: IMS_STM01\npid: 32258 start: 0 stacksize: 0 breaksize: 0 command: nginxConfUpdato arguments: IMS_NGINXCONF_UPDATOR01\n",
"start" => [
[0] "0",
[1] "0"
],
"host" => "tb929cscf01",
"pid" => "32199",
"breaksize" => [
[0] "16",
[1] "0"
],
"@timestamp" => 2019-04-23T17:37:50.448Z,
"arguments" => [
[0] "IMS_STM01\npid:",
[1] "IMS_NGINXCONF_UPDATOR01\n"
],
"@version" => "1",
"command" => [
[0] "STMgr",
[1] "nginxConfUpdato"
],
"type" => "proc_heap",
"stacksize" => [
[0] "0",
[1] "0"
]
}

~Neeraj

[Neeraj_Jain]

Hola!!
just when i posted this question i resolved the issue with simple split in the filter plugin at the start before kv plugin. initially i tried split but passing some arguments in it. but seems to work without any arguments.

filter {
split {
}
kv {
value_split => ":"
}
}

Thank you.

~Neeraj