Applying grok on a field value after csv plugin

Elastic Stack Logstash
1

Hi team, I have a csv file which contains the output from Oracle tables. We are using csv filter to parse the data to appropriate fields. A field in them has a value which is a command getting executed.
Ex. /home/run job.pl --job="job_name"

I need to apply grok pattern on that particular field and extract only that job name and assign it to a different field. Confused if this is possible.

Any help on this.

2

Yes, it's possible. The grok filter parses text. It doesn't know or care about if that text originally came from a CSV file. For further help please show your current configuration and an example input line.

3

Thanks for the response Magnus.

This is my dummy input file
08-JUL-16~;order_id~;appl_id~;job_name_old~;Night Job~;/home/job.pl --job="job_name"'~;

And this is my csv filter
csv {
columns => ["system_date","order_id","appl_id","job_name_old","description","cmd_line",
separator => "~;"
}

Need to write a grok to parse the field 'cmd_line' and assign just the content within double codes ( --job=" ") to another field like job_cmd.

Thanks.

4

Untested but should work:

filter {
  grok {
    match => {
      "cmd_line" => ' --job="(?<job_cmd>[^"]+)"'
    }
  }
}
5

Hi Magnus, Thanks for the response.

I tried this but don't see a new field getting created with the value.

My Config

 csv {
   columns => ["system_date","order_id","appl_id","ControlM_job_name","description","cmd_line","cpu_id","owner","from_time","to_time", "cyclic", "next_run_time", "status", "odate", "rerun_counter", "order_table", "application", "group_name", "job_id", "elapsed_runtime", "cpu_time", "memname", "state", "nodegroup","data_center","avg_start_time","avg_run_time","start_date","start_time","end_date","end_time"]

   separator => "~;"

    add_field => { "[technology]" => "Tech_name" 
                       "[field2]" => "%{memname}"
                       "[field3]" => "%{nodegroup}"
    }

  }

grok {
match => {
  "cmd_line" => ' --job="(?<mem_name_cmd>[^"]+)"'
}

}

Could you kindly check. The value of that field is /home/job.pl --job="job_name"

6

Please show the result of a stdout { codec => rubydebug } output so we can see what's going on.