Logstash- extraction of data from a column in csv

Elastic Stack Logstash
1

Hi ,

I have a csv file with where the is a column called "description" which has got random lines and varies with each record in csv . For one of the record it looks like this :

JOB RUN: 'AAL ATLAS ETL ATRMS Recipe' was run on 9/28/2016 at 6:05:00 AM

DURATION: 0 hours, 48 minutes, 21 seconds

STATUS: Failed

MESSAGES: The job failed. The Job was invoked by Schedule 97 (JobSchedule). The last step to run was step 7 (AtlasEtlAtrmsRecipeParameterInstance).

Now how do I parse it using logstash? I have tried using csv filter but since there are comma separated values inside the fields also (like description field) I not able to parse it properly. Please suggest how can I filter my data and insert. Also if there is certain value inside a field , eg : in "description" I want to retrieve the value for "STATUS" then how do I do the extraction?

2

This is a job for either the grok filter or possibly the dissect filter.

3

Thanks magnus. could you please give me an example? For the record highlighted above ?

4

I don't have time for that, but http://grokconstructor.appspot.com/ can assist you in crafting a grok expression.

5

This is great. Thanks!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.