Different field names depending on Record_Type in CSV file

Elastic Stack Logstash
1

Hi, I have a CSV file with different information depending on the second field value of the CSV file Record_Type. Here is an example of the input CSV file:

08-11-2016 21:08:04,2,4475632030300,655101002121838,25697419 08-11-2016 21:08:05,3, 4475632030300,25697419,*111*222#,39,OND 08-11-2016 21:08:05,8, 4475632030300,25697419,39,http://8.8.40.10:9080/request?ms=44763212345690&sessionid=1&type=1&im=123456789&msg=*119 1 <us><ms>************</ms><sessionid>********</sessionid><type>*</type><im>***************</im><msg>********#</msg></us> 08-11-2016 21:08:05,9, 4475632030300,25697419,39,http://8.8.40.10:9080/request?ms=4476321234567890&sessionid=1&type=1&im=123456789&msg=*119 1 Welcome0. Please enter

So the first event returns 2 and has the following fields:
08-11-2016 21:08:04,2,4475632030300,655101002121838,25697419

Date_Time = 08-11-2016 21:08:04 Record_Type = 2 MS = 4475632030300 IM = 655101002121838 Task_ID = 25697419

The Second event returns a 3 and has the following fields:
08-11-2016 21:08:05,3, 4475632030300,25697419,*111*222#,39,OND

Date_Time = 08-11-2016 21:08:05 Record_Type = 3 MS = 4475632030300 Task_ID = 25697419 Code_Nr = *111*222# Men_Nr. = 39 Customer = OND

The Third event returns an 8 and has the following fields:
08-11-2016 21:08:05,8, 4475632030300,25697419,39,http://8.8.40.10:9080/request?ms=44763212345690&sessionid=1&type=1&im=123456789&msg=*119 1 <us><ms>************</ms><sessionid>********</sessionid><type>*</type><im>***************</im><msg>********#</msg></us>

Date_Time = 08-11-2016 21:08:05 Record_Type = 8 MS = 4475632030300 Task_ID = 25697419 Men_Nr. = 39 URL1 = http://8.8.40.10:9080/request?ms=44763212345690&sessionid=1&type=1&im=123456789&msg=*119 1 <us><ms>************</ms><sessionid>********</sessionid><type>*</type><im>***************</im><msg>********#</msg></us>

The Fourth event returns an 9 and has the following fields:
08-11-2016 21:08:05,9, 4475632030300,25697419,39,http://8.8.40.10:9080/request?ms=4476321234567890&sessionid=1&type=1&im=123456789&msg=*119 1 Welcome0. Please enter

Date_Time = 08-11-2016 21:08:05 Record_Type = 9 MS = 4475632030300 Task_ID = 25697419 Men_Nr. = 39 URL2 = http://8.8.40.10:9080/request?ms=4476321234567890&sessionid=1&type=1&im=123456789&msg=*119 1 Welcome0. Please enter

The Record_Type has different number values and according to this value different fields should be assessed under the filter. How can I process a CSV file that contains different Record_Types with different field structures?

2

After doing an initial match, you'd need to use conditionals to check the value and then do renames.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.