# CSV Filter help

**URL:** <https://discuss.elastic.co/t/csv-filter-help/55937>\
**Category:** Logstash\
**Created:** [July 19, 2016, 11:34pm UTC](https://discuss.elastic.co/t/csv-filter-help/55937 "2016-07-19T23:34:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bishaka](https://avatars.discourse-cdn.com/v4/letter/b/e480ec/32.png) [@bishaka](https://discuss.elastic.co/u/bishaka)\
**Post date:** [July 19, 2016, 11:34pm UTC](https://discuss.elastic.co/t/csv-filter-help/55937/1 "2016-07-19T23:34:46Z")

</div>

Hi There,

I am using the following CSV filter after failing to use grok filter for a logfile containing multiple patterns:

filter {  
csv {  
columns =\> ["@timestamp", "threadType", "requestType", "requestStatus"]  
separator =\> "|"  
}  
}

The log file contains the following patterns:

2016-06-03 08:45:03 | INFO | [[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] | WAKEUP-REQ | SUCCESS | urn:uuid:34567b55-ee31-3f45-859a-899f43fcb39c - 200 - - {"statusCode":"success","statusMessage":"Wakeup request recieved"}

2016-06-03 08:45:04 | INFO | [[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] | WAKEUP-REQ | RECEIVED | urn:uuid:8251bbf1-6cab-381d-b482-a572834d989c

2016-06-03 08:46:02 | INFO | [jmsContainer-15] | AQ-REQ | PROCESSING | SUCCESS FOR urn:uuid:B9C6DC32-C2D7-4CBE-8268-B5F92CE85B73

However,  
I get csvparsefailure on some of the patterns. Please help!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 20, 2016, 1:23am UTC](https://discuss.elastic.co/t/csv-filter-help/55937/2 "2016-07-20T01:23:45Z")

</div>

The CSV filter can only use a single pattern. You may need to use conditionals when they receive`_csvparsefailure` to try another pattern in a different CSV filter.

I've also raised this FR - [https://github.com/logstash-plugins/logstash-filter-csv/issues/38](https://github.com/logstash-plugins/logstash-filter-csv/issues/38)

---

<div class="post-metadata">

**Author:** ![bishaka](https://avatars.discourse-cdn.com/v4/letter/b/e480ec/32.png) [@bishaka](https://discuss.elastic.co/u/bishaka)\
**Post date:** [July 20, 2016, 3:18am UTC](https://discuss.elastic.co/t/csv-filter-help/55937/3 "2016-07-20T03:18:55Z")

</div>

So, I went back and used the grok filter for the 2 different log patterns I have. The output i am getting is:

 ![](https://us1.discourse-cdn.com/elastic/original/2X/8/84e8209dcb48caae338227469bd377e89cea0c02.PNG)

As you can see...I am getting 2 different values for a single field. One is correct and another one is not. My question is..is there anyway I can get rid of the one i do not need? I tried using the remove\_field option...but it only removes the field but not with a particular value. If anyone could help me with this i would greatly appreciate it.

Also, Is there a better approach to this multiple pattern log processing than what I have done here?

The logstash config file I have is below

filter {  
grok {  
tag\_on\_failure =\> ["BROKEN\_GROK\_SYSLOG", "\_grokparsefailure"]  
overwrite =\> ["message"]  
match =\> ["message", "%{NOTSPACE} %{NOTSPACE:threadType} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE:requestType} %{NOTSPACE} %{NOTSPACE:requestStatus} %{NOTSPACE} %{GREEDYDATA:requestDetails}"]

```
}
grok {
	tag_on_failure => ["BROKEN_GROK_SYSLOG", "_grokparsefailure"]
	overwrite => ["message"]
    match => ["message", "%{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE:requestType} %{NOTSPACE} %{NOTSPACE:requestStatus} %{NOTSPACE} %{GREEDYDATA:requestDetails}"]

	}
mutate {
    remove => ["message"]
}
```

---

<div class="post-metadata">

**Author:** ![bishaka](https://avatars.discourse-cdn.com/v4/letter/b/e480ec/32.png) [@bishaka](https://discuss.elastic.co/u/bishaka)\
**Post date:** [July 20, 2016, 5:30am UTC](https://discuss.elastic.co/t/csv-filter-help/55937/4 "2016-07-20T05:30:42Z")

</div>

I am still stuck ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:47am UTC](https://discuss.elastic.co/t/csv-filter-help/55937/5 "2017-07-06T04:47:22Z")

</div>