# Logstash issue with dropping rows with a condition check on empty elements **URL:** \ **Category:** Logstash\ **Created:** [January 7, 2020, 9:07pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132 "2020-01-07T21:07:41Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Nick11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick11/32/53553_2.png) [@Nick11](https://discuss.elastic.co/u/Nick11)\ **Post date:** [January 7, 2020, 9:07pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/1 "2020-01-07T21:07:41Z")
I have a CSV file with the following 2 rows (sample) ``` reservation date, reservationID Jan 6th, res:id:adbcj-oksok-gjkk Jan 10th, Mar 10th, res:id:kkbcj-oksok-gjkk ``` My ask is to drop empty rows and apply a grok filter on reservationID to extract the last elements after the "-". This is what I did without success ``` csv { separator => "," skip_header => "true" autodetect_column_names => "true" skip_empty_columns => "true" skip_empty_rows => "true" } if [reservationID] =~ "" { grok { reservationID => "MY GROK PATTERN HERE, WHICH IS WORKING FINE EXTERNALLY THROUGH THE DEBUGGER" } } ``` I was expecting the first and the third row in the output (not worried about the grok). Instead I see all 3 rows. Am I missing anything. I do not want the 2nd row in my output. Thanks ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 7, 2020, 9:15pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/2 "2020-01-07T21:15:26Z")
You do not have any empty rows, so it will not skip any. The second line will not have a [reservationID] field (because you have set skip\_empty\_columns). So test that: if ! [reservationID] { drop {} } ---
**Author:** ![Nick11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick11/32/53553_2.png) [@Nick11](https://discuss.elastic.co/u/Nick11)\ **Post date:** [January 7, 2020, 9:44pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/3 "2020-01-07T21:44:53Z")
Thanks for your reply. I did try that and it did not work. I still get the 2nd row. I suspect that the skip\_empty\_columns condition is stripping the reservationID field even before I get a chance to do what you are suggesting. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 7, 2020, 9:54pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/4 "2020-01-07T21:54:21Z")
That's the idea. That is meant to test whether the reservationID field exists, and dropping the event if it does not. ---
**Author:** ![Nick11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick11/32/53553_2.png) [@Nick11](https://discuss.elastic.co/u/Nick11)\ **Post date:** [January 7, 2020, 11:03pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/5 "2020-01-07T23:03:50Z")
Yes, makes sense. However, for the expression to evaluate the reservationID needs to be present. What does skip\_empty\_colums exactly do? Does it strip out the column completely? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 8, 2020, 12:09am UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/6 "2020-01-08T00:09:19Z")
> [@Nick11](#): > > However, for the expression to evaluate the reservationID needs to be present No, that is not correct. If skip\_empty\_columns is set then columns containing no value [will not get set](https://github.com/logstash-plugins/logstash-filter-csv/blob/87d00ed77e1ecbc66b141b35a65208125c045d49/lib/logstash/filters/csv.rb#L153). ---
**Author:** ![Nick11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick11/32/53553_2.png) [@Nick11](https://discuss.elastic.co/u/Nick11)\ **Post date:** [January 8, 2020, 3:03am UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/7 "2020-01-08T03:03:36Z")
If that's the case then how can you use a conditional statement on that column in the next subsequent line. Did you try your solution? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 8, 2020, 2:51pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/8 "2020-01-08T14:51:51Z")
> [@Nick11](#): > > If that's the case then how can you use a conditional statement on that column in the next subsequent line. The conditional is testing whether the field exists. Note that your header row has a leading space on the column name so ``` if ! [reservationID] { drop {} } ``` will drop everything. ``` if ! [reservationID] { drop {} } ``` will just drop the second row. And yes, I tested it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 5, 2020, 2:51pm UTC](https://discuss.elastic.co/t/logstash-issue-with-dropping-rows-with-a-condition-check-on-empty-elements/214132/9 "2020-02-05T14:51:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.