# How to prevent empty lines

**URL:** <https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490>\
**Category:** Logstash\
**Created:** [May 28, 2018, 8:38am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490 "2018-05-28T08:38:24Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![npontes](https://avatars.discourse-cdn.com/v4/letter/n/ba8739/32.png) [@npontes](https://discuss.elastic.co/u/npontes)\
**Post date:** [May 31, 2018, 10:42am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/21 "2018-05-31T10:42:32Z")

</div>

No, the base conditional. Because I don't understand what this is validating:

`if [xxx]{}`

Is it validating if the field exists, if the fields is empty. What is it validating?

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [May 31, 2018, 10:56am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/22 "2018-05-31T10:56:36Z")

</div>

It is a Rubism, a missing field returns a `nil` and Ruby interprets that as `false`

In the config `if [xx]` is the same as `if event.get("[xx]")` in Ruby - if `event.get("[xx]")` is not nil or false then the branch is taken.

You must take the `false` value into account.

Most folks are getting a log line or similar of known(ish) content and so, after grokking etc., can use this Ruby side effect to test for missing fields.

In your case the documents are coming from ES, are fully formed (no chopping up into fields needed) and are of unknown content to you.

**I suggest that you spend some time in Kibana analysing the documents in the index you are querying and find some "signature" that is common to all docs of-interest or common to all docs to exclude.  
Then query by this signature to limit the docs retrieved - bang, no conditional required.**

---

<div class="post-metadata">

**Author:** ![npontes](https://avatars.discourse-cdn.com/v4/letter/n/ba8739/32.png) [@npontes](https://discuss.elastic.co/u/npontes)\
**Post date:** [May 31, 2018, 11:53am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/23 "2018-05-31T11:53:31Z")

</div>

I think I understood. Tomorrow I'll try with this condition to see if I have different results. And I'll try as well to use Kibana to try and find the signature you proposed.

As soon as I have news, I'll let you know.

Thank you

---

<div class="post-metadata">

**Author:** ![npontes](https://avatars.discourse-cdn.com/v4/letter/n/ba8739/32.png) [@npontes](https://discuss.elastic.co/u/npontes)\
**Post date:** [June 1, 2018, 8:31am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/24 "2018-06-01T08:31:09Z")

</div>

Hello,

I've just tested your solution but no file was created ☹

I've tried as well to keep looking on the logs file every field, but it's like searching a needle in a haystack.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [June 1, 2018, 8:48am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/25 "2018-06-01T08:48:33Z")

</div>

Wait a minute, all I have said before relates to the LS behaviour of newer versions 5+

I guess you can't use a newer version of Logstash because ES is old.

Maybe your best bet is to post process the file with some python or similar - copy the correct lines to a new file.

---

<div class="post-metadata">

**Author:** ![npontes](https://avatars.discourse-cdn.com/v4/letter/n/ba8739/32.png) [@npontes](https://discuss.elastic.co/u/npontes)\
**Post date:** [June 1, 2018, 9:05am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/26 "2018-06-01T09:05:28Z")

</div>

Yap, I'm using Logstash 2.1.1 to retrieve data from Elasticsearch 1.4.4.

That was indeed my last resort. Make a script to do the post processing of the file. 😢

Thanks everyone for your suggestions and your time 👌👍

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 29, 2018, 9:05am UTC](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490/27 "2018-06-29T09:05:34Z")

</div>

Closed on June 29, 2018, 9:05am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/how-to-prevent-empty-lines/133490.md?page=1)
