# Logtash ignores some lines randomly
**URL:** https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259
**Category:** Logstash
**Created:** [January 8, 2020, 2:46pm UTC](https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259 "2020-01-08T14:46:51Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@JustDevZero](https://discuss.elastic.co/u/JustDevZero)
#### Post date: [January 8, 2020, 2:46pm UTC](https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259/1 "2020-01-08T14:46:51Z")
Logtash apparently ignores some of my log lines done by nginx.
I'm trying to parse my customized nginx logs with logstash, I'm adding the response time by upstream, etc... and a few more fields that aren't normally on the nginx logs but have a lot of interest.
But somehow logstash seems to ignore some lines, even the grok debugger ([https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)) doesn't seem do discard them.
Following is the ignored line that apparently match with the first posibility:
> "08/Jan/2020:14:22:47 +0000" client=88.217.181.177 method=POST request="POST /common/message\_count HTTP/2.0" request\_length=402 status=200 bytes\_sent=884 body\_bytes\_sent=90 referer=[https://backoffice.andronautic.com/planner](https://backoffice.andronautic.com/planner) user\_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0" upstream\_addr=unix:/var/run/popeye/back.socket upstream\_status=200 request\_time=0.731 ssl\_session\_reused=r upstream\_response\_time=0.732 upstream\_connect\_time=0.000 upstream\_header\_time=0.732
And next the logstash.conf that I use, attached on pastebin due to weird pasting here:
> **[input { beats { port =\> 5044 }}filter { if "beats\_input\_code -...](https://pastebin.com/L5vQzbMU)**
>
> Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
Why is it ignored on logstash/kibana? /tmp/grok\_failures is empty for hours... but still, ignoring some lines ☹
Many thanks!
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 8, 2020, 3:18pm UTC](https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259/2 "2020-01-08T15:18:41Z")
I cannot say why some lines are ignored but you can simplify your configuration by having grok do the conversions using, for example,
```
bytes_sent=(?:%{NUMBER:[nginx][access][bytes_sent]:int}|-)
upstream_header_time=(?:%{NUMBER:[upstream][access][header_time]:float}|-)
```
Personally I wouldn't bother converting something like status\_code to an integer, since it is not helpful to be able to calculate the average status code, or the total of status codes over a set of requests.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 5, 2020, 3:18pm UTC](https://discuss.elastic.co/t/logtash-ignores-some-lines-randomly/214259/3 "2020-02-05T15:18:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.