# Is this config good ? Especially the regex one

**URL:** https://discuss.elastic.co/t/is-this-config-good-especially-the-regex-one/153382
**Category:** Beats
**Tags:** filebeat
**Created:** [October 22, 2018, 10:57am UTC](https://discuss.elastic.co/t/is-this-config-good-especially-the-regex-one/153382 "2018-10-22T10:57:59Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![merceskoba](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@merceskoba](https://discuss.elastic.co/u/merceskoba)
#### Post date: [October 22, 2018, 10:57am UTC](https://discuss.elastic.co/t/is-this-config-good-especially-the-regex-one/153382/1 "2018-10-22T10:57:59Z")

</div>

Hello dears,  
I have a filebeat config as [filebeat.yml](https://www.kimcil.id/assets/config.txt)  
Could you take a look on it and check if there is a mistake or not.  
Because, my filebeat is running correctly but when i check in Graylog2 web ui, i still find the 200 and 300 response code.  
If you look on this lines

> - type: log  
> enabled: true  
> paths:  
> - /var/log/nginx/_.access.log  
> exclude\_lines: ['HTTP[^"]_" (?:402|405|410|429|(2|3)[0-9]{2})']

200 and 300 response code should not be sent to Graylog2, right ?  
Do i make some mistakes in my filebeat.yml ?  
Thank you for your time dears.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [October 22, 2018, 3:20pm UTC](https://discuss.elastic.co/t/is-this-config-good-especially-the-regex-one/153382/2 "2018-10-22T15:20:45Z")

</div>

No idea. Have you confirmed your regex matching your logs (e.g. [online regex tester](https://regex101.com), select golang)?

Any errors in filebeat logs?

Any other contents you didn't expect?

Skimming your configuration file, I'd say it looks good. You have a quite hughe (and duplicate) list of exclude\_lines. Some way to reduce duplicates:

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/{{ app_name }}/{{ app_name }}*.access.log
  exclude_lines: ${app.shared.exclude_lines}
- type: log
  enabled: true
  paths:
    - /var/log/{{ app_name }}/{{ app_name }}*.error.log
  exclude_lines: ${app.shared.exclude_lines}

app.shared.exclude_lines: [
    'go',
    'beginning close',
    'readLoop exiting',
    'breaking out of writeLoop',
    'writeLoop exiting',
    'finished draining, cleanup exiting',
    'clean close complete',
    'connecting to nsqd',
    'stopping',
    'exiting router'
  ]

```

Given access/error log are mostly the same, why do you need 2 inputs? You plan to add some more meta-data per log type? Otherwise you can just put both paths into one input configuration:

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/{{ app_name }}/{{ app_name }}*.access.log
    - /var/log/{{ app_name }}/{{ app_name }}*.error.log
  exclude_lines: [
    'go',
    'beginning close',
    'readLoop exiting',
    'breaking out of writeLoop',
    'writeLoop exiting',
    'finished draining, cleanup exiting',
    'clean close complete',
    'connecting to nsqd',
    'stopping',
    'exiting router'
  ]

```

Exclude lines is based on regexes (sometime being optimized to sub-string matches). If the contents to be exclude is at the beginning or end of the line consider to add `^` or `$` to the regexes. This optimises the matches to into prefix/-suffix string matching (algorithm linear in time to pattern).

---

<div class="post-metadata">

### Author: ![merceskoba](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@merceskoba](https://discuss.elastic.co/u/merceskoba)
#### Post date: [October 22, 2018, 3:48pm UTC](https://discuss.elastic.co/t/is-this-config-good-especially-the-regex-one/153382/3 "2018-10-22T15:48:27Z")

</div>

I don't even think for this method `exclude_lines: ${app.shared.exclude_lines}` allowed 🙂  
Btw, thx for your suggestion... I fix the regex via regex101...  
Thx so much for your time...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 19, 2018, 3:51pm UTC](https://discuss.elastic.co/t/is-this-config-good-especially-the-regex-one/153382/4 "2018-11-19T15:51:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
