# Is my data just not conforming to the pipeline? Filebeat not processing iptables

**URL:** <https://discuss.elastic.co/t/is-my-data-just-not-conforming-to-the-pipeline-filebeat-not-processing-iptables/262259>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 26, 2021, 4:39pm UTC](https://discuss.elastic.co/t/is-my-data-just-not-conforming-to-the-pipeline-filebeat-not-processing-iptables/262259 "2021-01-26T16:39:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rfischman](https://avatars.discourse-cdn.com/v4/letter/r/2acd7d/32.png) [@rfischman](https://discuss.elastic.co/u/rfischman)\
**Post date:** [January 26, 2021, 4:39pm UTC](https://discuss.elastic.co/t/is-my-data-just-not-conforming-to-the-pipeline-filebeat-not-processing-iptables/262259/1 "2021-01-26T16:39:55Z")

</div>

Hopefully the group can help me out here... I've got my network infrastructure (coming out of a unifi controller) sending syslog to a filebeat over UDP/514. The messages are iptables logs. I'm not seeing any of the messages get processed into elasticsearch. I DID confirm that messages are coming into the filebeat host on 514 with a tcpdump. I also set up syslog-ng and created a test file for filebeat and tried to process it using "read from file" instead of "read from syslog". And even that fails...

Here's what my filebeat.yml looks like:

```auto
    filebeat.config:
      logging.to_files: false
      modules:
         path: ${path.config}/modules.d/*.yml
         reload.enabled: false

    logging.level: debug

    output.elasticsearch:
       hosts: '${ELASTICSEARCH_HOSTS}'
       username: '${ELASTICSEARCH_USERNAME}'
       password: '${ELASTICSEARCH_PASSWORD}'
    setup.kibana:
       host: "kib01:5601"

```

And here's my iptables.yml

```auto
    # Module: iptables
    # Docs: https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-module-iptables.html

    - module: iptables
      log:
         enabled: true
         var.convert_timezone: true
         var.syslog_host: localhost
         var.syslog_port: 514
         # Set which input to use between syslog (default) or file.
         var.input: "syslog"

```

And here's a sample of the log data that is being sent:

```auto
    Jan 26 09:18:47 30c04bee8c30,udm-1.8.5.2964 kernel: [322287.006104] IN=br30 OUT=eth4 MAC=f6:92:bf:7f:a0:e8:40:b4:cd:28:95:40:08:00 SRC=1.2.3.4 DST=4.3.2.1 LEN=136 TOS=0x00 PREC=0x00 TTL=63 ID=39401 DF PROTO=TCP SPT=37398 DPT=80 WINDOW=274 RES=0x00 ACK PSH URGP=0 
    Jan 26 09:18:47 30c04bee8c30,udm-1.8.5.2964 kernel: [322287.020222] IN=br10 OUT=eth4 MAC=f6:92:bf:7f:a0:e8:5c:3a:45:9a:c1:8d:08:00 SRC=1.2.3.4 DST=4.3.2.1 LEN=62 TOS=0x00 PREC=0x00 TTL=127 ID=38007 DF PROTO=UDP SPT=51931 DPT=443 LEN=42

```

Any help appreciated!

---

<div class="post-metadata">

**Author:** ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)\
**Post date:** [January 28, 2021, 3:05pm UTC](https://discuss.elastic.co/t/is-my-data-just-not-conforming-to-the-pipeline-filebeat-not-processing-iptables/262259/2 "2021-01-28T15:05:27Z")

</div>

hi @rfischman, can you check the Filebeat logs, maybe they contain more information on what the issue is. For a better look can enable debug level for logging so you can see if any events are being processed?

---

<div class="post-metadata">

**Author:** ![rfischman](https://avatars.discourse-cdn.com/v4/letter/r/2acd7d/32.png) [@rfischman](https://discuss.elastic.co/u/rfischman)\
**Post date:** [January 29, 2021, 3:54pm UTC](https://discuss.elastic.co/t/is-my-data-just-not-conforming-to-the-pipeline-filebeat-not-processing-iptables/262259/3 "2021-01-29T15:54:00Z")

</div>

So I'm not sure what happened but when I restarted it with with debug mode enabled, everything started working... I then switched it back to run without debugging and it continued to function.

Very odd behavior, but I'm not going to argue with it!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 26, 2021, 5:54pm UTC](https://discuss.elastic.co/t/is-my-data-just-not-conforming-to-the-pipeline-filebeat-not-processing-iptables/262259/4 "2021-02-26T17:54:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.