# NiFi + ElasticSearch + filebeat

**URL:** <https://discuss.elastic.co/t/nifi-elasticsearch-filebeat/296946>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [February 11, 2022, 8:13am UTC](https://discuss.elastic.co/t/nifi-elasticsearch-filebeat/296946 "2022-02-11T08:13:21Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![iconm](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@iconm](https://discuss.elastic.co/u/iconm)\
**Post date:** [February 11, 2022, 8:13am UTC](https://discuss.elastic.co/t/nifi-elasticsearch-filebeat/296946/1 "2022-02-11T08:13:21Z")

</div>

I urgently need to create a pipeline in NiFi that accepts syslog on local UDP port 514 and sends it to Elasticsearch.

And I have to use filebeat and build a simple Nifi pipeline that collects data from local /var /log/messages by filebeb and sends it to Elasticsearch.

On Imgur, I couldn't upload screenshot and I used 0bin.

1. Installed on Debian 11 - NiFi, Docker Elasticsearch, Docker Kibana, Filebeat from DEB file.  
[0bin - encrypted pastebin](https://0bin.net/paste/jgqR0oqj#ah8vLMyOkoY12hdDlefYEHyVJMZxr1zBRZuEALPQnQC)

I don't know why the image of the kibana docker stops sometimes, any suggestions?

When everything seems to work, Kibana throws something like this.

> **[0bin - encrypted pastebin](https://0bin.net/paste/+FpsBW+S#xCOFZFtHZQiwdfTWvWEEg6k7KUC4+i3nQUDGg7XpfBY)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

Elasticsearch by localhost seems to be working fine

> **[0bin - encrypted pastebin](https://0bin.net/paste/Lk+p9p8O#Kc7ipJDmYwuXY29uhc3CIedXZjiNweB63qPvE6v787t)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

1. I have built a NiFi pipe to send data to Elasticsearch.  
[0bin - encrypted pastebin](https://0bin.net/paste/O6uImVLM#1IwsbUQXqujLbfXemY6AUGtdMQBiPLDzRec1b3hcnlo)

ListenUDP setup -

> **[0bin - encrypted pastebin](https://0bin.net/paste/APSQ+1pa#cEpONXPOrC8ube5iBpiRTpR1yg+dQzc1lsh8VwOsT3l)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

PutElasticsearchHttp -

> **[0bin - encrypted pastebin](https://0bin.net/paste/H37h5VLK#z1ijsPbWumUv0ZYl7fn+HoqzjoCge8h0Of94c55xXh0)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

1. I tried to configure filebeat but I don't know if it is correct.

/etc/filebeat/filebeat.yml -

> **[0bin - encrypted pastebin](https://0bin.net/paste/MPP1NwVw#9hHUO3TbXVNC2OT29F6OLPhfKnwHrAXWqvemDUaTKNG)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

kibana.yml -

> **[0bin - encrypted pastebin](https://0bin.net/paste/cspLLYTq#ubeHqXLYHauTHs-1piFt4XlNlSkBKeXoaqBouhJaD3Z)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

Elasticsearch.yml-

> **[0bin - encrypted pastebin](https://0bin.net/paste/QCzamGhn#urJxzXwPLq8-wurXkniBn761mPrwqt8QsKxRCD5bTmX)**
>
> 0bin is a client-side-encrypted alternative pastebin. You can store code/text/images online for a set period of time and share with the world. Featuring burn after reading, history, clipboard.

I really need help 😕

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 11, 2022, 8:13am UTC](https://discuss.elastic.co/t/nifi-elasticsearch-filebeat/296946/2 "2022-03-11T08:13:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.