# Configuring Filebeat to pack openresty/nginx logs and visualize in Kibana dashboards

**URL:** <https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300>\
**Category:** Beats\
**Tags:** filebeat, dashboard\
**Created:** [July 26, 2023, 11:30am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300 "2023-07-26T11:30:08Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kpagcha](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kpagcha/32/123897_2.png) [@kpagcha](https://discuss.elastic.co/u/kpagcha)\
**Post date:** [July 26, 2023, 11:30am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300/1 "2023-07-26T11:30:08Z")

</div>

I am totally new with the ELK stack and not really a sysadmin either, just a web developer trying to figure this out.

I have two droplets:

- one where I installed the ELK stack successfully to some extent (managed to visualize syslogs with the filebeat system module)
- another where my openresty web server works. openresty is an nginx distribution.

I managed to configure filebeat in the openresty droplet to send the logs:

```auto
...
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/scheduler/logs/openresty/access.log
    - /home/scheduler/logs/openresty/error.log
  fields:
    type: nginx

...
# commented out Elasticsearch output
#output.elasticsearch:
...
output.logstash:
  hosts: ["<ELK_DROPLET_PUBLIC_IP>:5044"]
...

```

And in the ELK droplet I have this logstash configuration file:

```auto
input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "openresty-logs-%{+YYYY.MM.dd}"
    pipeline => "openresty_log_type_pipeline"
  }
}

```

This works and I have even set up a pipeline through Kibana to classify access/error entries.

Now, I am trying to get the Kibana dashboards for nginx to work with my openresty logs but after much research and effort I have hit a wall.

I have enabled the nginx module in the openresty droplet:

```auto
sudo filebeat modules enable nginx

```

And configured the filebeat module to use the openresty logs:

```auto
# /etc/filebeat/modules.d/nginx.yml
- module: nginx
  access:
    enabled: true
    var.paths: ["/home/scheduler/logs/openresty/access.log"]

  error:
    enabled: true
    var.paths: ["/home/scheduler/logs/openresty/error.log"]

  ingress_controller:
    enabled: false

```

But I am not even sure if this is working. Is there a way to tell whether the nginx data is being shipped with filebeat to logstash in the other droplet? Modules are imported by default in the filebeat config.

I am aware that I will probably need to take care of proper log formatting. But the first worry is not being able to get the openresty/nginx logs to Logstash and indexed properly for the dashboard to be able to take the data.

What do I need to do to have the Kibana nginx dashboards work with my openresty logs?

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [July 31, 2023, 4:44pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300/2 "2023-07-31T16:44:33Z")

</div>

Hey @kpagcha, welcome to discuss 🙂

If you are starting with the Elastic Stack now, let me give you a couple of tips 🙂

First, you probably don't need Logstash, Beats can push data directly to Elasticsearch. Logstash is great when you need advanced processing or you have complex architectures, but at this point you probably don't have any of them.

Second, take a look to Fleet. With it you only need to install a single agent on any of your droplets, and then use Kibana to manage the specific configurations you need on each one of them. Under the hood, Fleet and Agent also use Beats. And you can find the system and nginx integrations also in Fleet.  
Take a look here: [Fleet and Elastic Agent overview | Fleet and Elastic Agent Guide [8.9] | Elastic](https://www.elastic.co/guide/en/fleet/current/fleet-overview.html)

If you want to continue using Filebeat directly without Fleet, I would still recommend you to use the elasticsearch output directly, without Logstash. It will be easier to start with.

---

<div class="post-metadata">

**Author:** ![kpagcha](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kpagcha/32/123897_2.png) [@kpagcha](https://discuss.elastic.co/u/kpagcha)\
**Post date:** [August 1, 2023, 8:12am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300/3 "2023-08-01T08:12:00Z")

</div>

Thank you, I'll have a look.

So you're suggesting I skip logstash and send directly to elasticsearch? For that I'd comment out the logstash block in the filebeat.yml file right? Would the index be created automatically? And most importantly how do I plugin in the nginx Kibana dashboard? This is the same issue I had before.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [August 1, 2023, 8:18am UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300/4 "2023-08-01T08:18:57Z")

</div>

If connected directly to Elasticearch, Filebeat will take care of creating the index, yes. And also the ingest pipelines.

For the dashboard, if it is installed and you are ingesting data directly with Filebeat, it should work. If not, what is the problem you are having? Is it not showing any data?

If you haven't installed it, remember to run `filebeat setup`. Find docs [here](https://www.elastic.co/guide/en/beats/filebeat/8.9/filebeat-installation-configuration.html#setup-assets).

---

<div class="post-metadata">

**Author:** ![kpagcha](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kpagcha/32/123897_2.png) [@kpagcha](https://discuss.elastic.co/u/kpagcha)\
**Post date:** [August 1, 2023, 12:15pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300/6 "2023-08-01T12:15:25Z")

</div>

I have configured filebeat to point to Elasticsearch and Kibana on the corresponding server, have ports 9200 and 5601 listening and changed networks to 0.0.0.0. Filebeat seems to be properly connected.

It seems Filebeat is finally working, I can see updates in the index:

```auto
yellow open filebeat-7.17.11-2023.07.26-000001 AXJ_ISJyQTyspVe5w6JxfA 1 1 194682 0 45.8mb 45.8mb

```

However, the dashboard shows empty data: "no results found".

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 29, 2023, 2:16pm UTC](https://discuss.elastic.co/t/configuring-filebeat-to-pack-openresty-nginx-logs-and-visualize-in-kibana-dashboards/339300/7 "2023-08-29T14:16:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.