# Nginx logs in Kibana log streaming!

**URL:** <https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944>\
**Category:** Beats\
**Tags:** beats-module, filebeat\
**Created:** [August 6, 2019, 7:14am UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944 "2019-08-06T07:14:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sv\_bcvt92](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@sv\_bcvt92](https://discuss.elastic.co/u/sv_bcvt92)\
**Post date:** [August 6, 2019, 7:14am UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944/1 "2019-08-06T07:14:11Z")

</div>

Hi Guru's,  
I'm setting up Laravel with Nginx.  
And I expose Laravel access logs to Nginx logs as config below:

```
server {
        listen 80;
        listen [::]:80 ipv6only=on;
        server_name abcyxyz.ap-northeast-1.elb.amazonaws.com;
	
        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        root /usr/share/nginx/laravel/public;
        index index.php index.html index.htm;

        location / {
                try_files $uri $uri/ /index.php?$query_string;
        }

        location ~ \.php$ {
                try_files $uri =404;
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass unix:/run/php-fpm/www.sock;
                fastcgi_index index.php;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                include fastcgi_params;
        }
}

```

And in /var/log/nginx/access.log I get:

 ![nginx](https://us1.discourse-cdn.com/elastic/original/3X/c/c/cc02411bb3d8b7d765ed18a456696b38a97966b1.png)

But in Logs streaming I get:

 ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/2/1/21e26f2dfa04449e4ed82d2b8e15fedb49372fd1.png)

As you can see in access log, logs are same format, but in Kibana, it's missing nginx.access in many records. Can anyone help me in this issue?

I need to query logs as http response code

 ![http](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0f8ae3054692234ea8b014c49879de84a8394a46.png)

Thanks

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [August 6, 2019, 12:09pm UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944/2 "2019-08-06T12:09:01Z")

</div>

What's your NGINX version? Could you please share your Filebeat configuration and format it using `</>`?

---

<div class="post-metadata">

**Author:** ![sv\_bcvt92](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@sv\_bcvt92](https://discuss.elastic.co/u/sv_bcvt92)\
**Post date:** [August 7, 2019, 4:47am UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944/3 "2019-08-07T04:47:38Z")

</div>

Hi,  
After investigations, I realize that every record is duplicated. Now I cant explain why it is duplicated 🙂  
My filebeat.yml  
###################### Filebeat Configuration Example #########################  
#=========================== Filebeat inputs =============================

```
filebeat.inputs:
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/nginx/*.log
 
#============================= Filebeat modules ===============================

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

  host: "abcxyz.compute.amazonaws.com:5601"

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["abcxyz.compute.amazonaws.com:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

#================================= Migration ==================================

# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true
```

---

<div class="post-metadata">

**Author:** ![sv\_bcvt92](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@sv\_bcvt92](https://discuss.elastic.co/u/sv_bcvt92)\
**Post date:** [August 7, 2019, 6:57am UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944/4 "2019-08-07T06:57:14Z")

</div>

It's so weird!  
I setup Filebeat in another Docker container with the same configurations (nginx, filebeat...), but logs aren't formated in Kibana

 ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/9/8/9892c5564edb9ed8df08408a80c1b8664a78719c.png)  
Just logs from 10.1.x.x are formated, 10.2.x.x are not

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [August 9, 2019, 3:32pm UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944/5 "2019-08-09T15:32:12Z")

</div>

What is your NGINX version? I think you are trying to parse logs whose format is unsupported.  
Our modules were tested with NGINX 1.10: [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-nginx.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-nginx.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 6, 2019, 5:32pm UTC](https://discuss.elastic.co/t/nginx-logs-in-kibana-log-streaming/193944/6 "2019-09-06T17:32:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.