# Filebeat Nginx module

**URL:** <https://discuss.elastic.co/t/filebeat-nginx-module/101488>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 22, 2017, 12:56pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module/101488 "2017-09-22T12:56:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Youkoulayley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youkoulayley/32/22316_2.png) [@Youkoulayley](https://discuss.elastic.co/u/Youkoulayley)\
**Post date:** [September 22, 2017, 12:56pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module/101488/1 "2017-09-22T12:56:40Z")

</div>

Hello,

I I have a filebeat who send nginx logs to logstash with this conf :

```auto
###########################################
# HEARTBEAT.YML
# Generated by Ansible
###########################################

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/syslog

filebeat.modules:
- module: nginx
  access:
    var.paths:
      - /var/log/nginx/*_access.log
  error:
    - /var/log/nginx/*_error.log

output.logstash:
  hosts:
    - [host]:5044

```

In my Logstash configuration, I have this :

```auto
input {
  beats {
    port => 5044
  }
}

filter {
    if [source] =~ "_access.log" {
  grok {
    match => { "message" => ["%{IPORHOST:[nginx][access][remote_ip]} - %{DATA:[nginx][access][user_name]} \[%{HTTPDATE:[nginx][access][time]}\] \"%{WORD:[nginx][access][method]} %{DATA:[nginx][access][url]} HTTP/%{NUMBER:[nginx][access][http_version]}\" %{NUMBER:[nginx][access][response_code]} %{NUMBER:[nginx][access][body_sent][bytes]} \"%{DATA:[nginx][access][referrer]}\" \"%{DATA:[nginx][access][agent]}\""] }
    remove_field => "message"
    tag_on_failure => ["nginx_access_log"]
  }
  mutate {
    rename => { "@timestamp" => "read_timestamp" }
  }
  date {
    match => ["[nginx][access][time]", "dd/MMM/YYYY:H:m:s Z" ]
    remove_field => "[nginx][access][time]"
  }
  useragent {
    source => "[nginx][access][agent]"
    target => "[nginx][access][user_agent]"
    remove_field => "[nginx][access][agent]"
  }
  geoip {
    source => "[nginx][access][remote_ip]"
    target => "[nginx][access][geoip]"
  }
}
else if [source] =~ "_error.log" {
  grok {
    match => { "message" => ["%{DATA:[nginx][error][time]} \[%{DATA:[nginx][error][level]}\] %{NUMBER:[nginx][error][pid]}#%{NUMBER:[nginx][error][tid]}: (\*%{NUMBER:[nginx][error][connection_id]} )?%{GREEDYDATA:[nginx][error][message]}"] }
    remove_field => "message"
    tag_on_failure => ["nginx_error_log"]
  }
  mutate {
    rename => { "@timestamp" => "read_timestamp" }
  }
  date {
    match => ["[nginx][error][time]", "YYYY/MM/dd H:m:s" ]
    remove_field => "[nginx][error][time]"
  }
  geoip {
    source => "[nginx][error][remote_ip]"
    target => "[nginx][error][geoip]"
  }
}
else {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    tag_on_failure => ["syslog"]
  }
  date {
    match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
  }
}

}

output {
  elasticsearch {
    hosts => ["[host]:9200"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

Everything was working fine except that I can't create map in visualization because my nginx.access.geoip.location field wasn't in geoip format, so I have done this :

```auto
{
  "filebeat": {
    "order": 0,
    "template": "filebeat-*",
    "settings": {
      "index": {
        "number_of_shards": "2",
        "number_of_replicas": "1"
      }
    },
    "mappings": {
      "my_type": {
        "dynamic": "true",
        "properties": {
          "geoip": {
            "dynamic": true,
            "properties": {
              "location": {
                "type": "geo_point"
              }
            }
          }
        }
      }
    },
    "aliases": {}
  }
}

```

And then my field was correct but, now, I don't have any informations send by filebeat !

Do you have any idea why ?

---

<div class="post-metadata">

**Author:** ![Youkoulayley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/youkoulayley/32/22316_2.png) [@Youkoulayley](https://discuss.elastic.co/u/Youkoulayley)\
**Post date:** [October 3, 2017, 7:38am UTC](https://discuss.elastic.co/t/filebeat-nginx-module/101488/2 "2017-10-03T07:38:53Z")

</div>

No ideas ?

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [October 3, 2017, 1:40pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module/101488/3 "2017-10-03T13:40:21Z")

</div>

Did you [manually install the Filebeat index template](https://www.elastic.co/guide/en/beats/filebeat/5.6/filebeat-template.html#load-template-manually)?

The [index template](https://github.com/elastic/beats/blob/v5.6.2/filebeat/filebeat.template.json#L377-L414) has a `nginx.access.geoip` field where `location` is a `geo_point` type.

I see you have a geoip filter declared for the error log, but there is no remote\_ip field in the grok pattern for the error log. I think this can be removed.

> [@Youkoulayley](#):
>
> ```auto
> access:
> var.paths:
> - /var/log/nginx/*_access.log
> error:
> - /var/log/nginx/*_error.log
> 
> ```

Looks like you are missing `var.paths` in the error log config.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 31, 2017, 1:40pm UTC](https://discuss.elastic.co/t/filebeat-nginx-module/101488/4 "2017-10-31T13:40:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.