# Filebeat nginx access logs not loading with required fields in kibana dashboard

**URL:** <https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 17, 2020, 8:47am UTC](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944 "2020-09-17T08:47:23Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pratap1991](https://avatars.discourse-cdn.com/v4/letter/p/b487fb/32.png) [@pratap1991](https://discuss.elastic.co/u/pratap1991)\
**Post date:** [September 17, 2020, 8:47am UTC](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944/1 "2020-09-17T08:47:24Z")

</div>

filebeat nginx access logs not loading with required fields in kibana dashboard

i am using below logstash filter for to get access logs fields but i am not able get required fields ( access.message and other fields)  
################################

filter {  
if [fileset][module] == "nginx" {  
if [fileset][name] == "access" {  
grok {  
match =\> { "message" =\> ["%{HTTPDATE:[nginx][access][time]} [%{GREEDYDATA:[nginx][access][message]}] %{NUMBER:[nginx][access][pid]}#%{NUMBER:[nginx][access][tid]}: (\*%{NUMBER:[nginx][access][connection\_id]} )?%{GREEDYDATA:[nginx][access][message]}"] }  
remove\_field =\> "message"  
}  
mutate {  
add\_field =\> { "read\_timestamp" =\> "%{@timestamp}" }  
}  
date {  
match =\> ["[nginx][access][time]", "dd/MMM/YYYY:H:m:s Z" ]  
remove\_field =\> "[nginx][access][time]"  
}  
useragent {  
source =\> "[nginx][access][agent]"  
target =\> "[nginx][access][user\_agent]"  
remove\_field =\> "[nginx][access][agent]"  
}  
geoip {  
source =\> "[nginx][access][remote\_ip]"  
target =\> "[nginx][access][geoip]"  
}  
}  
else if [fileset][name] == "error" {  
grok {  
match =\> { "message" =\> ["%{DATA:[nginx][error][time]} [%{DATA:[nginx][error][level]}] %{NUMBER:[nginx][error][pid]}#%{NUMBER:[nginx][error][tid]}: (\*%{NUMBER:[nginx][error][connection\_id]} )?%{GREEDYDATA:[nginx][error][message]}"] }  
remove\_field =\> "message"  
}  
mutate {  
rename =\> { "@timestamp" =\> "read\_timestamp" }  
}  
date {  
match =\> ["[nginx][error][time]", "YYYY/MM/dd H:m:s" ]  
remove\_field =\> "[nginx][error][time]"  
}  
}  
}  
}  
#########################

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/4/8439d248b4609e8d7e045f4f170695ef7464f0cb.png)

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [September 17, 2020, 10:28pm UTC](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944/2 "2020-09-17T22:28:53Z")

</div>

Any reason you are not using Filebeat's [`nginx`](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-nginx.html) module and sending the data directly from Filebeat to Elasticsearch instead?

---

<div class="post-metadata">

**Author:** ![pratap1991](https://avatars.discourse-cdn.com/v4/letter/p/b487fb/32.png) [@pratap1991](https://discuss.elastic.co/u/pratap1991)\
**Post date:** [September 18, 2020, 5:12am UTC](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944/3 "2020-09-18T05:12:44Z")

</div>

i am using filebeat local and sending nginx logs to filebeat --\> logstassh --\> elasticsearch  
##############################  
filebeat.yml

#----------------------------- Logstash output --------------------------------  
output.logstash:

# The Logstash hosts

hosts: ["localhost:5044"]

########################

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 16, 2020, 7:13am UTC](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944/4 "2020-10-16T07:13:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.