# 7.7: broken filebeat pipeline for Nginx access logs **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 13, 2020, 10:08pm UTC](https://discuss.elastic.co/t/7-7-broken-filebeat-pipeline-for-nginx-access-logs/232519 "2020-05-13T22:08:24Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Slavik\_Fursov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slavik_fursov/32/48975_2.png) [@Slavik\_Fursov](https://discuss.elastic.co/u/Slavik_Fursov)\ **Post date:** [May 13, 2020, 10:08pm UTC](https://discuss.elastic.co/t/7-7-broken-filebeat-pipeline-for-nginx-access-logs/232519/1 "2020-05-13T22:08:24Z")
I just upgraded my stack to 7.7 Now, I see that I have 2 pipelines for nginx access logs: 1. filebeat-7.7.0-nginx-access-default description: "Pipeline for parsing Nginx access logs. Requires the geoip and user\_agent plugins." 2. filebeat-7.7.0-nginx-ingress\_controller-pipeline description: "Pipeline for parsing Nginx ingress controller access logs. Requires the geoip and user\_agent plugins." What is that new "ingress\_controller-pipeline"? And now here is what I see in the Kibana "Discover": ```auto event.dataset: nginx.ingress_controller message: noty.propovednik.com 2a01:4f8:161:7181::2 - - [13/May/2020:06:19:54 +0000] "GET /index.php/res/Public/%D0%93/%D0%93%D0%BE%D1%81%D0%BF%D0%BE%D0%B4%D1%8C%20%D0%B4%D0%B0%D0%B9%20%D0%B6%D0%B8%D0%B7%D0%BD%D1%8C%20%D0%B4%D1%83%D1%88%D0%B5%20%D0%BC%D0%BE%D0%B5%D0%B9/Public/S/Public/O/Oh,%20What%20a%20Change.nwc HTTP/1.1" 301 178 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)" error.message: Provided Grok expressions do not match field value: [noty.propovednik.com 2a01:4f8:161:7181::2 - - [13/May/2020:06:19:54 +0000] \"GET /index.php/res/Public/%D0%93/%D0%93%D0%BE%D1%81%D0%BF%D0%BE%D0%B4%D1%8C%20%D0%B4%D0%B0%D0%B9%20%D0%B6%D0%B8%D0%B7%D0%BD%D1%8C%20%D0%B4%D1%83%D1%88%D0%B5%20%D0%BC%D0%BE%D0%B5%D0%B9/Public/S/Public/O/Oh,%20What%20a%20Change.nwc HTTP/1.1\" 301 178 \"-\" \"Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)\"] ``` ---
**Author:** ![Slavik\_Fursov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slavik_fursov/32/48975_2.png) [@Slavik\_Fursov](https://discuss.elastic.co/u/Slavik_Fursov)\ **Post date:** [May 13, 2020, 10:23pm UTC](https://discuss.elastic.co/t/7-7-broken-filebeat-pipeline-for-nginx-access-logs/232519/2 "2020-05-13T22:23:43Z")
hm, may be it's not broken. looks like that ingress controller is a Kubernetes thing: > **[kubernetes/ingress-nginx](https://github.com/kubernetes/ingress-nginx)** > > NGINX Ingress Controller for Kubernetes. Contribute to kubernetes/ingress-nginx development by creating an account on GitHub. But I don't have anything to do with Kubernetes. I just have standalone Ubuntu web-server. Looks like that `nginx.ingress_controller` things got auto-enabled during upgrade somehow? Looks to be caused by this MR: > Can be disabled like this: ```auto - module: nginx ingress_controller: enabled: false ``` IMHO, ingress\_controller should have been disabled by default. Or, at least, not automatically added during upgrade. ---
**Author:** ![David\_Oceans](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_oceans/32/51452_2.png) [@David\_Oceans](https://discuss.elastic.co/u/David_Oceans)\ **Post date:** [May 14, 2020, 1:08pm UTC](https://discuss.elastic.co/t/7-7-broken-filebeat-pipeline-for-nginx-access-logs/232519/3 "2020-05-14T13:08:06Z")
Hi Slavik, I have some problems with the update two of them are in that post, but I'm not having lucky with the answers. > [@Problem to update to filebeat 7.7.0 and parser nginx-ingress-controller on Kubernetes](https://discuss.elastic.co/t/problem-to-update-to-filebeat-7-7-0-and-parser-nginx-ingress-controller-on-kubernetes/232461): > > Hi! I was using Kubernetes with this version of Filebeat: [docker.elastic.co/beats/filebeat:7.3.2](http://docker.elastic.co/beats/filebeat:7.3.2) In order to use the new functionalities to parser nginx-ingress-controller I've update to [docker.elastic.co/beats/filebeat:7.7.0](http://docker.elastic.co/beats/filebeat:7.7.0) But with my configuration, only changing the image I have some errors. 2020-05-13T14:26:25.084Z ERROR [kubernetes] add\_kubernetes\_metadata/matchers.go:91 Error extracting container id - source value does not contain matcher's logs\_path '/var/lib/docker/containers/'. I … I'm very interested of your update. Do you use ```auto add_kubernetes_metadata: ``` With the update from 7.3 to 7.7 I cannot use this piece ```auto - add_kubernetes_metadata: in_cluster: true host: ${NODE_NAME} matchers: - logs_path: logs_path: "/var/log/containers/" ``` How did you solve it? Thank you very much ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [May 28, 2020, 1:45pm UTC](https://discuss.elastic.co/t/7-7-broken-filebeat-pipeline-for-nginx-access-logs/232519/4 "2020-05-28T13:45:01Z")
> [@David\_Oceans](#): > > With the update from 7.3 to 7.7 I cannot use this piece > > ```auto > - add_kubernetes_metadata: > in_cluster: true > host: ${NODE_NAME} > matchers: > - logs_path: > logs_path: "/var/log/containers/" > > ``` > > How did you solve it? There was an unexpected regression in 7.7.0, a possible workaround is to disable default matchers: ```auto - add_kubernetes_metadata: in_cluster: true host: ${NODE_NAME} default_matchers.enabled: false matchers: - logs_path: logs_path: "/var/log/containers/" ``` I have opened a PR to revert to pre-7.7.0 behaviour: [Use indexers and matchers in config when defaults are enabled by jsoriano · Pull Request #18818 · elastic/beats · GitHub](https://github.com/elastic/beats/pull/18818) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 25, 2020, 1:45pm UTC](https://discuss.elastic.co/t/7-7-broken-filebeat-pipeline-for-nginx-access-logs/232519/5 "2020-06-25T13:45:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.