# Configuration example of filebeat haproxy module

**URL:** <https://discuss.elastic.co/t/configuration-example-of-filebeat-haproxy-module/164372>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 15, 2019, 8:38pm UTC](https://discuss.elastic.co/t/configuration-example-of-filebeat-haproxy-module/164372 "2019-01-15T20:38:19Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\
**Post date:** [January 15, 2019, 8:38pm UTC](https://discuss.elastic.co/t/configuration-example-of-filebeat-haproxy-module/164372/1 "2019-01-15T20:38:19Z")

</div>

I'm trying to get the filebeat haproxy module to work. I probably need one working example (or a list of what is required from all the dependencies)

> haproxy version  
> 1.7.7

> filebeat -version  
> Flag --version has been deprecated, use version subcommand  
> filebeat version 6.5.4 (amd64), libbeat 6.5.4

> /opt/elasticsearch/bin/elasticsearch -version  
> Version: 6.5.4, Build: oss/tar/d2ef93d/2018-12-17T21:17:40.758843Z, JVM: 1.8.0\_191

Current configuration

haproxy.config

```auto
global
   log 127.0.0.1:9001 local0 debug
   log 127.0.0.1:9001 local1 debug

defaults
    log global
    mode http
    option httplog 
    option dontlognull

frontend test
  mode http
  option httplog
  log global
  default_backend b_test

backend b_test
  ..

```

* * *

filebeat configuration

```auto
filebeat.modules:
  - module: nginx
    access:
      enabled: true #
    error:
      enabled: true

  - module: haproxy
    log:
      enabled: true

```

The filebeat shows the message is send

```auto
2019-01-15T20:31:18.336Z DEBUG [publish] pipeline/processor.go:308 Publish event: {                                  
  "@timestamp": "2019-01-15T20:31:18.000Z",                                                                                                         
  "@metadata": {                                                                                                                      
    "beat": "filebeat",                                                                                                             
    "type": "doc",                                                                                                                                                  
    "version": "6.5.4",                                                                                       
    "truncated": false,                                                                                                                   
    "pipeline": "filebeat-6.5.4-haproxy-log-pipeline"                                                                      
  },                                                                                                                                      
  "hostname": "haproxy[7594]:",                                                                                                                                      
  "message": "127.0.0.1:40042 [15/Jan/2019:20:31:18.331] lrs b_lrs/frontend-phoeix 0/0/0/1/1 200 284 - - ---- 0/0/0/0/0 0/0 \"GET /test HTTP/1.0\"",
  "fileset": {                                                                                                                        
    "module": "haproxy",                                                                                                        
    "name": "log"                                                                                          
  },                                                                                                                               
  "prospector": {                                                                                                                      
    "type": "syslog"                                                                                                                                 
  },                                                                                                                                
  "host": {                                                                                                                         
    "name": "frontend-phoeix"                                                                                                        
  },                                                                                                                                
  "source": "127.0.0.1:44693",                                                                                                                
  "event": {                                                                                                                                        
    "severity": 6                                                                                                          
  },                                                                                                                                  
  "process": {                                                                                                                  
    "program": "127.0.0.1"                                                                                                                                           
  },                                                                                                                       
  "input": {                                                                                                                                         
    "type": "syslog"                                                                                                                   
  },                                                                                                                                                 
  "beat": {                                                                                                                               
    "name": "frontend-phoeix",                                                                                                                      
    "hostname": "frontend-phoeix",                                                                                                    
    "version": "6.5.4"                                                                                                              
  },                                                                                                                                
  "syslog": {                                                                                                    
    "priority": 142,                                                                                 
    "severity_label": "Informational",                                                                                     
    "facility": 17,                                                                                                         
    "facility_label": "local1"                                                                                                                      
  }                                                                                                                                   
} 

```

But the haproxy dashboard is looking for  
`haproxy.backend_name`

what am I missing?

---

<div class="post-metadata">

**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\
**Post date:** [January 15, 2019, 8:55pm UTC](https://discuss.elastic.co/t/configuration-example-of-filebeat-haproxy-module/164372/2 "2019-01-15T20:55:20Z")

</div>

I actually did found an error. So I'm choking on an Grok expression.

```auto
Provided Grok expressions do not match field value: [127.0.0.1:40042 [15/Jan/2019:20:31:18.331] lrs b_lrs/frontend-phoeix 0/0/0/1/1 200 284 - - ---- 0/0/0/0/0 0/0 \"GET /test HTTP/1.0\"]

```

---

<div class="post-metadata">

**Author:** ![sentient](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentient/32/34996_2.png) [@sentient](https://discuss.elastic.co/u/sentient)\
**Post date:** [January 16, 2019, 5:52am UTC](https://discuss.elastic.co/t/configuration-example-of-filebeat-haproxy-module/164372/3 "2019-01-16T05:52:52Z")

</div>

I got it finally working after putting additional capture flag in haproxy.cfg

What leaves my config now at

```auto
global
    log 127.0.0.1:9001 local0
    log 127.0.0.1:9001 local1 debug
    user haproxy
    group haproxy
    daemon
    ssl-server-verify none

defaults
    log global
    mode http
    option httplog
    timeout connect 5000
    timeout client 50000
    timeout server 50000
    timeout http-keep-alive 1s

frontend lrs
    bind *:10100
    capture request header Host len 15
    capture request header X-Forwarded-For len 15
    capture request header Referer len 15
    capture response header Content-length len 9
    capture response header Location len 15
    default_backend b_lrs

backend b_lrs
    balance roundrobin
    capture request header Host len 15
    capture request header X-Forwarded-For len 15
    capture request header Referer len 15
    capture response header Content-length len 9
    capture response header Location len 15
    server frontend-phoeix 172.19.0.2:8070 check

```

But it took me another few hours to get the dashboard to work

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/6/d6a2d1cfb57290661a651852829874901e7c1a85.png)

I think the main issue was that I have old beats posting version 6.3.2 into a 6.5 elastic search.

So basically you get multiple different document versions going into ES.

Is there a way to stop this on the server side?

E.g. only access beats 6.5.\* and reject any other document inserts?

The out of the box experience on updating beats, or enabling new modules  
has not been ideal. I understand there are a lot of moving parts. Would be great if we can put some restrictions and get going more easily

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2019, 5:55am UTC](https://discuss.elastic.co/t/configuration-example-of-filebeat-haproxy-module/164372/4 "2019-02-13T05:55:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
