Configuration example of filebeat haproxy module

[sentient]

I'm trying to get the filebeat haproxy module to work. I probably need one working example (or a list of what is required from all the dependencies)

haproxy version
1.7.7

filebeat -version
Flag --version has been deprecated, use version subcommand
filebeat version 6.5.4 (amd64), libbeat 6.5.4

/opt/elasticsearch/bin/elasticsearch -version
Version: 6.5.4, Build: oss/tar/d2ef93d/2018-12-17T21:17:40.758843Z, JVM: 1.8.0_191

Current configuration

haproxy.config

global
   log 127.0.0.1:9001 local0 debug
   log 127.0.0.1:9001 local1 debug

defaults
    log global
    mode http
    option httplog 
    option dontlognull

frontend test
  mode http
  option httplog
  log global
  default_backend b_test

backend b_test
  ..

---

filebeat configuration

filebeat.modules:
  - module: nginx
    access:
      enabled: true #
    error:
      enabled: true

  - module: haproxy
    log:
      enabled: true

The filebeat shows the message is send

2019-01-15T20:31:18.336Z        DEBUG   [publish]       pipeline/processor.go:308       Publish event: {                                  
  "@timestamp": "2019-01-15T20:31:18.000Z",                                                                                                         
  "@metadata": {                                                                                                                      
    "beat": "filebeat",                                                                                                             
    "type": "doc",                                                                                                                                                  
    "version": "6.5.4",                                                                                       
    "truncated": false,                                                                                                                   
    "pipeline": "filebeat-6.5.4-haproxy-log-pipeline"                                                                      
  },                                                                                                                                      
  "hostname": "haproxy[7594]:",                                                                                                                                      
  "message": "127.0.0.1:40042 [15/Jan/2019:20:31:18.331] lrs b_lrs/frontend-phoeix 0/0/0/1/1 200 284 - - ---- 0/0/0/0/0 0/0 \"GET /test HTTP/1.0\"",
  "fileset": {                                                                                                                        
    "module": "haproxy",                                                                                                        
    "name": "log"                                                                                          
  },                                                                                                                               
  "prospector": {                                                                                                                      
    "type": "syslog"                                                                                                                                 
  },                                                                                                                                
  "host": {                                                                                                                         
    "name": "frontend-phoeix"                                                                                                        
  },                                                                                                                                
  "source": "127.0.0.1:44693",                                                                                                                
  "event": {                                                                                                                                        
    "severity": 6                                                                                                          
  },                                                                                                                                  
  "process": {                                                                                                                  
    "program": "127.0.0.1"                                                                                                                                           
  },                                                                                                                       
  "input": {                                                                                                                                         
    "type": "syslog"                                                                                                                   
  },                                                                                                                                                 
  "beat": {                                                                                                                               
    "name": "frontend-phoeix",                                                                                                                      
    "hostname": "frontend-phoeix",                                                                                                    
    "version": "6.5.4"                                                                                                              
  },                                                                                                                                
  "syslog": {                                                                                                    
    "priority": 142,                                                                                 
    "severity_label": "Informational",                                                                                     
    "facility": 17,                                                                                                         
    "facility_label": "local1"                                                                                                                      
  }                                                                                                                                   
} 

But the haproxy dashboard is looking for
haproxy.backend_name

what am I missing?

[sentient]

I actually did found an error. So I'm choking on an Grok expression.

Provided Grok expressions do not match field value: [127.0.0.1:40042 [15/Jan/2019:20:31:18.331] lrs b_lrs/frontend-phoeix 0/0/0/1/1 200 284 - - ---- 0/0/0/0/0 0/0 \"GET /test HTTP/1.0\"]

[sentient]

I got it finally working after putting additional capture flag in haproxy.cfg

What leaves my config now at

global
    log 127.0.0.1:9001 local0
    log 127.0.0.1:9001 local1 debug
    user haproxy
    group haproxy
    daemon
    ssl-server-verify none

defaults
    log     global
    mode    http
    option  httplog
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    timeout http-keep-alive 1s

frontend lrs
    bind *:10100
    capture request header Host len 15
    capture request header X-Forwarded-For len 15
    capture request header Referer len 15
    capture response header Content-length len 9
    capture response header Location len 15
    default_backend b_lrs

backend b_lrs
    balance roundrobin
    capture request header Host len 15
    capture request header X-Forwarded-For len 15
    capture request header Referer len 15
    capture response header Content-length len 9
    capture response header Location len 15
    server frontend-phoeix 172.19.0.2:8070 check

But it took me another few hours to get the dashboard to work

I think the main issue was that I have old beats posting version 6.3.2 into a 6.5 elastic search.

So basically you get multiple different document versions going into ES.

Is there a way to stop this on the server side?

E.g. only access beats 6.5.* and reject any other document inserts?

The out of the box experience on updating beats, or enabling new modules
has not been ideal. I understand there are a lot of moving parts. Would be great if we can put some restrictions and get going more easily