Dont have Query count and cumulated duration [Filebeat PostgreSQL]

[patsevanton (Anton Patsev)]

Hi. ELK stack version 6.6.2. I deployed a new server with PostgreSQL and I use Filebeat to send PostgreSQL logs the the elasticsearch cluster. I added the pluggin PostgreSQL and imported the dashboard (filebeat setup -e).

Dont have:
Query count and cumulated duration [Filebeat PostgreSQL]
Query Durations [Filebeat PostgreSQL]
Slow Queries [Filebeat PostgreSQL]

cat modules.d/postgresql.yml

- module: postgresql
  # All logs
  log:
    enabled: true

    var.paths:
     - /var/lib/pgsql/11/data/log/*.log

cat /var/lib/pgsql/11/data/postgresql.conf | grep -v "^#" | grep -v "^$"

max_connections = 100 
shared_buffers = 128MB
dynamic_shared_memory_type = posix 
max_wal_size = 1GB
min_wal_size = 80MB
log_destination = 'stderr'
logging_collector = on
log_directory = 'log' 
log_filename = 'postgresql-%a.log' 
log_truncate_on_rotation = on 
log_rotation_age = 1d 
log_rotation_size = 0 
log_duration = on
log_line_prefix = '%m [%p] '
log_timezone = 'W-SU'
datestyle = 'iso, mdy'
timezone = 'W-SU'
lc_messages = 'en_US.UTF-8' 
lc_monetary = 'en_US.UTF-8' 
lc_numeric = 'en_US.UTF-8'
lc_time = 'en_US.UTF-8' 
default_text_search_config = 'pg_catalog.english'

[kvch (Noémi Ványi)]

I am not sure I understand your problem. Could you please provide more information?

[patsevanton (Anton Patsev)]

Can you send example postgresql.conf for parse log of Filebeat?

[kvch (Noémi Ványi)]

Your configuration is correct.

[fredgalvao (Frederico Costa Galvão)]

I'm having an even harder time getting this to work for me. Grok can't understand logs from the default PostgreSQL@11 pattern, no matter how I setup log_line_prefix. I have tried:

• empty/default
• '%m [%p] ' like the OP
• '%t [%p]: [%l-1] user=%u,db=%d,client=%h,appname=%a' like used in production for me

I was hoping the default-everything would "just work ©". I went through all of 7.0.0-rc1 beat docs and found nothing about having to specify log pattern for grok, not even how to do it.