Pgsql invalid column_length

Elastic Stack Beats
1

Hi there,

I'm running Packetbeat 5.0.0~alpha1, and getting a lot of entries like the following in syslog:

Apr 28 16:04:08 cat-pg92 /usr/bin/packetbeat[30951]: parse.go:531: Pgsql invalid column_length=4294967295, buffer_length=234, i=5
Apr 28 16:04:08 cat-pg92 /usr/bin/packetbeat[30951]: parse.go:531: Pgsql invalid column_length=4294967295, buffer_length=558, i=11

I can't find any reference to this message online, other than in the source on github. I'm not sure if this is an issue with Postgres or Packetbeat. This is a new error since upgrading Packetbeat from 1.1.2, however the previous version was giving me "Slice bounds out of range" errors as per this thread.

Config is as follows:

interfaces:
  device: any
protocols:
  pgsql:
    ports: [5432]
output:
  logstash:
    hosts: ["ingress1:10514", "ingress2:10514"]
    tls:
      certificate_authorities: ["/etc/ssl/certs/beats-ca.pem"]
      insecure: false

System is Ubuntu 12.04.5
Packetbeat 5.0.0~alpha1
Postgres 9.2.15-1.pgdg12.4+1

2

the parser has been hardened not to generate the 'Slice bounds out of range' panic. You're likely experiencing packet loss? On packet-loss (or on startup), the parser/analyzer must try to 'sync' with the TCP stream. It will fail parsing until it manages to find a valid pgsql messages indicating parser to be in sync with actual TCP stream. Problem with packet loss is, the parser is out of sync again and must resync.

3

Hi @steffens, I'm assuming this packet loss at the client end. It's pretty unlikely the issue is with the server, and our monitoring shows nothing. Is there anything I can do with packetbeat to identify the actual cause here?

I sent you a trace file back in January to help with investigating the "slice bounds out of range issue", and this is the same PGSQL server from that issue. Do you know if the issue identified there was also packet loss?

4

yes, the stack trace was due to packetloss. packetbeat is a passive network sniffer bypassing some of the network stack. Even if your servers do not experience packet loss, packetbeat still might not see all packets.