# Filebeat Module Postgresql **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [April 26, 2023, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860 "2023-04-26T14:35:32Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Giancarlo\_Huapaya\_Ra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giancarlo_huapaya_ra/32/120262_2.png) [@Giancarlo\_Huapaya\_Ra](https://discuss.elastic.co/u/Giancarlo_Huapaya_Ra)\ **Post date:** [April 26, 2023, 2:35pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/1 "2023-04-26T14:35:32Z")
Hello everyone, Please I need your help, I have problems with the Postgresql module filbeat, at the time of viewing the log I see that I get the following error message: [2023-04-26 09:20:02.534 -05 [2828024] u\_sistema@scp LOG: statement: insert into monitor.log(host,swap,menlibre,cargacpu,disco90,disco95,ip10,aswap,amen,uptime,llast)values('dnsserv2','36','86','0,01,','-','-','10. 1.1.96','974','976','up 18 weeks, 3 days, 11 hours, 58 minutes','reboot system boot 5.4.0-105-generi Sat Dec 17 21:21 still running')]] I have already checked all my configuration of both filebeat and Postgresql module is in "true". ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/8/e8fe615744391e4fff5ace8507632814fca8c4f4.png) If you could help me it would be of great help. Thanks. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 26, 2023, 4:58pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/2 "2023-04-26T16:58:45Z")
hi @Giancarlo_Huapaya_Ra Welcome to the community! What version of the Stack are you on... and did you run `filebeat setup -e` _ **Before** _ you started to ingest the logs? And can you share a few lines from the raw postgres logs please. Quick look ... it looks like your log lines are not the postgres standard logs and that they have been customized in some way. ---
**Author:** ![Giancarlo\_Huapaya\_Ra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giancarlo_huapaya_ra/32/120262_2.png) [@Giancarlo\_Huapaya\_Ra](https://discuss.elastic.co/u/Giancarlo_Huapaya_Ra)\ **Post date:** [April 26, 2023, 5:13pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/3 "2023-04-26T17:13:19Z")
> [@stephenb](#): > > hi @Giancarlo_Huapaya_Ra Welcome to the community! > > What version of the Stack are you on... > > and did you run > > `filebeat setup -e` > > _ **Before** _ you started to ingest the logs? > > And can you share a few lines from the raw postgres logs please. Hello, I am using version 8.7.3 If I run filebeat setup -e The log lines are as follows, I get error message with GROK ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/0/10ef08d8e94e550ef82548b5dc6c1b7961952444.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 26, 2023, 5:45pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/4 "2023-04-26T17:45:51Z")
I don't mean the logs inside elastic ... I want to see some of the raw lines from the postgres.log itself Also per the [docs](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-postgresql.html#_supported_log_formats) > The `postgresql` module using `.log` was tested with logs from versions 9.5 on Ubuntu, 9.6 on Debian, and finally 10.11, 11.4 and 12.2 on Arch Linux 9.3. My quick analysis is that your raw postgress logs do not adhere to the supported postgres log format. Please take a look at this [supported log formats](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-postgresql.html#_supported_log_formats) That not to say we can not fix this ... but that is why it is not working it looks to me like you logs are of the format which is not standard... one part that appears to be not standard is the `-05` timezone which looks like a non standard pattern. ```auto 2023-04-26 09:20:02.534 -05 [2828024] u_sistema@scp LOG: statement: insert into monitor.log(host,swap,menlibre,cargacpu,disco90,disco95,ip10,aswap,amen,uptime,llast)values('dnsserv2','36','86','0,01,','-','-','10. 1.1.96','974','976','up 18 weeks, 3 days, 11 hours, 58 minutes','reboot system boot 5.4.0-105-generi Sat Dec 17 21:21 still running') ``` So first can you check what version ... and 2nd I suspect there has been some customizations to the postgressql log format and 3rd can you changes to a standard format... If not any of those we will need to create a custom version of the parsing pipeline.... Which is not too bad but it is something you will need to maintain and the timezone may not be correct Also _ **please** _ do NOT paste images of text, that is not helpful at all ... please past text and format it with the `` button or just put ````` 3 back ticks the line before and after the text ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 26, 2023, 7:16pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/5 "2023-04-26T19:16:07Z")
BTW I just stood up a quick postgres with docker no changes and ran it created a few tables etc. ```auto 2023-04-26 19:01:02.525 UTC [1] LOG: starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit ``` ![Screenshot 2023-04-26 at 12.14.35 PM](https://us1.discourse-cdn.com/elastic/original/3X/d/c/dc319c20e77269a8458828766698b04ba293b2c6.jpeg) ---
**Author:** ![Giancarlo\_Huapaya\_Ra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giancarlo_huapaya_ra/32/120262_2.png) [@Giancarlo\_Huapaya\_Ra](https://discuss.elastic.co/u/Giancarlo_Huapaya_Ra)\ **Post date:** [April 26, 2023, 7:29pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/6 "2023-04-26T19:29:59Z")
Thanks for the information provided I will do what you tell me, I just have one more question, in the doc you shared I see that make changes to postgresql.conf, I have to make these changes or I can leave it by default? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 26, 2023, 7:42pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/7 "2023-04-26T19:42:52Z")
I am not a postgress guru... The docs say > Log lines should be preffixed with the timestamp in milliseconds, the process id, the user id and the database name. This uses to be the default in most distributions, and is translated to this setting in the configuration file: I suspect it should work with the defaults... you will need to test. BUT if you want more advanced like duration [follow the docs](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-postgresql.html#_supported_log_formats) > PostgreSQL server can be configured to log statements and their durations and this module is able to collect this information. To be able to correlate each duration with their statements, they must be logged in the same line. This happens when the following options are used: > log\_duration = 'on' > log\_statement = 'none' > log\_min\_duration\_statement = 0 > Setting a zero value in `log_min_duration_statement` will log all statements executed by a client. You probably want to configure it to a higher value, so it logs only slower statements. This value is configured in milliseconds. > When using `log_statement` and `log_duration` together, statements and durations are logged in different lines, and Filebeat is not able to correlate both values, for this reason it is recommended to disable `log_statement`. ---
**Author:** ![Giancarlo\_Huapaya\_Ra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giancarlo_huapaya_ra/32/120262_2.png) [@Giancarlo\_Huapaya\_Ra](https://discuss.elastic.co/u/Giancarlo_Huapaya_Ra)\ **Post date:** [April 26, 2023, 8:00pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/8 "2023-04-26T20:00:14Z")
Thanks for the support, and I will review the docs sent. ---
**Author:** ![Giancarlo\_Huapaya\_Ra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/giancarlo_huapaya_ra/32/120262_2.png) [@Giancarlo\_Huapaya\_Ra](https://discuss.elastic.co/u/Giancarlo_Huapaya_Ra)\ **Post date:** [April 27, 2023, 5:04pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/9 "2023-04-27T17:04:25Z")
Hello again, To update my case which has already been solved thanks to your comments. As an addition the change I applied was the following: log\_timezone = 'Etc/UTC'. Before it was as America-Latin. Again thank you very much for your support. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 25, 2023, 7:05pm UTC](https://discuss.elastic.co/t/filebeat-module-postgresql/330860/10 "2023-05-25T19:05:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.