# Postgresql Module in Filebeat Not Getting Start **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 9, 2018, 11:01am UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693 "2018-01-09T11:01:12Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 9, 2018, 11:01am UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/1 "2018-01-09T11:01:12Z")
Hi Team, I am trying to integrate postgresql logs in Kibana using postgres module. Here is my filebeat.yml ```auto ######################## Filebeat Configuration ############################ #========================== Modules configuration ============================ filebeat.modules: #----------------------------- PostgreSQL Module ----------------------------- - module: postgresql # Logs log: enabled: true # Set custom paths for the log files. If left empty, # Filebeat will choose the paths depending on your OS. var.paths: /home/test.log # Prospector configuration (advanced). Any prospector configuration option # can be added under this section. #prospector: #================================ Outputs ====================================== # Configure what output to use when sending the data collected by the beat. #-------------------------- Elasticsearch output ------------------------------- output.elasticsearch: # Boolean flag to enable or disable the output module. enabled: true hosts: ["ip:port"] # Optional index name. The default is "filebeat" plus date # and generates [filebeat-]YYYY.MM.DD keys. # In case you modify this pattern you must update setup.template.name and setup.template.pattern accordingly. #index: "postgresqllog-%{+yyyy.MM.dd}" #============================== Dashboards ===================================== # These settings control loading the sample dashboards to the Kibana index. Loading # the dashboards are disabled by default and can be enabled either by setting the # options here, or by using the `-setup` CLI flag or the `setup` command. setup.dashboards.enabled: true #============================== Kibana ===================================== # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API. # This requires a Kibana endpoint configuration. setup.kibana: # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 host: "ip:port" ``` When i am trying to setup filebeat using docker then following error comes up, CRIT Exiting: Error getting config for fielset postgresql/log: Error interpreting the template of the prospector: template: text:3:22: executing "text" at \<.paths\>: range can't iterate over /home/test.log Exiting: Error getting config for fielset postgresql/log: Error interpreting the template of the prospector: template: text:3:22: executing "text" at \<.paths\>: range can't iterate over /home/test.log ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 9, 2018, 2:51pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/2 "2018-01-09T14:51:06Z")
I think the message indicates `paths` is supposed to be a list of paths. Try: ```auto - module: postgresql # Logs log: enabled: true # Set custom paths for the log files. If left empty, # Filebeat will choose the paths depending on your OS. var.paths: - '/home/test.log' ``` ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 9, 2018, 3:22pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/3 "2018-01-09T15:22:43Z")
Hi @steffens I tried this one also, no success. Git the same error message. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 10, 2018, 12:01pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/4 "2018-01-10T12:01:33Z")
The error occurs when expanding the [filebeat configuration template](https://github.com/elastic/beats/blob/master/filebeat/module/postgresql/log/config/log.yml#L3). This sample script passes a string instead of a list and fails with the same error: [https://play.golang.org/p/sSwki4hdhKQ](https://play.golang.org/p/sSwki4hdhKQ) Updating the environments 'paths' variable to be a list resolves the problem: [https://play.golang.org/p/iAJ6QJqb8bO](https://play.golang.org/p/iAJ6QJqb8bO) How are you starting filebeat? Are you sure you use the correct config file + have correct indentation + used the `-` character to start a list? If problem persists, please share output of: `./filebeat -e -v -d '*,config' -c ` . ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 10, 2018, 12:13pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/5 "2018-01-10T12:13:04Z")
@steffens I am running filebeat in docker , using following command. docker run -v /home/filebeat.yml:/usr/share/filebeat/filebeat.yml [docker.elastic.co/beats/filebeat:6.1.1](http://docker.elastic.co/beats/filebeat:6.1.1) My console logs show the error 2018/01/10 12:11:29.461369 beat.go:203: INFO Setup Beat: filebeat; Version: 6.1.1 2018/01/10 12:11:29.461528 client.go:123: INFO Elasticsearch url: [http://ip](http://ip):port 2018/01/10 12:11:29.463108 module.go:76: INFO Beat name: e3affd7e454c 2018/01/10 12:11:29.482046 filebeat.go:62: INFO Enabled modules/filesets: postgresql (log), () 2018/01/10 12:11:29.482692 beat.go:635: CRIT Exiting: Error getting config for fielset postgresql/log: Error interpreting the template of the prospector: template: text:3:22: executing "text" at \<.paths\>: range can't iterate over /home/test.log Exiting: Error getting config for fielset postgresql/log: Error interpreting the template of the prospector: template: text:3:22: executing "text" at \<.paths\>: range can't iterate over /home/test.log Also i have already shared my filebeat.yml with you. I tried using var.paths: - "/home/test.log" But getting the above error. Can you help me with what I am doing wrong? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 10, 2018, 12:37pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/6 "2018-01-10T12:37:50Z")
please properly format you output and configs using the ``-Button. The config file is sensitive to proper formatting (use spaces, no tabs). Please the me the output as requested by at adding the run flag `-d '*,config'`. ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 10, 2018, 1:27pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/7 "2018-01-10T13:27:37Z")
@steffens I was missing the braces part in specify the log file path. It worked for me when i changed to: var.paths: ["/home/Postgres\_vv-qa-pgdb/\*.log"] Thanks. 🙂 But now i am getting another error, My filebeat is getting started but it is not picking up my logs , though there are several log files at specified location. Here are the logs for the same: 2018/01/10 13:22:21.326242 metrics.go:23: INFO Metrics logging every 30s 2018/01/10 13:22:21.326387 beat.go:443: INFO Beat UUID: 893b3c74-2102-4990-ba12-bdf6d1816d7b 2018/01/10 13:22:21.326397 beat.go:203: INFO Setup Beat: filebeat; Version: 6.1.1 2018/01/10 13:22:21.326539 client.go:123: INFO Elasticsearch url: [http://10.3.100.210:32328](http://10.3.100.210:32328) 2018/01/10 13:22:21.327001 module.go:76: INFO Beat name: d9d69fba8427 2018/01/10 13:22:21.327886 filebeat.go:62: INFO Enabled modules/filesets: postgresql (log), () 2018/01/10 13:22:21.328210 client.go:123: INFO Elasticsearch url: [http://10.3.100.210:32328](http://10.3.100.210:32328) 2018/01/10 13:22:21.909385 client.go:651: INFO Connected to Elasticsearch version 6.0.0 2018/01/10 13:22:21.909555 client.go:69: INFO Kibana url: [http://10.3.100.210:32454](http://10.3.100.210:32454) 2018/01/10 13:22:48.442468 beat.go:551: INFO Kibana dashboards successfully loaded. 2018/01/10 13:22:48.442527 beat.go:276: INFO filebeat start running. 2018/01/10 13:22:48.442713 registrar.go:71: INFO No registry file found under: /usr/share/filebeat/data/registry. Creating a new registry file. 2018/01/10 13:22:48.659456 registrar.go:108: INFO Loading registrar data from /usr/share/filebeat/data/registry 2018/01/10 13:22:48.659509 registrar.go:119: INFO States Loaded from registrar: 0 2018/01/10 13:22:48.659529 crawler.go:48: INFO Loading Prospectors: 1 2018/01/10 13:22:48.659581 registrar.go:150: INFO Starting Registrar 2018/01/10 13:22:48.659890 prospector.go:87: INFO Starting prospector of type: log; ID: 17973440715270663115 2018/01/10 13:22:48.659920 crawler.go:82: INFO Loading and starting Prospectors completed. Enabled prospectors: 1 2018/01/10 13:22:51.327000 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30001 beat.memstats.gc\_next=4194304 beat.memstats.memory\_alloc=2287344 beat.memstats.memory\_total=8811168 filebeat.harvester.open\_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.output.type=elasticsearch libbeat.pipeline.clients=1 libbeat.pipeline.events.active=0 registrar.states.current=0 registrar.writes=1 2018/01/10 13:23:21.326620 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30000 beat.memstats.gc\_next=4194304 beat.memstats.memory\_alloc=2302880 beat.memstats.memory\_total=8826704 filebeat.harvester.open\_files=0 filebeat.harvester.running=0 libbeat.config.module.running=0 libbeat.pipeline.clients=1 libbeat.pipeline.events.active=0 registrar.states.current=0 ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 11, 2018, 11:12am UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/8 "2018-01-11T11:12:06Z")
@steffens Any help on this from your side will be very helpful for me. ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [January 11, 2018, 11:56am UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/9 "2018-01-11T11:56:42Z")
If you add `-d "porspector"` to the filebeat CLI, it will print a `File Configs:` that prints the globs that Filebeat uses. Double check that the files are at the right location. Also, you might want to delete the registry file (on a tar.gz installation, it's `data/registry`) to force re-reading the log files in case they were already read. ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [January 11, 2018, 11:58am UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/10 "2018-01-11T11:58:26Z")
Ah, now I see that you are running FIlebeat through docker. In that case, did you mount the folder where the PostgreSQL log files are in the docker container? ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 11, 2018, 12:34pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/11 "2018-01-11T12:34:07Z")
@tudor Yes i am running Filebeat through docker, and has mounted the folder inside docker container. This is the command i am using to run filebeat through docker docker run -v /$path/filebeat.yml:/usr/share/filebeat/filebeat.yml -v /home/Postgres\_vv-qa-pgdb/\*.log:/home [docker.elastic.co/beats/filebeat:6.1.1](http://docker.elastic.co/beats/filebeat:6.1.1) ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 11, 2018, 12:34pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/12 "2018-01-11T12:34:22Z")
Is there something i am missing in this? In my filebeat.yml, i have given the var.paths: ["/home/\*.log"] It is the path which i have mounted inside the docker container. ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [January 11, 2018, 5:47pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/13 "2018-01-11T17:47:38Z")
I'd recommend starting an interactive container with something like this: ```auto docker run -ti -v /$path/filebeat.yml:/usr/share/filebeat/filebeat.yml -v /home/Postgres_vv-qa-pgdb/*.log:/home docker.elastic.co/beats/filebeat:6.1.1 bash ``` Then check that all paths are in the expected places. Then from the same interactive session start `filebeat -e -d "prospector"` to debug if Filebeat opens the right paths. ---
**Author:** ![manya12](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@manya12](https://discuss.elastic.co/u/manya12)\ **Post date:** [January 16, 2018, 2:11pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/14 "2018-01-16T14:11:37Z")
@tudor Yes, it worked for me. But the postgres module is not able to parse my logs, it gives me an error that grok debugger is failed to parse logs. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 19, 2018, 3:49pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/15 "2018-01-19T15:49:55Z")
Can you share the error and sample logs? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 16, 2018, 3:50pm UTC](https://discuss.elastic.co/t/postgresql-module-in-filebeat-not-getting-start/114693/16 "2018-02-16T15:50:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.