Remote Filebeat cannot ship logs to elastic ERROR [publisher_pipeline_output]

Elastic Stack Beats
1

I have been trying to ship logs from a remote server to my elk. Below is my conf file. I tested my conf file using filebeat -c filebeat.yml test output everything seems fine. Could you please help. Thank you

        elasticsearch: [<elastic_host>:9200](http://<elastic_host>/)...
        parse url... OK
        connection...
        parse host... OK
        dns lookup... OK
        addresses: <elastic_host>
        dial up... OK
        TLS... WARN secure connection disabled
        talk to server... OK
        version: 7.8.1

Below is the error in the filebeat logs

```220-09-16T17:31:28.991-0700	ERROR	[publisher_pipeline_output]	pipeline/output.go:154	Failed to connect to backoff(elasticsearch(http://10.176.140.123:9200)): Connection marked as failed because the onConnect callback failed: 1 error: Error loading pipeline for fileset system/auth: couldn't load pipeline: couldn't load json. Error: 400 Bad Request: {"error":{"root_cause":[{"type":"parse_exception","reason":"processor [set] doesn't support one or more provided configuration parameters [ignore_empty_value]","processor_type":"set"}],"type":"parse_exception","reason":"processor [set] doesn't support one or more provided configuration parameters [ignore_empty_value]","processor_type":"set"},"status":400}. Response body: {"error":{"root_cause":[{"type":"parse_exception","reason":"processor [set] doesn't support one or more provided configuration parameters [ignore_empty_value]","processor_type":"set"}],"type":"parse_exception","reason":"processor [set] doesn't support one or more provided configuration parameters [ignore_empty_value]","processor_type":"set"},"status":400} ```

My filebeat config file is:

                filebeat:
                  config:
                    modules:
                      path: /etc/filebeat/modules.d/*.yml
                      reload:
                        enabled: false
                  inputs:
                  - enabled: true
                    paths:
                    - /var/log/st2/st2actionrunner.gelf.log
                    - /var/log/st2/st2sensorcontainer.gelf.log
                    type: log
                logging:
                  files:
                    keepfiles: 7
                    name: filebeat
                    path: /var/log/filebeat
                    permissions: 420
                  level: warning
                  selectors:
                  - '*'
                  to_files: true
                output:
                  elasticsearch:
                    hosts:
                    - <elastic_ip_edited_for_confidentiality>:9200
                    index: ss-logs-%{[agent.version]}-%{+yyyy.MM.dd}
                path:
                  config: /etc/filebeat
                  data: /var/lib/filebeat
                  home: /usr/share/filebeat
                  logs: /var/log/filebeat
                processors:
                - add_host_metadata: null
                - add_cloud_metadata: null
                - add_docker_metadata: null
                - add_kubernetes_metadata: null
                - decode_json_fields:
                    fields:
                    - message
                setup:
                  ilm:
                    enabled: false
                  kibana: null
                  template:
                    fields: fields.yml
                    name: ss-logs
                    pattern: ss-logs-*
                    settings:
                      index:
                        number_of_shards: 1
2

Welcome to our community! :smiley: FYI it's not the best idea to just post an error and a config file without explaining what you are looking for.

However; Can you connect to your Elasticsearch instance?

3

Thank you @warkolm. Edited it. Yes,
I can connect to my Elastic instance

4

You'll need Elasticsearch 7.9 to use this Filebeat 7.9 module. When Filebeat 7.9.2 comes out this requirement to have ES 7.9 will be relaxed. See [Filebeat] backwards compatibility for set processor by leehinman · Pull Request #20908 · elastic/beats · GitHub.

5

So downgrading to 7.8 should help right?

6

Yes, using Filebeat 7.8 until the new release comes out should work as those Ingest Node pipelines are not using the ignore_empty_value option in the set processor that was introduced in Elasticsearch 7.9.

Or you could update your Elasticsearch cluster to 7.9.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.