# Not setting the elasticsearchRef for setting beat output

**URL:** <https://discuss.elastic.co/t/not-setting-the-elasticsearchref-for-setting-beat-output/340878>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 16, 2023, 7:44am UTC](https://discuss.elastic.co/t/not-setting-the-elasticsearchref-for-setting-beat-output/340878 "2023-08-16T07:44:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexns](https://avatars.discourse-cdn.com/v4/letter/a/779978/32.png) [@alexns](https://discuss.elastic.co/u/alexns)\
**Post date:** [August 16, 2023, 7:44am UTC](https://discuss.elastic.co/t/not-setting-the-elasticsearchref-for-setting-beat-output/340878/1 "2023-08-16T07:44:42Z")

</div>

Hello,

We have a k8s cluster dedicated running monitoring software. Elastic Search is installed here using the ECK operator and the CRD's.  
On another cluster, we have filebeat running using the deprecated helm charts. So Elastic Search is not running in the same cluster. Using the filebeat config `output.elasticsearch` we have it configured to send data to Elastic Search running in the external cluster.

We are now trying to move to the newer Helm charts, using the ECK operator and CRDs. We run into problems setting up filebeat with output to our external Elastic Search.  
When reading the documentation, I stumbled upon [Configuration | Elastic Cloud on Kubernetes [2.9] | Elastic](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-beat-configuration.html#k8s-beat-set-beat-output)  
" Output can be set to any value that is supported by a given Beat. To use it, remove the `elasticsearchRef` element from the specification and include an appropriate output configuration in the `config` or `configRef` elements."

The problem is, this does not work as expected. We have our config as follows:

```auto
    config:
      filebeat.inputs:
      - type: container
        paths:
          - /var/log/containers/*.log
        processors:
        - add_labels:
            labels:
              group: customer
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
      logging.level: error
      output.elasticsearch:
        hosts: ["https://es.domain.tld:443"]
        api_key: ${ELASTICSEARCH_API_KEY}
        pipeline: filebeat-mask

```

And we did not add any `elasticsearchRef`. Still, the operator adds a default `elasticsearchRef`:

```auto
elasticsearchRef:
    name: elasticsearch

```

This results in an error `Association backend for elasticsearch is not configured` and the DaemonSet is not created by the operator. If I understand the documentation, I did not expect the `elasticsearchRef` to be added by the operator.

The only way we are able to work around this, is by explicitly setting the ref to an empty value in the spec:

```auto
    elasticsearchRef:
        name: ''

```

But this does not feel right. Are we missing something in our config? Or is this the right way? This is our full spec for the filebeat Helm chart:

```auto
  fullname: filebeat
  spec:
    elasticsearchRef:
        name: ''
    type: filebeat
    daemonSet:
      podTemplate:
        spec:
          serviceAccount: elastic-beat-filebeat
          automountServiceAccountToken: true
          containers:
          - name: filebeat
            env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
    secureSettings:
    - secretName: beats-secret
    config:
      filebeat.inputs:
      - type: container
        paths:
          - /var/log/containers/*.log
        processors:
        - add_labels:
            labels:
              group: customer
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
      logging.level: error
      output.elasticsearch:
        hosts: ["https://es.domain.tld:443"]
        api_key: ${ELASTICSEARCH_API_KEY}
        pipeline: filebeat-mask

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 13, 2023, 9:45am UTC](https://discuss.elastic.co/t/not-setting-the-elasticsearchref-for-setting-beat-output/340878/2 "2023-09-13T09:45:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.