# 8.7.1: Stand Alone Kubernetes Deployment - filestream input with ID '' already exists

**URL:** https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148
**Category:** Elastic Agent
**Created:** [May 11, 2023, 4:55am UTC](https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148 "2023-05-11T04:55:19Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![berg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/berg/32/14962_2.png) [@berg](https://discuss.elastic.co/u/berg)
#### Post date: [May 11, 2023, 4:55am UTC](https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148/1 "2023-05-11T04:55:19Z")

</div>

I've used the [example K8s manifest](https://www.elastic.co/guide/en/fleet/master/running-on-kubernetes-standalone.html) to deploy Elastic Agent on our AWS EKS cluster. I followed the documentation in the EKS section to comment out modules that are unavailable in AWS EKS.

Recently, I upgraded to 8.7.0 and noticed a huge amount of log volume increase. A large portion of this was coming from Elastic Agent logs itself. Some of this was resolved in 8.7.1 due to issues with the logging, and since then setting the logging level to warning has reduced my events per minute from about 100k to about 2k. Most of the remaining 2k Elastic Agent logs are the following:

`filestream input with ID '' already exists, this will lead to data duplication, please use a different ID`

I added IDs to every data\_stream in my configmap, and discovered it's related to:

```yaml
- type: filestream
        id: container-log-${kubernetes.pod.name}-${kubernetes.container.id}
        use_output: default
        meta:
          package:
            name: kubernetes
            version: 1.29.2
        data_stream:
          namespace: default
        streams:
          - data_stream:
              dataset: kubernetes.container_logs
              type: logs
            prospector.scanner.symlinks: true
            parsers:
              - container: ~
              # - ndjson:
              # target: json
              # - multiline:
              # type: pattern
              # pattern: '^\['
              # negate: true
              # match: after
            paths:
              - /var/log/containers/*${kubernetes.container.id}.log
            processors:
              - add_fields:
                  target: orchestrator.cluster
                  fields:
                    name: "our-cluster-name"
                    url: "https://our-cluser.uri.aws.com"

```

In combination with this, I have also been receiving the warning:

`DEPRECATED: Log input. Use Filestream input instead.`

Is there a working `filestream` configuration that could be used for a Standalone Deployment that doesn't result large volumes of the warnings described above?

---

<div class="post-metadata">

### Author: ![berg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/berg/32/14962_2.png) [@berg](https://discuss.elastic.co/u/berg)
#### Post date: [May 22, 2023, 11:33pm UTC](https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148/2 "2023-05-22T23:33:29Z")

</div>

Raised as an issue on the GitHub page for diagnostics [#2701](https://github.com/elastic/elastic-agent/issues/2701)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 19, 2023, 11:34pm UTC](https://discuss.elastic.co/t/8-7-1-stand-alone-kubernetes-deployment-filestream-input-with-id-already-exists/333148/3 "2023-06-19T23:34:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
