# Filebeat raise error:filestream input id already exist
**URL:** https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241
**Category:** Beats
**Tags:** filebeat, elastic-agent
**Created:** [May 20, 2022, 3:01am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241 "2022-05-20T03:01:10Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@duduniao](https://discuss.elastic.co/u/duduniao)
#### Post date: [May 20, 2022, 3:01am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/1 "2022-05-20T03:01:10Z")
I install filebeat-8.1.2 on centos 7, then use filestream input to read log files:
```nohighlight
- type: filestream
id: ro-server-run-log
paths: ['/data/ro_jplive/log/*-*-*-????-??-??-??.log']
prospector.scanner.exclude_files: ['/data/ro_jplive/log/commonlib.*', '/data/ro_jplive/log/.+-perf.+']
ignore_older: 10m
close.on_state_change.inactive: 2m
clean_inactive: 15m
index: ro-server
```
when I star filebeat.service , beat log raise error log:
```auto
{"log.level":"error","@timestamp":"2022-05-20T10:44:48.068+0800","log.logger":"input","log.origin":{"file.name":"input-logfile/manager.go","file.line":181},"message":"filestream input with ID 'ro-server-run-log' already exists, this will lead to data duplication, please use a different ID","service.name":"filebeat","ecs.version":"1.6.0"}
```
I didn't find a solution to this problem
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [May 20, 2022, 5:27am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/2 "2022-05-20T05:27:18Z")
> [@duduniao](#):
>
> ID 'ro-server-run-log' already exists
You must have unique IDs. ro-server-run-log exists somewhere else, change it.
---
### Author:  [@duduniao](https://discuss.elastic.co/u/duduniao)
#### Post date: [May 20, 2022, 6:16am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/3 "2022-05-20T06:16:07Z")
I have only one configuration file, named ro-server.yml, and there is only one input configuration in it
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [May 20, 2022, 6:18am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/4 "2022-05-20T06:18:10Z")
If is possible rename it temporarily.
---
### Author:  [@duduniao](https://discuss.elastic.co/u/duduniao)
#### Post date: [May 20, 2022, 6:27am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/5 "2022-05-20T06:27:49Z")
> [@Rios](#):
>
> y.
My filebeat.yml is as follows:
```yaml
name: "log-center-filebeat"
max_procs: 2
fields:
owner:
vendor: "huanle"
region: "shanghai"
zone: shanghai-a
project: "ro"
app: "log-center"
os:
private_ip: 10.4.7.57
public_ip: 118.39.23.25
kernel: "3.10.0-1160.45.1.el7.x86_64"
architecture: "x86_64"
distribution: "CentOS"
version: "7.9"
fields_under_root: true
filebeat.shutdown_timeout: 5s
logging:
level: error
to_files: true
files:
path: /opt/logs/log-center-filebeat
name: filebeat.log
keepfiles: 5
rotateonstartup: false
permissions: 0644
filebeat.config.inputs:
enabled: true
path: inputs.d/*.yml
setup:
ilm.enabled: false
template:
name: system-template
pattern: "system*"
output.console:
pretty: true
enabled: False
output.file:
enabled: True
path: /opt/logs/log-center-filebeat
filename: filebeat.output
rotate_every_kb: 40960
permissions: 0644
```
The inputs.d directory has only one config:
```auto
[root@centos-7-57 log-center-filebeat]# tree inputs.d/
inputs.d/
└── ro-server.yml
0 directories, 1 file
```
The ro-server.yml :
```yaml
- type: filestream
id: "ro-server-run-log"
paths: ['/data/ro_jplive/log/*-*-*-????-??-??-??.log']
prospector.scanner.exclude_files: ['/data/ro_jplive/log/commonlib.*', '/data/ro_jplive/log/.+-perf.+']
ignore_older: 10m
close.on_state_change.inactive: 2m
clean_inactive: 15m
index: ro-server
parsers:
- multiline:
type: pattern
pattern: '^\['
negate: true
match: after
processors:
- dissect:
tokenizer: '[%{level}] %{time},%{millisecond} %{service} - %{function}(%{code}) %{msg}'
field: "message"
- dissect:
tokenizer: '/data/ro_jplive/log/%{instance_name}-%{server_id}-%{instance_id}-%{date_str}.log'
field: "log.file.path"
- dissect:
tokenizer: '/data/ro_jplive/log/%{log_file}'
field: "log.file.path"
- timestamp:
field: "dissect.time"
timezone: "Local"
layouts:
- "2006-01-02 15:04:05"
when:
has_fields: "dissect.time"
- drop_fields:
fields: ["log", "dissect.millisecond", "dissect.date_str", "dissect.time", "ecs"]
ignore_missing: true
```
I remove filebeat data directory ,then start filebeat.server . The log shows the following error:
```json
{"log.level":"error","@timestamp":"2022-05-20T14:18:47.953+0800","log.logger":"input","log.origin":{"file.name":"input-logfile/manager.go","file.line":181},"message":"filestream input with ID 'ro-server-run-log' already exists, this will lead to data duplication, please use a different ID","service.name":"filebeat","ecs.version":"1.6.0"}
```
Is there something wrong with my configuration?
---
### Author:  [@duduniao](https://discuss.elastic.co/u/duduniao)
#### Post date: [May 20, 2022, 7:08am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/6 "2022-05-20T07:08:37Z")
filebeat v8.1.2 & v8.1.3 filestream raise error. Same configuration, v8.1.1 & v8.1.0 no errors. Maybe a bug?
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [May 20, 2022, 7:50am UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/7 "2022-05-20T07:50:24Z")
Try to set _clean\_removed: false_
[link](https://github.com/elastic/beats/issues/31239)
---
### Author:  [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [May 23, 2022, 2:55pm UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/8 "2022-05-23T14:55:22Z")
I do not see anything wrong with you configuration. It seems that the logging of the filestream is a bit too dramatic. What you see is not an error, but rather a warning if you configured the same ID for different inputs. We should definitely reword this.
---
### Author:  [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [May 26, 2022, 4:15pm UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/9 "2022-05-26T16:15:01Z")
This is a bug, unfortunately. I've opened an issue: [`filestream` input returns error when an existing input is reloaded with the same ID · Issue #31767 · elastic/beats · GitHub](https://github.com/elastic/beats/issues/31767)
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 23, 2022, 6:15pm UTC](https://discuss.elastic.co/t/filebeat-raise-error-filestream-input-id-already-exist/305241/10 "2022-06-23T18:15:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.