# Filebeat output to existing index

**URL:** <https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 15, 2021, 7:49pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329 "2021-09-15T19:49:38Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![caraboy](https://avatars.discourse-cdn.com/v4/letter/c/b2d939/32.png) [@caraboy](https://discuss.elastic.co/u/caraboy)\
**Post date:** [September 15, 2021, 7:49pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/1 "2021-09-15T19:49:38Z")

</div>

Hi everyone,  
I'm trying to configure a filebeat output to an existing index but apparently i can't correctly set the template name and pattern.

```auto
output.elasticsearch.index: "sai5x"
setup.template.name: "sai5x"
setup.template.pattern: "sai5x"

```

DevOps created the index for me, and they seem not have used any template, just reindexed documents form an older ES, cause if i `GET /sai5x` no template is mentioned, and i know they used dynamic mapping.  
I've tried creating de index\_template i reference in `setup.template.name` via:

```auto
PUT /_index_template/sai5x
{
some_config...
}

```

But files still get indexed in `filebeat-7.12.0`.  
What am I doing wrong?? Is there a default template used by elastic in these cases?  
Thank you in advance.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 15, 2021, 10:01pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/2 "2021-09-15T22:01:24Z")

</div>

It'd be useful if you shared your full Filebeat config.

> [@caraboy](#):
>
> apparently i can't correctly set the template name and pattern.

Why, what happens when you do this?

---

<div class="post-metadata">

**Author:** ![caraboy](https://avatars.discourse-cdn.com/v4/letter/c/b2d939/32.png) [@caraboy](https://discuss.elastic.co/u/caraboy)\
**Post date:** [September 16, 2021, 11:50am UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/3 "2021-09-16T11:50:46Z")

</div>

Nothing happens. I expressed my self wrong, it acknowledge: true, but i can't get that template applied to my existing index, cause files keep on going to the default index filebeat created.

---

<div class="post-metadata">

**Author:** ![caraboy](https://avatars.discourse-cdn.com/v4/letter/c/b2d939/32.png) [@caraboy](https://discuss.elastic.co/u/caraboy)\
**Post date:** [September 16, 2021, 12:20pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/4 "2021-09-16T12:20:39Z")

</div>

Logs show index it's not being set correctly, but don't know why:

```auto
2021-09-16T12:05:38.516Z INFO instance/beat.go:660 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2021-09-16T12:05:38.525Z INFO instance/beat.go:668 Beat ID: e3b0f4f4-f4a8-4065-9bd4-1c8e0613698e
2021-09-16T12:05:38.526Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed
2021-09-16T12:05:38.526Z INFO [beat] instance/beat.go:996 Beat info {"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "e3b0f4f4-f4a8-4065-9bd4-1c8e0613698e"}}}
2021-09-16T12:05:38.526Z INFO [beat] instance/beat.go:1005 Build info {"system_info": {"build": {"commit": "08e20483a651ea5ad60115f68ff0e53e6360573a", "libbeat": "7.12.0", "time": "2021-03-18T06:16:51.000Z", "version": "7.12.0"}}}
2021-09-16T12:05:38.526Z INFO [beat] instance/beat.go:1008 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.15.8"}}}
2021-09-16T12:05:38.526Z INFO [beat] instance/beat.go:1012 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-09-16T11:13:58Z","containerized":true,"name":"0ee7a6d770a0","ip":["127.0.0.1/8","172.20.0.2/16"],"kernel_version":"5.4.0-84-generic","mac":["02:42:ac:14:00:02"],"os":{"type":"linux","family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0,"id":"44ccc339d95bd51386fcfc5d8f041927"}}}
2021-09-16T12:05:38.526Z INFO [beat] instance/beat.go:1041 Process info {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 8, "ppid": 1, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2021-09-16T12:05:36.760Z"}}}
2021-09-16T12:05:38.527Z INFO instance/beat.go:304 Setup Beat: filebeat; Version: 7.12.0
2021-09-16T12:05:38.536Z INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'filebeat-7.12.0' as ILM is enabled.
2021-09-16T12:05:38.536Z INFO eslegclient/connection.go:99 elasticsearch url: http://xxx.xxx.xxx.xxx:9200
2021-09-16T12:05:38.537Z INFO [publisher] pipeline/module.go:113 Beat name: filebeat:prueba
2021-09-16T12:05:38.537Z INFO [monitoring] log/log.go:117 Starting metrics logging every 30s
2021-09-16T12:05:38.537Z INFO instance/beat.go:468 filebeat start running.
2021-09-16T12:05:38.538Z INFO memlog/store.go:119 Loading data file of '/usr/share/filebeat/data/registry/filebeat' succeeded. Active transaction id=0
2021-09-16T12:05:38.538Z INFO memlog/store.go:124 Finished loading transaction log file for '/usr/share/filebeat/data/registry/filebeat'. Active transaction id=7
2021-09-16T12:05:38.538Z INFO [registrar] registrar/registrar.go:109 States Loaded from registrar: 1
2021-09-16T12:05:38.538Z INFO [crawler] beater/crawler.go:71 Loading Inputs: 1
2021-09-16T12:05:38.538Z INFO log/input.go:157 Configured paths: [/var/log/logger.json]
2021-09-16T12:05:38.538Z INFO [crawler] beater/crawler.go:141 Starting input (ID: 5347375231622932818)
2021-09-16T12:05:38.538Z INFO [crawler] beater/crawler.go:108 Loading and starting Inputs completed. Enabled inputs: 1
2021-09-16T12:06:08.540Z INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000}},"id":"0ee7a6d770a00ad9f44b6c9cc313453506a8ce140680f43183bb8c2f1c8d745a"},"cpuacct":{"id":"0ee7a6d770a00ad9f44b6c9cc313453506a8ce140680f43183bb8c2f1c8d745a","total":{"ns":257949678}},"memory":{"id":"0ee7a6d770a00ad9f44b6c9cc313453506a8ce140680f43183bb8c2f1c8d745a","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":43708416}}}},"cpu":{"system":{"ticks":60,"time":{"ms":63}},"total":{"ticks":180,"time":{"ms":192},"value":180},"user":{"ticks":120,"time":{"ms":129}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral_id":"e45ec394-5d14-423f-a572-55befcc58b4c","uptime":{"ms":30312}},"memstats":{"gc_next":18462496,"memory_alloc":9723624,"memory_sys":74793984,"memory_total":45335960,"rss":71954432},"runtime":{"goroutines":22}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0},"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":1,"update":1},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":4},"load":{"1":1.38,"15":1.84,"5":1.85,"norm":{"1":0.345,"15":0.46,"5":0.4625}}}}}}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 19, 2021, 10:10pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/5 "2021-09-19T22:10:39Z")

</div>

I think it'd be useful if you could show your entire Filebeat config please.

---

<div class="post-metadata">

**Author:** ![caraboy](https://avatars.discourse-cdn.com/v4/letter/c/b2d939/32.png) [@caraboy](https://discuss.elastic.co/u/caraboy)\
**Post date:** [September 20, 2021, 1:23pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/6 "2021-09-20T13:23:42Z")

</div>

Thanks for the answer!  
Here you got. I've tried changing the order of the settings, placing index config upper and lower. Also changed from dot notation, to indent (YAML) notation, nothing works. I think the issue is when setting the template cause, like i said, it was created without specifying one.

```auto
name: "filebeat:prueba2"
tags: ["filebeat", "prueba"]

filebeat.inputs:
- type: log
  enabled: true
  #tail_files:true
  paths:
    - /var/log/logger.json
  json: #storing logs as json
    add_error_key: true 
    keys_under_root: true

output.elasticsearch:
  hosts: ["XXXXXXXXXXX:9200"]
  username: "XXXXX"
  password: "XXXXXXXX"
  index: "sai5x-test"
setup.template:
  name: "sai5xa"
  pattern: "sai5xa*"
  type: "index"

logging.level: debug
logging.to_files: true
logging.files:
  path: /home/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

```

---

<div class="post-metadata">

**Author:** ![caraboy](https://avatars.discourse-cdn.com/v4/letter/c/b2d939/32.png) [@caraboy](https://discuss.elastic.co/u/caraboy)\
**Post date:** [September 23, 2021, 12:47pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/7 "2021-09-23T12:47:30Z")

</div>

Solved! Had to set `setup.ilm.enabled: false`  
Thank's for the help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 21, 2021, 2:47pm UTC](https://discuss.elastic.co/t/filebeat-output-to-existing-index/284329/8 "2021-10-21T14:47:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.