# Help with Setting up new Index/ dashboards Filebeat 7.1.1 and logstash indexes **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [June 12, 2019, 12:14pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405 "2019-06-12T12:14:10Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Lee\_Lilleorg-Meilleu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_lilleorg-meilleu/32/47953_2.png) [@Lee\_Lilleorg-Meilleu](https://discuss.elastic.co/u/Lee_Lilleorg-Meilleu)\ **Post date:** [June 12, 2019, 12:14pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/1 "2019-06-12T12:14:11Z")
Dear Support forum, I have upgraded my ELK to 7.1.1 and wanted to use the plugins and dashboards that filbeat ships with natively however I am unable to get the index to stay consistant. When using file beat and the setup command, it created a default index of filebeat-7.1.1-$DATE but the issue with thhis is it has no roll up jobs / ILM functionality. So I kick off logstash with my ES output as : output { elasticsearch { hosts =\> ["10.70.104.50:9200"] manage\_template =\> true template\_overwrite =\> true and it automatically creates a uindex of logstash-$DATE-00000 and also sets a rollup job So I figure If I set the template in my logstash output to "filebeat" instead of logstash it would create the rollup jobs but it fails to link the alias and thus doesnt work. I have tried to override it with my command : filebeat setup -e -E output.logstash.enabled=false -E output.elasticsearch.hosts=['10.70.104.50:9200'] -E setup.kibana.host=10.70.104.55:80 -E output.elasticsearch.index="logstash-%{+yyyy.MM.dd}" -E setup.template.pattern="logstash-\*" -E setup.template.name="logstash-" but it still gives me the filebeat index instead ☹ right now filebeat sends data to logstash and i get all my logs in kibana but the dashes dont work since it wants a index pattern of filebeat-\* vs the one I use logstash-\* ---
**Author:** ![thekm1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thekm1/32/35926_2.png) [@thekm1](https://discuss.elastic.co/u/thekm1)\ **Post date:** [June 12, 2019, 12:52pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/2 "2019-06-12T12:52:55Z")
@Lee_Lilleorg-Meilleu I am not sure if I understand your problem correctly. Is it right that you want to index data from logstash into elasticsearch on the same index as from the filebeat index? If yes, then you can set in logstash elasticsearch output plugin the attribute `index`with the value of the filebeat index alias on elasticsearch. And I think that filebeat got the default alias: `filebeat-{agent.version}` So try this in logstash output ```auto output { elasticsearch { hosts : ["10.70.104.50:9200"] index : "filebeat-7.1.1" } } ``` So that the logstash output is into the same index as the other filebeats are indexing data. [Doc](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-index) to index attribute in logstash output plugin elasticsearch ---
**Author:** ![Lee\_Lilleorg-Meilleu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_lilleorg-meilleu/32/47953_2.png) [@Lee\_Lilleorg-Meilleu](https://discuss.elastic.co/u/Lee_Lilleorg-Meilleu)\ **Post date:** [June 12, 2019, 1:06pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/3 "2019-06-12T13:06:33Z")
Yup, the idea is to use the same index as what the filebeat setup command creates . though for what ever reason. using the filbeat created indexes , it wont do index managemnt / create a new one each day switching the index on the logstash output -\> Elasticsearch to use a specific index (instead of the default) makes it incompatible with the policies ---
**Author:** ![thekm1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thekm1/32/35926_2.png) [@thekm1](https://discuss.elastic.co/u/thekm1)\ **Post date:** [June 12, 2019, 1:16pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/4 "2019-06-12T13:16:21Z")
So you have filebeats indexing data directly to elasticsearch. And you have filebeats indexing data through logstash and then to elasticsearch. If you want to have the data which is piped through logstash to land in the same index as filebeat indices. you would need to specify he index. Maybe there is an another way. But I don't know it yet. Yes if the filebeats are running with ILM then it will not create each day a new index. It will create an if the policy allows this. And the basic policy is something around 50GB and 7 days. so it will create a new index if the index gets 50GB big or it is 7 days old. [Doc](https://www.elastic.co/guide/en/elasticsearch/reference/6.7/index-lifecycle-management.html) Otherway would be to disable ILM, but it is a pretty nice feature which can manage your indices very well. Not sure if i helped you =) ---
**Author:** ![Lee\_Lilleorg-Meilleu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_lilleorg-meilleu/32/47953_2.png) [@Lee\_Lilleorg-Meilleu](https://discuss.elastic.co/u/Lee_Lilleorg-Meilleu)\ **Post date:** [June 12, 2019, 1:27pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/5 "2019-06-12T13:27:19Z")
the idea is that it would all be one main index : filebeat -\> logstash -\> es but the initialization of the dashboards has to happen on filebeat-\>es so I pick a client running file beat and execute the setup. : index.lifecycle.name in template to {filebeat-7.1.1 map[policy:{"phases":{"hot":{"actions":{"rollover":{"max\_age":"30d","max\_size":"50gb"}}}}}]} as ILM is enabled. after the setup: is creates the filebeat-7.1.1-$YYYY-mm-dd-00001 INDEX and the alias filebeat-7.1.1 then I need to point the ES in the logstash pipeline to use the same index as the alias name? instead of trying to make it filebeat-7.1.1-YYYY-MM-dd ? ---
**Author:** ![thekm1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thekm1/32/35926_2.png) [@thekm1](https://discuss.elastic.co/u/thekm1)\ **Post date:** [June 12, 2019, 1:32pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/6 "2019-06-12T13:32:38Z")
Yes, you need to point the ES in logstash pipeline to the same alias name which has been defined by filebeat setup ilm. That is the way how we use it and this seems to work fine. Let me know if this suits you as well. ---
**Author:** ![Lee\_Lilleorg-Meilleu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lee_lilleorg-meilleu/32/47953_2.png) [@Lee\_Lilleorg-Meilleu](https://discuss.elastic.co/u/Lee_Lilleorg-Meilleu)\ **Post date:** [June 12, 2019, 1:46pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/7 "2019-06-12T13:46:55Z")
Perfect, the Alias was what I was missing. so filebeat creates the index / alias and ES just needs to be pointed to the alias ... Thanks for clarifying that. !! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 10, 2019, 1:46pm UTC](https://discuss.elastic.co/t/help-with-setting-up-new-index-dashboards-filebeat-7-1-1-and-logstash-indexes/185405/8 "2019-07-10T13:46:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.