# New index pattern to current index - topbeat-\*?
**URL:** https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627
**Category:** Elasticsearch
**Created:** [February 12, 2016, 3:06pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627 "2016-02-12T15:06:38Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 12, 2016, 3:06pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/1 "2016-02-12T15:06:38Z")
Hi all,
I'm trying to update an existing `topbeat-[YYYY-MM-DD]` index to accept an updated index pattern.
I am doing this because the topbeat sample dashboard has issues with `beat.name` and `beat.hostname` showing up as analysed fields. The dashboards do not appreciate hostnames that contain a hyphen "-" them, if it is an analysed field. The result is that the dashboard shows multiple results (The issue is also reported [in this thread](https://discuss.elastic.co/t/topbeat-beat-hostname-analyzed/36930)). The index pattern in the sample dashboards release seems to contain an updated index pattern.
By deleting, and running the `load.ps1` import script I am able to import the recommended/updated index pattern, but I understand that I need to update the indices themselves.
I have also checked how to update the indices settings, and seen [this elastic article](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html#update-settings-analysis) on how to do so using the REST API.
I am just wondering at this point, what JSON command(s) do I need to accomplish this.
Does anyone know the correct JSON required to update an existing index to inherit the updated index pattern, in particular the `beat.name` and `beat.hostname` fields?
_NOTE: I'm still somewhat new to this, so if I'm barking up the wrong tree here, I'd welcome any pointers._
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 14, 2016, 4:38pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/2 "2016-02-14T16:38:08Z")
You can't change the mapping of existing fields without reindexing, i.e. creating a new index to which the existing data is copied, followed by a deletion of the old index. How to reindex data is a fairly common topic here (it has been covered in the past couple of days).
---
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 15, 2016, 11:01am UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/3 "2016-02-15T11:01:11Z")
Thanks @magnusbaeck, much appreciated advice.
I also got some advice from @andrewkroh in the other topic [here](https://discuss.elastic.co/t/topbeat-beat-hostname-analyzed/36930):
> [@Topbeat beat.hostname analyzed](https://discuss.elastic.co/t/topbeat-beat-hostname-analyzed/36930/11):
>
> @plonka2000 You would need to re-index your existing data. It takes a bit of work and the way you accomplish the task varies by your technology. Basically you read the data out of ES then re-write it to a new index. See the links below.
>
> [Reindexing Your Data | Elasticsearch: The Definitive Guide [2.x] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html)
> [https://david.pilato.fr/blog/2015-05-20-reindex-elasticsearch-with-logstash/](http://david.pilato.fr/blog/2015/05/20/reindex-elasticsearch-with-logstash/)
>
> In the future this will be easier to do: [Reindex API · Issue #15201 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/15201)
Thanks very much @andrewkroh, I'm going to investigate the possibility of reindexing, and if it proves too complicated/time-consuming/scary I may go with the original 'delete and create' solution (I'll chalk this one up to experience and better planning next time).
Thanks to you both for your help.
---
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 15, 2016, 4:37pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/4 "2016-02-15T16:37:35Z")
In the end I deleted all the indices and started over, but the problem persists.
I double-checked to make sure the index pattern is as desired:
In the end I deleted all traces of the sample dashboards:
-Deleted all imported `Searches`
-Deleted all imported `Dashboards`
-Deleted all imported `index patterns`
-Deleted all imported `Visualisations`
Then:
-Imported latest `beats-dashboards-1.1.1` sample dashboards.
The same results after all changes.
Screenshot of hostname listed in sample Kibana dashboard:
(_ **This should be a single hostname win-lhc7bf0fdt1 but registers as two** _)
I'm at somewhat of a loss as to why this is happening now. Shouldn't this work?
I've even downloaded and used the latest `topbeat-1.1.1` release on the client.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 15, 2016, 6:01pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/5 "2016-02-15T18:01:59Z")
Are you sending directly from Topbeat to ES or do you have a Logstash instance inbetween? In the former case, what does your configuration look like?
---
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 16, 2016, 9:44am UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/6 "2016-02-16T09:44:39Z")
I'm sending directly to ES, not via Logstash in this instance.
Below is my topbeat config (I've removed all "#" lines):
```auto
input:
period: 5
procs: [".*"]
stats:
system: true
proc: true
filesystem: true
cpu_per_core: false
output:
elasticsearch:
hosts: ["http://myesclusterurl:9200"]
protocol: "http"
username: "beatuser"
password: "beatpassword"
index: "topbeat"
shipper:
logging:
to_files: true
files:
path: c:\beatlogs
name: topbeat.log
keepfiles: 20
level: debug
```
I think its a pretty standard config.
Thanks for having a look.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 16, 2016, 9:50am UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/7 "2016-02-16T09:50:15Z")
When you recreated the index(es) after deleting them, did you make sure to mark the hostname field as not analyzed? The evidence suggests that it's still analyzed.
---
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 16, 2016, 10:07am UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/8 "2016-02-16T10:07:06Z")
I don't believe I did that, no.
I thought the index patterns would be applied to the newly created indices?
The screenshot I provided shows in Kibana that the `beat.hostname` and `beat.name` fields are not analyzed.
Again, I could be misunderstanding what I'm looking at here.
Is there a way to manually mark them?
**_Update: I assume this is possible via the ES REST API, and I'm trying to determine what instructions I need, but REST API functions are where I'm unsure with ES._**
---
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 16, 2016, 10:36am UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/9 "2016-02-16T10:36:01Z")
It does seem that its not working.
I have just deleted all indices, added new data and refreshed the fields, and now my fields show up analyzed:
I'm unsure what to do to make the configuration stick.
---
### Author:  [@plonka2000](https://discuss.elastic.co/u/plonka2000)
#### Post date: [February 16, 2016, 11:16am UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/10 "2016-02-16T11:16:57Z")
I think I got it to work!
Kibana now shows:
**What I did:**
I read up on the Index templates article [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html).
Ran a query against the cluster to check if there was a valid template for topbeat:
`curl -XGET localhost:9200/_template/`
Response:
## [code]logstash
@{order=0; template=logstash-\*; settings=; mappings=; aliases=}[/code]
So, I think there is no topbeat index template...
Next I used the content of `topbeat.template.json` file (included with `topbeat-1.1.1`), and used [kopf](https://github.com/lmenezes/elasticsearch-kopf) to create a new index template for topbeat:
I saw that refreshing still did no good.
So once again, I deleted everything (Didn't delete index template in kopf) and re-imported the entire stack:
> [@plonka2000](#):
>
> In the end I deleted all traces of the sample dashboards:
> -Deleted all imported `Searches`
> -Deleted all imported `Dashboards`
> -Deleted all imported `index patterns`
> -Deleted all imported `Visualisations`
>
> Then:
> -Imported latest `beats-dashboards-1.1.1` sample dashboards.
_ **Now it looks like it works!** _
I'm quite sure there is some way to do this all via the ES REST API, but I have not figured out that part yet.
---
### Author:  [@DenysK](https://discuss.elastic.co/u/DenysK)
#### Post date: [April 11, 2016, 4:29pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/11 "2016-04-11T16:29:15Z")
Hello Eveybody,
All previous solutions did not help me with Elasticsearch analyzer.
I did the next steps:
1. Stop topbeat on source host
2. Remove "topbeat-\*" index from Kibana
3. Remove topbeat template from Elasticsearch by ` curl -XDELETE 'http://IP:9200/topbeat-*'`
4. Add new custom template by using "kopf" plugin:
Custom part in template default section:
> "hostname": {
> "type": "string",
> "index": "not\_analyzed"
> },
> "name": {
> "type": "string",
> "index": "not\_analyzed"
> },
topbeat.template.json:
> {
> "mappings": {
> "_default_": {
> "\_all": {
> "enabled": true,
> "norms": {
> "enabled": false
> }
> },
> "dynamic\_templates": [
> {
> "template1": {
> "mapping": {
> "doc\_values": true,
> "ignore\_above": 1024,
> "index": "not\_analyzed",
> "type": "{dynamic\_type}"
> },
> "match": "_"
> }
> }
> ],
> "properties": {
> "@timestamp": {
> "type": "date"
> },
> "hostname": {
> "type": "string",
> "index": "not\_analyzed"
> },
> "name": {
> "type": "string",
> "index": "not\_analyzed"
> },
> "cpu": {
> "properties": {
> "system\_p": {
> "doc\_values": "true",
> "type": "float"
> },
> "user\_p": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> },
> "fs": {
> "properties": {
> "used\_p": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> },
> "load": {
> "properties": {
> "load1": {
> "doc\_values": "true",
> "type": "float"
> },
> "load15": {
> "doc\_values": "true",
> "type": "float"
> },
> "load5": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> },
> "mem": {
> "properties": {
> "actual\_used\_p": {
> "doc\_values": "true",
> "type": "float"
> },
> "used\_p": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> },
> "proc": {
> "properties": {
> "cpu": {
> "properties": {
> "user\_p": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> },
> "mem": {
> "properties": {
> "rss\_p": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> }
> }
> },
> "swap": {
> "properties": {
> "used\_p": {
> "doc\_values": "true",
> "type": "float"
> }
> }
> }
> }
> }
> },
> "settings": {
> "index.refresh\_interval": "5s"
> },
> "template": "topbeat-_"
> }
5 Start topbeat on source host
6 Problem solved. Now beat.hostname and beat.name works correctly.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:00pm UTC](https://discuss.elastic.co/t/new-index-pattern-to-current-index-topbeat/41627/12 "2017-07-05T23:00:40Z")