# Metricbeat 6.3.2 upgrade breaks?

**URL:** https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241
**Category:** Beats
**Created:** [July 30, 2018, 6:59pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241 "2018-07-30T18:59:28Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)
#### Post date: [July 30, 2018, 6:59pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/1 "2018-07-30T18:59:29Z")

</div>

CentOS 7.5.1804 (from /etc/centos-release)  
ES 6.3.2 -- checked via curl  
Metricbeat 6.3.2 (from debug log)  
System module. (from JSON in debug log)

Debug logs don't show a lot that seems helpful. But there's a constant error in it. An example (slightly anonymized):

2018-07-30T14:31:09.705Z WARN elasticsearch/client.go:502 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbecfe62b2a128110, ext:10139927047, loc:(\*time.Location)(0x3b01420)}, Meta:common.MapStr(nil), Fields:common.MapStr{"metricset":common.MapStr{"module":"system", "rtt":434659, "name":"process"}, "system":common.MapStr{"process":common.MapStr{"pid":7, "username":"xxxx", "memory":common.MapStr{"rss":common.MapStr{"bytes":0x0, "pct":0}, "share":0x0, "size":0x0}, "cpu":common.MapStr{"total":common.MapStr{"value":0, "pct":0, "norm":common.MapStr{"pct":0}}, "start\_time":common.Time{wall:0x0, ext:63662610177, loc:(\*time.Location)(nil)}}, "ppid":2, "pgid":0, "fd":common.MapStr{"limit":common.MapStr{"soft":0x400, "hard":0x1000}, "open":0x0}, "name":"migration/0", "state":"sleeping"}}, "host":common.MapStr{"name":"babe"}, "beat":common.MapStr{"name":"yyyy", "hostname":"yyyy", "version":"6.3.2"}}, Private:interface {}(nil)}, Flags:0x0} (status=400): {"type":"mapper\_parsing\_exception","reason":"Failed to parse mapping [doc]: Mapping definition for [system] has unsupported parameters: [properties : {ticks={type=long}}]","caused\_by":{"type":"mapper\_parsing\_exception","reason":"Mapping definition for [system] has unsupported parameters: [properties : {ticks={type=long}}]"}}

I got a similar error after upgrading packet beat to 6.3.2, but that went away after forcing a template overwrite. Tried that here, no joy. Nothing seems to work.

---

<div class="post-metadata">

### Author: ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)
#### Post date: [July 31, 2018, 8:50am UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/2 "2018-07-31T08:50:23Z")

</div>

Hi,

From which version are you upgrading?

Are you using custom indices or the default?

---

<div class="post-metadata">

### Author: ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)
#### Post date: [July 31, 2018, 1:00pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/3 "2018-07-31T13:00:39Z")

</div>

Using the default (did use custom a while back, but switched before going from 6.2 to 6.3.0)

Upgrade that broke was going from 6.3.0 to 6.3.2 for Metricbeat. Elasticsearch was still back at 6.2.x (I forget which point number it was. It had upgraded to 6.3.0, but the version actually running was still 6.2.x until after the upgrade to 6.3.2

I'm wondering if the fact the metricbeat fields.yml defines more fields than the ES default field limit has anything to do with it? -- Sorry, cranial flatulence...when I checked it last time I dropped a zero. Sigh.

---

<div class="post-metadata">

### Author: ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [July 31, 2018, 7:14pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/4 "2018-07-31T19:14:17Z")

</div>

Can you share the template Elasticsearch has for metricbeat-\*?

---

<div class="post-metadata">

### Author: ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)
#### Post date: [July 31, 2018, 8:07pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/5 "2018-07-31T20:07:31Z")

</div>

I put the output from /\_template/metricbeat-6.3.2 at [https://gist.github.com/Paladin/d0ff4402c40364270130a12ee278719a](https://gist.github.com/Paladin/d0ff4402c40364270130a12ee278719a) because it was about 4x larger than the max for the message body here.

I should probably note that's the 6.3.2 template. There are more templates, but that should be the one metricbeat 6.3.2 uses by default, since I'm not specifying a pattern in metricbeat.yml

---

<div class="post-metadata">

### Author: ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)
#### Post date: [August 8, 2018, 5:56pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/6 "2018-08-08T17:56:05Z")

</div>

So. No Ideas?

My temptation is to nuke the entire metric data set and start over from scratch with no data, no indexes, no templates, nothing at all. see if it works.

---

<div class="post-metadata">

### Author: ![arlen](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@arlen](https://discuss.elastic.co/u/arlen)
#### Post date: [August 8, 2018, 7:07pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/7 "2018-08-08T19:07:45Z")

</div>

And that's what I did. No idea what caused the issue, or how to fix it, so I just wiped everything and started over.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 5, 2018, 9:07pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/8 "2018-09-05T21:07:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

### Author: ![json](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/json/32/4125_2.png) [@json](https://discuss.elastic.co/u/json)
#### Post date: [February 28, 2019, 6:56am UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/9 "2019-02-28T06:56:02Z")

</div>

For anyone that runs into this, the issue is caused by a mapping conflict between the current `metricbeat-*` index and new Metricbeat data. The easiest solution is delete the old template and current index to let Metricbeat create a fresh index with the upgraded template.

```auto
DELETE metricbeat-<today>
DELETE _template/metricbeat-<old>

```

Be sure the new template is written to the cluster.

```auto
metricbeat setup --template

```

---

<div class="post-metadata">

### Author: ![json](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/json/32/4125_2.png) [@json](https://discuss.elastic.co/u/json)
#### Post date: [February 28, 2019, 3:43pm UTC](https://discuss.elastic.co/t/metricbeat-6-3-2-upgrade-breaks/142241/10 "2019-02-28T15:43:58Z")

</div>

I should be more specific about what happens here. The old Metricbeat template causes new data to have the wrong mapping. If preserving data is required, then these steps should work:

1. place a write block on the "bad" `metricbeat-<version>.*` index with `index.blocks.write: true`
2. delete the old template
3. reindex the bad index it to an index with a different name
4. delete the bad index

The new Metricbeat will then be free to index to `metricbeat-<version>` without a mapping conflict.
