# Remove \`\_all\` mapping from older indexes

**URL:** <https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263>\
**Category:** Elasticsearch\
**Created:** [January 24, 2022, 5:30pm UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263 "2022-01-24T17:30:12Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![willdennis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willdennis/32/100683_2.png) [@willdennis](https://discuss.elastic.co/u/willdennis)\
**Post date:** [January 24, 2022, 5:30pm UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/1 "2022-01-24T17:30:12Z")

</div>

Upgraded to v6.8 recently; trying to ensure I can now upgrade to v7.x Having an issue when trying to reindex my old logstash indexes created on v5.6.x - getting this message when the system goes to create the new index (using the "7.0 Upgrade Assistant" thru Kibana):

```auto
[mapper_parsing_exception] [include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field.

```

Is there a way to get rid of the `_all` mapping on the current indexes? (Sorry, have been running ELK stack for a while, but never dug deep into Elasticsearch... getting my education now!) Or, how else could I handle this error?

---

<div class="post-metadata">

**Author:** ![willdennis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willdennis/32/100683_2.png) [@willdennis](https://discuss.elastic.co/u/willdennis)\
**Post date:** [January 24, 2022, 6:38pm UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/2 "2022-01-24T18:38:09Z")

</div>

Looks like I also have a similar issue with Logstash writing to ES as well --

```auto
[2022-01-24T13:35:37,059][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2022.01.24", :_type=>"doc", :routing=>nil}, #<LogStash::Event:0x5d051aa4>], :response=>{"index"=>{"_index"=>"logstash-2022.01.24", "_type"=>"doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"Failed to parse mapping [_default_]: [include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field.", "caused_by"=>{"type"=>"mapper_parsing_exception", "reason"=>"[include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field."}}}}}

```

Is this some kind of template issue?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 24, 2022, 10:49pm UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/3 "2022-01-24T22:49:47Z")

</div>

Welcome to our community! 😃

What's your index template look like for the Logstash indices?

---

<div class="post-metadata">

**Author:** ![willdennis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willdennis/32/100683_2.png) [@willdennis](https://discuss.elastic.co/u/willdennis)\
**Post date:** [January 24, 2022, 11:31pm UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/4 "2022-01-24T23:31:37Z")

</div>

I listed the templates I have on my system, and I see this:

```auto
root@logstash01:~# curl -XGET http://localhost:9200/_cat/templates?v
name index_patterns order version
.monitoring-alerts [.monitoring-alerts-6] 0 6070299
.ml-notifications [.ml-notifications] 0 6082299
.ml-anomalies- [.ml-anomalies-*] 0 6082299
security_audit_log [.security_audit_log*] 1000
.watch-history-9 [.watcher-history-9*] 2147483647
kibana_index_template:.kibana [.kibana] 0
filebeat [filebeat-*] 0
.watches [.watches*] 2147483647
.triggered_watches [.triggered_watches*] 2147483647
.ml-config [.ml-config] 0 6082299
.monitoring-beats [.monitoring-beats-6-*] 0 6070299
.monitoring-kibana [.monitoring-kibana-6-*] 0 6070299
logstash [logstash-*] 0 50001
.monitoring-es [.monitoring-es-6-*] 0 6070299
logstash-index-template [.logstash] 0
.ml-state [.ml-state*] 0 6082299
security-index-template [.security-*] 1000
.management-beats [.management-beats] 0 67000
.monitoring-logstash [.monitoring-logstash-6-*] 0 6070299
.kibana_task_manager [.kibana_task_manager] 0 6082399
.ml-meta [.ml-meta] 0 6082299

```

So I see the "logstash" one has a 'version' from v5.0; If I list this template, I see it does indeed have the `_all` field, included from the (now-deprecated) `_default_` mapping

```auto
root@logstash01:~# curl -X GET -u undefined:$ESPASS "localhost:9200/_template/logstash?pretty"
{
  "logstash" : {
    "order" : 0,
    "version" : 50001,
    "index_patterns" : [
      "logstash-*"
    ],
    "settings" : {
      "index" : {
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
      "_default_" : {
        "_all" : {
          "enabled" : true,
          "norms" : false
        },
        "dynamic_templates" : [
          {
            "message_field" : {
              "path_match" : "message",
              "match_mapping_type" : "string",
              "mapping" : {
                "type" : "text",
                "norms" : false
              }
            }
          },
          {
            "string_fields" : {
              "match" : "*",
              "match_mapping_type" : "string",
              "mapping" : {
                "type" : "text",
                "norms" : false,
                "fields" : {
                  "keyword" : {
                    "type" : "keyword"
                  }
                }
              }
            }
          }
        ],
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "include_in_all" : false
          },
          "@version" : {
            "type" : "keyword",
            "include_in_all" : false
          },
          "geoip" : {
            "dynamic" : true,
            "properties" : {
              "ip" : {
                "type" : "ip"
              },
              "location" : {
                "type" : "geo_point"
              },
              "latitude" : {
                "type" : "half_float"
              },
              "longitude" : {
                "type" : "half_float"
              }
            }
          }
        }
      }
    },
    "aliases" : { }
  }
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 25, 2022, 12:38am UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/5 "2022-01-25T00:38:39Z")

</div>

Ok, try stopping Logstash, deleting that template and then restarting Logstash.

---

<div class="post-metadata">

**Author:** ![willdennis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/willdennis/32/100683_2.png) [@willdennis](https://discuss.elastic.co/u/willdennis)\
**Post date:** [January 25, 2022, 1:06am UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/6 "2022-01-25T01:06:50Z")

</div>

This seems to have done the trick; saw this in Logstash startup messages:

```auto
[2022-01-24T20:01:50,431][INFO][logstash.outputs.elasticsearch] Using default mapping template
[2022-01-24T20:01:50,487][INFO][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2022-01-24T20:01:50,601][INFO][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash

```

Now when I list the templates, the `logstash` one looks better:

```auto
root@logstash01:~# curl -XGET http://localhost:9200/_cat/templates?v
name index_patterns order version
--{snip}--
logstash [logstash-*] 0 60001

```

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 22, 2022, 1:07am UTC](https://discuss.elastic.co/t/remove-all-mapping-from-older-indexes/295263/7 "2022-02-22T01:07:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.