# Help please.. breaking changes?
**URL:** https://discuss.elastic.co/t/help-please-breaking-changes/108010
**Category:** Logstash
**Created:** [November 16, 2017, 7:54pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010 "2017-11-16T19:54:11Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@Maekee](https://discuss.elastic.co/u/Maekee)
#### Post date: [November 16, 2017, 7:54pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/1 "2017-11-16T19:54:11Z")
Upgrading to 6.0.0 of the ELK stack
Can anyone please help me why i get this entrys over and over in the Logstash log
> [2017-11-16T20:49:36,973][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"winlogbeat-2017.11.16", :\_type=\>"wineventlog", :\_routing=\>nil}, #LogStash::Event:0x59ffd445], :response=\>{"index"=\>{"\_index"=\>"winlogbeat-2017.11.16", "\_type"=\>"wineventlog", "\_id"=\>"LcRhxl8BA5Maqd7NKABD", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Rejecting mapping update to [winlogbeat-2017.11.16] as the final mapping would have more than 1 type: [doc, wineventlog]"}}}}
This is my logstash.json
> input {
> beats {
> port =\> 5044
> type =\> "log"
> }
> }
>
> filter {
> if [task] == "Network Policy Server" {
> mutate { add\_tag =\> ["nps","radius"] }
> }
> }
>
> output {
> if "radius" in [tags] {
> elasticsearch {
> hosts =\> "elasticsearchhost:9200"
> manage\_template =\> false
> index =\> "%{[@metadata][beat]}-radius-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> } else if "dc" in [tags] {
> elasticsearch {
> hosts =\> "elasticsearchhost:9200"
> manage\_template =\> false
> index =\> "%{[@metadata][beat]}-dc-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> } else {
> elasticsearch {
> hosts =\> "elasticsearchhost:9200"
> manage\_template =\> false
> index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> }
> }
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 16, 2017, 8:16pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/2 "2017-11-16T20:16:38Z")
> [@Maekee](#):
>
> ```auto
> "Rejecting mapping update to [winlogbeat-2017.11.16] as the final mapping would have more than 1 type: [doc, wineventlog]"}}}}
>
> ```
You are apparently trying to send multiple types to a single index name. This is disallowed in 6.0.
See:
[Breaking Changes in 6.0](https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-6.0.html)
[Removal of Mapping Types](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html)
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 16, 2017, 8:20pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/3 "2017-11-16T20:20:00Z")
Have you upgraded winlogbeat?
---
### Author:  [@Maekee](https://discuss.elastic.co/u/Maekee)
#### Post date: [November 16, 2017, 8:22pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/4 "2017-11-16T20:22:01Z")
I have tried, but the events dont come into elasticsearch with the 6.0.0 version of the winlogbeat agent.
Dont know which part of the Beat-\>Logstash-\>ElasticSearch chain i have incorrect right now.
There is a breaking change somewhere.
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 16, 2017, 8:23pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/5 "2017-11-16T20:23:49Z")
My guess is that this is because you're trying to do:
```auto
document_type => "%{[@metadata][type]}"
```
The new Winlogbeat template is probably using `doc` by default (as that's what the defaults is for a single type per index in 6.0), which would cause the collision if you tried to add `wineventlog` as another type.
---
### Author:  [@Maekee](https://discuss.elastic.co/u/Maekee)
#### Post date: [November 16, 2017, 8:24pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/6 "2017-11-16T20:24:48Z")
What does that mean?
It it because if this output logstash filter maybe?
> output {
> if "radius" in [tags] {
> elasticsearch {
> hosts =\> "elasticsearchserver:9200"
> manage\_template =\> false
> index =\> "%{[@metadata][beat]}-radius-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> } else if "dc" in [tags] {
> elasticsearch {
> hosts =\> "elasticsearchserver:9200"
> manage\_template =\> false
> index =\> "%{[@metadata][beat]}-dc-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> } else {
> elasticsearch {
> hosts =\> "elasticsearchserver:9200"
> manage\_template =\> false
> index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
> document\_type =\> "%{[@metadata][type]}"
> }
> }
> }
---
### Author:  [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)
#### Post date: [November 16, 2017, 8:32pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/7 "2017-11-16T20:32:49Z")
Yes, it's because you're manually specifying the `document_type` with the value in `@metadata`. It might be as simple as just changing it to:
```auto
document_type => "doc"
```
---
### Author:  [@Maekee](https://discuss.elastic.co/u/Maekee)
#### Post date: [November 17, 2017, 7:42am UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/8 "2017-11-17T07:42:23Z")
I removed the document\_type from my logstash.json and updated the index name to a more static name, so it looks like this:
> output {
> if "radius" in [tags] {
> elasticsearch {
> hosts =\> "vascmdbtest01:9200"
> manage\_template =\> false
> index =\> "winlogbeat-radius-%{+YYYY.MM.dd}"
> }
> } else if "dc" in [tags] {
> elasticsearch {
> hosts =\> "vascmdbtest01:9200"
> manage\_template =\> false
> index =\> "winlogbeat-dc-%{+YYYY.MM.dd}"
> }
> } else {
> elasticsearch {
> hosts =\> "vascmdbtest01:9200"
> manage\_template =\> false
> index =\> "winlogbeat-%{+YYYY.MM.dd}"
> }
> }
> }
these bastards are still coming in:
> [2017-11-17T08:39:20,548][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"winlogbeat-2017.11.17", :\_type=\>"wineventlog", :\_routing=\>nil}, #LogStash::Event:0x42a0754], :response=\>{"index"=\>{"\_index"=\>"winlogbeat-2017.11.17", "\_type"=\>"wineventlog", "\_id"=\>"w9bqyF8BA5Maqd7N7qJR", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Rejecting mapping update to [winlogbeat-2017.11.17] as the final mapping would have more than 1 type: [doc, wineventlog]"}}}}
Where does the doc and wineventlog coming from??
---
### Author:  [@mbsj](https://discuss.elastic.co/u/mbsj)
#### Post date: [November 17, 2017, 10:09am UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/9 "2017-11-17T10:09:44Z")
I have the same issue for a couple of my inputs. My configuration looks like this:
```
input {
tcp {
port => 5560
codec => "json"
type => ["performancecounter"]
}
}
filter {
if [type] == "performancecounter" {
grok {
match => { "CounterPath" => [ "\\\\%{DATA:hostname}\\%{DATA:countergroup}\(%{DATA:hostname}\\private\$\\%{DATA:service}\)\\%{GREEDYDATA:Counter}",
"\\\\%{DATA:hostname}\\%{DATA:countergroup}\\%{GREEDYDATA:Counter}" ] }
}
date {
"match" => ["MeasureTime", "yyyy-MM-dd HH:mm:ss,SSS"]
target => "@timestamp"
}
mutate {
replace => { "type" => "performancecounter" }
}
}
}
output {
if [type] == "performancecounter" {
elasticsearch {
hosts => ["172.22.22.195:9200"]
index => "performance-%{+YYYY.MM.dd}"
}
}
}
```
With this i get the following error:
```
[2017-11-17T10:50:46,492][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"performance-2017.11.17", :_type=>"performancecounter", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"performance-2017.11.17", "_type"=>"performancecounter", "_id"=>"9KhjyV8BDxseqbvrQakY", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [performance-2017.11.17] as the final mapping would have more than 1 type: [performancecounter, json]"}}}}
```
I have tried to work around it by removing the type and using add\_field to create a custom field, then check for that instead. But then i end up with this error:
```
[2017-11-17T10:58:27,378][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"performance-2017.11.17", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"performance-2017.11.17", "_type"=>"doc", "_id"=>"dqhqyV8BDxseqbvrRboa", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [performance-2017.11.17] as the final mapping would have more than 1 type: [json, doc]"}}}}
```
Is there a way to strip this in the filter or output, removing one value or forcing a specific value? I have tried using mutate in the filter to replace the type field with a new (the same) value but it doesn't change anything.
---
### Author:  [@Maekee](https://discuss.elastic.co/u/Maekee)
#### Post date: [November 17, 2017, 12:12pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/10 "2017-11-17T12:12:32Z")
I got help in the beat-forum here: [Migrating from Filebeat 5.5.2 to 6.0.0](https://discuss.elastic.co/t/migrating-from-filebeat-5-5-2-to-6-0-0/107968/4)
I had not uploaded a new beat template, after doing that, i updated the logstash.json and restarted the beat agents the errors disappeared.
Changed the logstash output to this:
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
Removed this from the input block: type =\> "log"
---
### Author:  [@rsaeks](https://discuss.elastic.co/u/rsaeks)
#### Post date: [November 22, 2017, 6:01pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/11 "2017-11-22T18:01:58Z")
I'm seeing a similar issue to this, but without even using beats. I've really tried to simplify our config for testing this out and have no references to type anywhere in my config:
```auto
input {
udp {
port => 5514
tags => ["cisco-switch"]
}
}
filter {
if "cisco-switch" in [tags] {
grok {
match => ["message" , "<%{NUMBER:message_type_id}>%{BASE10NUM:event_no}: %{CISCOTIMESTAMP:cisco_time} %{WORD}: %%{NOTSPACE:cisco_type}: %{GREEDYDATA:description} %{NOTSPACE:interface}, %{GREEDYDATA:status}"]
match => ["message" , "<%{NUMBER:message_type_id}>%{BASE10NUM:event_no}: %{CISCOTIMESTAMP:cisco_time} %{WORD}: %%{NOTSPACE:cisco_type}: %{GREEDYDATA:description} %{NOTSPACE:interface}: %{GREEDYDATA:status}"]
add_tag => ["cisco-switch-processed"]
add_tag => ["30-filter-ciscoSwitch"]
tag_on_failure => ["cisco-switch-todo"]
}
date {
match => ["cisco_time", "MMM dd HH:mm:ss.SSS","MMM d HH:mm:ss.SSS"]
timezone => "America/Chicago"
target => ["eventTime"]
}
}
}
output {
if "cisco-switch-processed" in [tags] {
elasticsearch { index => "ciscoswitch-%{+YYYY.MM.dd}"}
}
else {
elasticsearch { index => "cleanup-%{+YYYY.MM.dd}"}
}
}
```
But still see an error with "cisco-switch" listed as s type:
[2017-11-22T11:45:02,127][DEBUG][o.e.a.b.TransportShardBulkAction] [ciscoswitch-2017.11.22][0] failed to execute bulk item (index) BulkShardRequest [[ciscoswitch-2017.11.22][0]] containing [index {[ciscoswitch-2017.11.22][doc][8YrV5F8BPUaikWKCQbOo], source[{"cisco\_type":"LINK-3-UPDOWN","message\_type\_id":"187","description":"Interface","message":"\<187\>705992: Nov 22 11:45:01.101 CST: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/15, changed state to up","interface":"GigabitEthernet1/0/15","tags":["cisco-switch","cisco-switch-processed","30-filter-ciscoSwitch"],"cisco\_time":"Nov 22 11:45:01.101","@timestamp":"2017-11-22T17:45:02.110Z","event\_no":"705992","@version":"1","host":"192.168.52.4","eventTime":"2017-11-22T17:45:01.101Z","status":"changed state to up"}]}]
java.lang.IllegalArgumentException: Rejecting mapping update to [ciscoswitch-2017.11.22] as the final mapping would have more than 1 type: [cisco-switch, doc]
Any ideas on what else in the config file may need to be changed so "cisco-switch" no longer appears in the type?
---
### Author:  [@Maekee](https://discuss.elastic.co/u/Maekee)
#### Post date: [November 22, 2017, 7:32pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/12 "2017-11-22T19:32:59Z")
As i understand, one change in 6.0.0 is that only one document type is supported, and in 7.0 types/mapping will be removed. But where do my types come from? beats?
---
### Author:  [@Bharathkumarraju](https://discuss.elastic.co/u/Bharathkumarraju)
#### Post date: [November 23, 2017, 8:19am UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/13 "2017-11-23T08:19:51Z")
/tmp/elastic\_dev/filebeat/current/filebeat -c /tmp/elastic\_dev/filebeat/config/filebeat.yml -e
2017/11/23 08:05:06.633737 beat.go:426: INFO Home path: [/tmp/elastic\_dev/filebeat/current] Config path: [/tmp/elastic\_dev/filebeat/current] Data path: [/tmp/elastic\_dev/filebeat/current/data] Logs path: [/tmp/elastic\_dev/filebeat/current/logs]
2017/11/23 08:05:06.633916 beat.go:433: INFO Beat UUID: ca5704f8-9b1a-4c94-8766-1dc76b119230
2017/11/23 08:05:06.633952 beat.go:192: INFO Setup Beat: filebeat; Version: 6.0.0
2017/11/23 08:05:06.634604 metrics.go:23: INFO Metrics logging every 30s
2017/11/23 08:05:06.635838 client.go:123: INFO Elasticsearch url: [https://sample.test.raju.com:9200](https://sample.test.raju.com:9200)
2017/11/23 08:05:06.636048 client.go:123: INFO Elasticsearch url: [https://sample.test.raju.com:9220](https://sample.test.raju.com:9220)
2017/11/23 08:05:06.636161 client.go:123: INFO Elasticsearch url: [https://sample.test.raju.com:9230](https://sample.test.raju.com:9230)
2017/11/23 08:05:06.636812 module.go:80: INFO Beat name: 10.20.175.66
2017/11/23 08:05:06.641468 beat.go:260: INFO filebeat start running.
2017/11/23 08:05:06.642313 registrar.go:88: INFO Registry file set to: /tmp/elastic\_dev/filebeat/current/data/registry
2017/11/23 08:05:06.642475 registrar.go:108: INFO Loading registrar data from /tmp/elastic\_dev/filebeat/current/data/registry
2017/11/23 08:05:06.643372 registrar.go:119: INFO States Loaded from registrar: 4
2017/11/23 08:05:06.643439 crawler.go:44: INFO Loading Prospectors: 2
2017/11/23 08:05:06.643746 registrar.go:150: INFO Starting Registrar
2017/11/23 08:05:06.644503 prospector.go:103: INFO Starting prospector of type: log; id: 9119168733948319376
2017/11/23 08:05:06.645260 harvester.go:207: INFO Harvester started for file: /opt/hello1/test\_ServiceAudit.log
2017/11/23 08:05:06.645842 prospector.go:103: INFO Starting prospector of type: log; id: 17106901312407876564
2017/11/23 08:05:06.645874 crawler.go:78: INFO Loading and starting Prospectors completed. Enabled prospectors: 2
2017/11/23 08:05:06.648357 harvester.go:207: INFO Harvester started for file: /opt/hello2/test\_ProtocolAudit.log
2017/11/23 08:05:07.697281 client.go:651: INFO Connected to Elasticsearch version 6.0.0
2017/11/23 08:05:07.700284 client.go:651: INFO Connected to Elasticsearch version 6.0.0
2017/11/23 08:05:07.704069 client.go:651: INFO Connected to Elasticsearch version 6.0.0
2017/11/23 08:05:08.722058 client.go:465: WARN Can not index event (status=400): {"type":"illegal\_argument\_exception","reason":"Rejecting mapping update to [service-audit-2017.11.23] as the final mapping would have more than 1 type: [log, doc]"}
2017/11/23 08:05:08.722107 client.go:465: WARN Can not index event (status=400): {"type":"illegal\_argument\_exception","reason":"Rejecting mapping update to [protocol-audit-2017.11.23] as the final mapping would have more than 1 type: [log, doc]"}
---
### Author:  [@Bharathkumarraju](https://discuss.elastic.co/u/Bharathkumarraju)
#### Post date: [November 23, 2017, 8:20am UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/14 "2017-11-23T08:20:27Z")
> [@Bharathkumarraju](#):
>
> {"type":"illegal\_argument\_exception","reason":"Rejecting mapping update to [service-audit-2017.11.23] as the final mapping would have more than 1 type: [log, doc]"}
>
> 2017/11/23 08:05:08.722107 client.go:465: WARN Can not index event (status=400): {"type":"illegal\_argument\_exception","reason":"Rejecting mapping update to [protocol-audit-2017.11.23] as the final mapping would have more than 1 type: [log, doc]"}
{"type":"illegal\_argument\_exception","reason":"Rejecting mapping update to [service-audit-2017.11.23] as the final mapping would have more than 1 type: [log, doc]"}
2017/11/23 08:05:08.722107 client.go:465: WARN Can not index event (status=400): {"type":"illegal\_argument\_exception","reason":"Rejecting mapping update to [protocol-audit-2017.11.23] as the final mapping would have more than 1 type: [log, doc]"}
---
### Author:  [@uschellh](https://discuss.elastic.co/u/uschellh)
#### Post date: [November 27, 2017, 1:16pm UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/15 "2017-11-27T13:16:15Z")
Hello,
we also suffer from this change, and i have no idea where to check now, that i got rid of types and changed the input filters and parsers to tags:
(as i tried putting in 1 type)
as the final mapping would have more than 1 type: [cisconx, WindowsEventLog]"}}}}
now change to tags:
tcp {
port =\> 5025
tags =\> "WindowsEventLog"
now other error details:
as the final mapping would have more than 1 type: [doc, cisconx
i really wonder where that cisconx comes from, its one of alot of types (we HAD)
see our conf.d folder
root@bngfvnjvhbns:/etc/logstash/conf.d# find . -type f -exec grep -H cisconx {} ;
./01\_input.conf: tags =\> "cisconx"
./01\_input.conf: tags =\> "cisconx"
./13\_filter\_cisconx.conf: if "cisconx" in [tags] {
our logstash index template: looks pretty stock i believe
(sorry for readability)
{"logstash":
{"order":0,"version":60001,"index\_patterns":["logstash-_"],"settings":
{"index":
{"refresh\_interval":"5s"}
}
,"mappings":
{"default":
{"dynamic\_templates":
[
{"message\_field":
{"path\_match":"message","match\_mapping\_type":"string","mapping":
{"type":"text","norms":false}
}
}
,
{"string\_fields":
{"match":"_","match\_mapping\_type":"string","mapping":
{"type":"text","norms":false,"fields":
{"keyword":
{"type":"keyword","ignore\_above":256}
}
}
}
}
],"properties":
{"@timestamp":
{"type":"date"},"@version":
{"type":"keyword"},"geoip":
{"dynamic":true,"properties":
{"ip":{"type":"ip"},"location":
{"type":"geo\_point"},"latitude":
{"type":"half\_float"},"longitude":
{"type":"half\_float"}}}}}},"aliases":{}}}
thanks for insights !! we dont want type cisconx
---
### Author:  [@uschellh](https://discuss.elastic.co/u/uschellh)
#### Post date: [December 8, 2017, 8:52am UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/16 "2017-12-08T08:52:58Z")
that behaviour was gone the next day, idk what it has todo with the unchanged template for the daily indexes, but whatever
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 5, 2018, 8:53am UTC](https://discuss.elastic.co/t/help-please-breaking-changes/108010/17 "2018-01-05T08:53:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.