# Logstash Error: Invalid FieldReference: \`\[iso-8859-1\]to\` **URL:** \ **Category:** Logstash\ **Created:** [September 15, 2020, 4:08pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712 "2020-09-15T16:08:51Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![adamzarn](https://avatars.discourse-cdn.com/v4/letter/a/7ab992/32.png) [@adamzarn](https://discuss.elastic.co/u/adamzarn)\ **Post date:** [September 15, 2020, 4:08pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/1 "2020-09-15T16:08:52Z")
I'm using ELK v7.9.1 and I get this error with some of records that are coming from CouchDB: ```auto logstash | [2020-09-15T16:03:20,662][ERROR][logstash.javapipeline][main][69664fe8e111893f386d879e5cd848742ac1a327faf79e98fcfea9f1054190c6] A plugin had an unrecoverable error. Will restart this plugin. logstash | Pipeline_id:main logstash | Plugin: true, codec=>"json_ac28a3cf-c546-4bbc-8377-4789a73ab494", enable_metric=>true, charset=>"UTF-8">, password=>, port=>5984, host=>"host", id=>"69664fe8e111893f386d879e5cd848742ac1a327faf79e98fcfea9f1054190c6", db=>"incidents", username=>"admin", enable_metric=>true, secure=>false, heartbeat=>1000, keep_id=>false, keep_revision=>false, always_reconnect=>true, reconnect_delay=>10> logstash | Error: Invalid FieldReference: `[iso-8859-1]to` logstash | Exception: Java::OrgLogstash::FieldReference::IllegalSyntaxException logstash | Stack: org.logstash.FieldReference$StrictTokenizer.tokenize(FieldReference.java:303) ``` As far as I can tell, `[iso-8859-1]to` never appears in any of the JSON documents. What is causing this error and how can I track down which documents are causing it? Here's my `.conf` file: ```auto input { couchdb_changes { host => "host" port => "5984" username => "username" password => "password" db => "incidents" codec => json ignore_attachments => true } } filter { date { match => ["time", "yyyy-MM-dd'T'HH:mm:ss.SSSZ"] } mutate { add_field => { "doc_id" => "%{[@metadata][_id]}" } } if ([doc][all_headers][from]) { mutate { add_field => { "sender" => "%{[doc][all_headers][from]}" } } } if ([doc][subject]) { mutate { add_field => { "subject" => "%{[doc][subject]}" } } } if ([doc][email]) { mutate { add_field => { "reporter" => "%{[doc][email]}" } } } if ([doc][time]) { mutate { add_field => { "time" => "%{[doc][time]}" } } } if ([doc][categories][mlclassifier][time]) { mutate { add_field => { "timeClassification" => "%{[doc][categories][mlclassifier][time]}" } } } if ([doc][categories][mlclassifier][category]) { mutate { add_field => { "category_auto" => "%{[doc][categories][mlclassifier][category]}" } } } if ([doc][categories][manual][category]) { mutate { add_field => { "category_manual" => "%{[doc][categories][manual][category]}" } } } # Remove original doc mutate { remove_field => ["@timestamp", "doc_as_upsert", "@version", "doc"] } } output { stdout { codec => "rubydebug" } elasticsearch { hosts => "http://host:9200" document_id => "%{[@metadata][_id]}" index => "incidents" } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 15, 2020, 5:33pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/2 "2020-09-15T17:33:10Z")
The couchdb\_changes input sets include\_docs to true, so, as I understand it, the changes stream will include the documents. If any of those documents contain square brackets then the json codec will interpret them as field references. I think the answer is to use a plain codec and sanitize the JSON before using a json filter. ---
**Author:** ![adamzarn](https://avatars.discourse-cdn.com/v4/letter/a/7ab992/32.png) [@adamzarn](https://discuss.elastic.co/u/adamzarn)\ **Post date:** [September 15, 2020, 5:42pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/3 "2020-09-15T17:42:27Z")
So this problem will arise if a document contains square brackets anywhere in a key or value? `[iso-8859-1]to` isn't clear because that literal string doesn't exist anywhere. And when you say sanitize the JSON before using a json filter, would that have to be before the couchdb\_changes input step? As in, sanitize the JSON before it's saved in CouchDB? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 15, 2020, 6:17pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/4 "2020-09-15T18:17:19Z")
> [@adamzarn](#): > > As in, sanitize the JSON before it's saved in CouchDB? If you do that then you could continue to use a json codec. I was thinking of using mutate to modify anything in the JSON that looks like a field reference. ---
**Author:** ![adamzarn](https://avatars.discourse-cdn.com/v4/letter/a/7ab992/32.png) [@adamzarn](https://discuss.elastic.co/u/adamzarn)\ **Post date:** [September 15, 2020, 8:00pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/5 "2020-09-15T20:00:27Z")
So just for testing purposes, I removed the `codec => json` so it defaults to `plain`, and I removed the `filter` and `elasticsearch` `output` steps. Even with this configuration, `Error: Invalid FieldReference: '[iso-8859-1]to'` is thrown. Doesn't this mean that the error is thrown before it even reaches the `filter`? If that's the case, how can I fix this without modifying the data in CouchDB? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 15, 2020, 8:40pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/6 "2020-09-15T20:40:11Z")
> [@adamzarn](#): > > So just for testing purposes, I removed the `codec => json` so it defaults to `plain` The problem is that the input does not use a codec. You can configure it, since that is in logstash/inputs/base, but the input ignores it. The code blows up when it creates the event at [this](https://github.com/logstash-plugins/logstash-input-couchdb_changes/blob/15439afe191ec91ebebaca9697db8a08d86bccf8/lib/logstash/inputs/couchdb_changes.rb#L235) line of code. I see no alternative other than modifying the data in CouchDB. Extracting the key lines from the input that are effectively the codec... ``` input { generator { count => 1 lines => ['{ "foo[1]": "bar" }' ] } } filter { ruby { code => ' m = event.get("message") data = LogStash::Json.load(m) event = LogStash::Event.new(data) ' } } output { stdout { codec => rubydebug { metadata => false } } } ``` will produce the same error ``` [logstash.filters.ruby][main][...] Ruby exception occurred: Invalid FieldReference: `foo[1]` ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 13, 2020, 8:40pm UTC](https://discuss.elastic.co/t/logstash-error-invalid-fieldreference-iso-8859-1-to/248712/7 "2020-10-13T20:40:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.