# Getting Invalid FieldReference error because of non-indexed property name in document

**URL:** <https://discuss.elastic.co/t/getting-invalid-fieldreference-error-because-of-non-indexed-property-name-in-document/274205>\
**Category:** Logstash\
**Created:** [May 27, 2021, 12:17pm UTC](https://discuss.elastic.co/t/getting-invalid-fieldreference-error-because-of-non-indexed-property-name-in-document/274205 "2021-05-27T12:17:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rs0000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rs0000/32/89345_2.png) [@rs0000](https://discuss.elastic.co/u/rs0000)\
**Post date:** [May 27, 2021, 12:17pm UTC](https://discuss.elastic.co/t/getting-invalid-fieldreference-error-because-of-non-indexed-property-name-in-document/274205/1 "2021-05-27T12:17:57Z")

</div>

Hi,

My Logstash pipeline keeps failing during scroll because of an error. There are multiple documents causing that issue. Those are referral urls that sometimes look weird. The property itself is not indexed in elasticsearch (`{"type": "object", "enabled": false}`). Do you have any suggestions how to fix this?

Logstash Versions tested: 7.12.1, 7.10.1

`Plugin: <LogStash::Inputs::Elasticsearch`

Input Configuration:

```auto
input {
    elasticsearch {
        hosts => ["my-host:9200"]
        index => "my-index"
        docinfo => true
        docinfo_fields => ["_type", "_id", "_index"]
        query => '{"sort":["date"],"query": {"bool": {"must": [{"range": {"date": {"gte": "2020-01-01T00:00:00"}}}]}}}'
        size => 10000
    }
}

```

Example (shortened) document causing the error:

```json
{
  "date": "2020-01-11T08:00:00.000Z",
  "count-by-referral-url": {
    "https://bs.serving-sys.com/serving/adServer.bs?cn=trd&mc=click&pli=26859493&PluID=0&ord=[timestamp]&fbclid=IwAR3p4--K2EfVX0fzTv5FgX9sEJuIsWuhHVvr6W0zOTkAG166d6QB-yMGbwI": 1
  }
}

```

Stacktrace:

```bash
  Error: Invalid FieldReference: `https://bs.serving-sys.com/serving/adServer.bs?cn=trd&mc=click&pli=26859493&PluID=0&ord=[timestamp]&fbclid=IwAR3p4--K2EfVX0fzTv5FgX9sEJuIsWuhHVvr6W0zOTkAG166d6QB-yMGbwI`
  Exception: Java::OrgLogstash::FieldReference::IllegalSyntaxException
  Stack: org.logstash.FieldReference$StrictTokenizer.tokenize(FieldReference.java:312)
org.logstash.FieldReference.parse(FieldReference.java:213)
org.logstash.FieldReference.parseToCache(FieldReference.java:204)
org.logstash.FieldReference.from(FieldReference.java:127)
org.logstash.FieldReference.lambda$from$0(FieldReference.java:118)
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1737)
org.logstash.FieldReference.from(FieldReference.java:118)
org.logstash.ConvertedMap.convertKey(ConvertedMap.java:122)
org.logstash.ConvertedMap.access$000(ConvertedMap.java:44)
org.logstash.ConvertedMap$1.visit(ConvertedMap.java:55)
org.logstash.ConvertedMap$1.visit(ConvertedMap.java:49)
org.jruby.RubyHash.visitLimited(RubyHash.java:698)
org.jruby.RubyHash.visitAll(RubyHash.java:683)
org.logstash.ConvertedMap.newFromRubyHash(ConvertedMap.java:89)
org.logstash.ConvertedMap.newFromRubyHash(ConvertedMap.java:84)
org.logstash.Valuefier.lambda$initConverters$12(Valuefier.java:171)
org.logstash.Valuefier.convert(Valuefier.java:94)
org.logstash.ConvertedMap$1.visit(ConvertedMap.java:55)
org.logstash.ConvertedMap$1.visit(ConvertedMap.java:49)
org.jruby.RubyHash.visitLimited(RubyHash.java:698)
org.jruby.RubyHash.visitAll(RubyHash.java:683)
org.logstash.ConvertedMap.newFromRubyHash(ConvertedMap.java:89)
org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_initialize(JrubyEventExtLibrary.java:92)
usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_elasticsearch_minus_4_dot_9_dot_1.lib.logstash.inputs.elasticsearch.RUBY$method$push_hit$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.1/lib/logstash/inputs/elasticsearch.rb:308)
usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_elasticsearch_minus_4_dot_9_dot_1.lib.logstash.inputs.elasticsearch.RUBY$block$do_run_slice$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.1/lib/logstash/inputs/elasticsearch.rb:274)
org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
org.jruby.runtime.Block.yield(Block.java:184)
org.jruby.RubyArray.each(RubyArray.java:1809)
usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_elasticsearch_minus_4_dot_9_dot_1.lib.logstash.inputs.elasticsearch.RUBY$method$do_run_slice$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.1/lib/logstash/inputs/elasticsearch.rb:274)
usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_elasticsearch_minus_4_dot_9_dot_1.lib.logstash.inputs.elasticsearch.RUBY$method$do_run$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.1/lib/logstash/inputs/elasticsearch.rb:250)
usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_elasticsearch_minus_4_dot_9_dot_1.lib.logstash.inputs.elasticsearch.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.9.1/lib/logstash/inputs/elasticsearch.rb:238)
usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405)
usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0$ __VARARGS__ (/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)
org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)
usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:396)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
org.jruby.runtime.Block.call(Block.java:139)
org.jruby.RubyProc.call(RubyProc.java:318)
org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
java.base/java.lang.Thread.run(Thread.java:834)

```

---

<div class="post-metadata">

**Author:** ![rs0000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rs0000/32/89345_2.png) [@rs0000](https://discuss.elastic.co/u/rs0000)\
**Post date:** [May 31, 2021, 9:50am UTC](https://discuss.elastic.co/t/getting-invalid-fieldreference-error-because-of-non-indexed-property-name-in-document/274205/2 "2021-05-31T09:50:36Z")

</div>

Since the issue is the property name itself, is there any way to not parse the source document or declare properties that should not be parsed on input? It would also be acceptable to skip documents that fail, but currently the whole plugin crashes...

---

<div class="post-metadata">

**Author:** ![rs0000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rs0000/32/89345_2.png) [@rs0000](https://discuss.elastic.co/u/rs0000)\
**Post date:** [June 1, 2021, 7:33am UTC](https://discuss.elastic.co/t/getting-invalid-fieldreference-error-because-of-non-indexed-property-name-in-document/274205/3 "2021-06-01T07:33:18Z")

</div>

I think this is the root cause: ["[" key in an event crashes logstash · Issue #12123 · elastic/logstash · GitHub](https://github.com/elastic/logstash/issues/12123)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 29, 2021, 7:33am UTC](https://discuss.elastic.co/t/getting-invalid-fieldreference-error-because-of-non-indexed-property-name-in-document/274205/4 "2021-06-29T07:33:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.