# ES-Hadoop configuration parameter es.read.source.filter doesn't accept a list

**URL:** <https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [February 26, 2018, 11:25pm UTC](https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576 "2018-02-26T23:25:22Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kahndor](https://avatars.discourse-cdn.com/v4/letter/k/97f17d/32.png) [@kahndor](https://discuss.elastic.co/u/kahndor)\
**Post date:** [February 26, 2018, 11:25pm UTC](https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576/1 "2018-02-26T23:25:22Z")

</div>

When I try to limit my ES results using Spark and the Python API I am able to retrieve only the desired fields with:  
` es_conf = {"es.nodes": "localhost", "es.port" : "9200", "es.read.source.filter": "some_field"}`

But if I try to query for more than one field e.g.

` es_conf = {"es.nodes": "localhost", "es.port" : "9200", "es.read.source.filter": ["some_field", "another_field]}`

I get:

> An error occurred while calling z:org.apache.spark.api.python.PythonRDD.newAPIHadoopRDD.  
> : java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.lang.String  
> at org.apache.spark.api.python.PythonHadoopUtil$$anonfun$mapToConf$1.apply(PythonHadoopUtil.scala:160)  
> at org.apache.spark.api.python.PythonHadoopUtil$$anonfun$mapToConf$1.apply(PythonHadoopUtil.scala:160)  
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)  
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)  
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)  
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)  
> at org.apache.spark.api.python.PythonHadoopUtil$.mapToConf(PythonHadoopUtil.scala:160)  
> at org.apache.spark.api.python.PythonRDD$.newAPIHadoopRDD(PythonRDD.scala:580)  
> at org.apache.spark.api.python.PythonRDD.newAPIHadoopRDD(PythonRDD.scala)  
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
> at java.lang.reflect.Method.invoke(Method.java:498)  
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
> at py4j.Gateway.invoke(Gateway.java:280)  
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
> at py4j.commands.CallCommand.execute(CallCommand.java:79)  
> at py4j.GatewayConnection.run(GatewayConnection.java:214)  
> at java.lang.Thread.run(Thread.java:748)

According to the [documentation](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/configuration.html#_required_settings) I should be able to provide a list.

Can anyone help me with this seemingly simple task?

---

<div class="post-metadata">

**Author:** ![kahndor](https://avatars.discourse-cdn.com/v4/letter/k/97f17d/32.png) [@kahndor](https://discuss.elastic.co/u/kahndor)\
**Post date:** [February 26, 2018, 11:49pm UTC](https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576/2 "2018-02-26T23:49:45Z")

</div>

The documentation should have been a bit more explicit or descriptive or provided an example cause it was simple but confusihng. Placing the desired fields in a comma separated string worked.

`es_conf = {"es.nodes": "localhost", "es.port" : "9200", "es.read.source.filter": "some_field, another_field"}`

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [February 28, 2018, 8:12pm UTC](https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576/3 "2018-02-28T20:12:10Z")

</div>

Sorry about the confusing docs, I'll push an update to them to make them more clear. Thanks for your patience!

---

<div class="post-metadata">

**Author:** ![kahndor](https://avatars.discourse-cdn.com/v4/letter/k/97f17d/32.png) [@kahndor](https://discuss.elastic.co/u/kahndor)\
**Post date:** [February 28, 2018, 9:39pm UTC](https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576/4 "2018-02-28T21:39:25Z")

</div>

Thanks! Really enjoy the community -- The products, the effort, and the responsiveness. I hope in the future I'll have time to help out.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 28, 2018, 9:39pm UTC](https://discuss.elastic.co/t/es-hadoop-configuration-parameter-es-read-source-filter-doesnt-accept-a-list/121576/5 "2018-03-28T21:39:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.