# ElasticSearch Spark MLLib connectivity

**URL:** <https://discuss.elastic.co/t/elasticsearch-spark-mllib-connectivity/32456>\
**Category:** Elasticsearch\
**Created:** [October 19, 2015, 6:24am UTC](https://discuss.elastic.co/t/elasticsearch-spark-mllib-connectivity/32456 "2015-10-19T06:24:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sar82](https://avatars.discourse-cdn.com/v4/letter/s/8491ac/32.png) [@Sar82](https://discuss.elastic.co/u/Sar82)\
**Post date:** [October 19, 2015, 6:24am UTC](https://discuss.elastic.co/t/elasticsearch-spark-mllib-connectivity/32456/1 "2015-10-19T06:24:16Z")

</div>

Hello, I want to use MLLib APIs on Elasticsearch Data. I have read data from Elasticsearch using ES-Hadoop Library.  
Data is in `JavaPairRDD<String, Map<String, Object>>` format. MLLib API need data in `JavaRDD<LabeledPoint>`.  
We are managed to convert data into `RDD<LabeledPoint>` but it seems that converted data is not in correct format. Conversion code is given below.

```
static class Transformer implements
		Function<Tuple2<String, Map<String, Object>>, LabeledPoint> {

	@Override
	public LabeledPoint call(Tuple2 arg0) throws Exception {
		HashingTF tf = new HashingTF();
		Map<String, Object> map = (Map<String, Object>) arg0._2();
		//get values from Map
		Set<String> keys = map.keySet();
		List<Object> valuesList = new ArrayList<Object>();
	    for (Iterator<String> i = keys.iterator(); i.hasNext();) {
	      String key = (String) i.next();
	      Object value = (Object) map.get(key);
	      valuesList.add(value);
	    }
		return new LabeledPoint(1d, tf.transform(valuesList));
	}
}

```

Using above function I managed to create Random Forest Model. But while loading the saved model I am getting following exception

> org.json4s.package$MappingException: Did not find value which can be converted into java.lang.String

Also in all example of Spark MLLib, LIBSVM files are provided for machine learning. But in our case we are reading data from Elasticsearch. So I have 2 queries,

1. Is there any way to convert JavaPairRDD to LIBSVM format. Which will be used by MLLib APIs
2. Is above Transformer function has any problem?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:44pm UTC](https://discuss.elastic.co/t/elasticsearch-spark-mllib-connectivity/32456/2 "2017-07-05T23:44:08Z")

</div>