# Submitting jobs to apache spark

**URL:** <https://discuss.elastic.co/t/submitting-jobs-to-apache-spark/57014>\
**Category:** Kibana\
**Created:** [August 2, 2016, 1:51pm UTC](https://discuss.elastic.co/t/submitting-jobs-to-apache-spark/57014 "2016-08-02T13:51:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 2, 2016, 1:51pm UTC](https://discuss.elastic.co/t/submitting-jobs-to-apache-spark/57014/1 "2016-08-02T13:51:34Z")

</div>

Hi!

I am now creating a custom kibana plugin.

\*\* What I want to do \*\*  
I want to submit jobs to Apache spark from using result that were queried from elasticsearch.  
By submitting the job , I will generate predictive analytic data points using spark library .

I have read below article and creating a new server API looks to be a starting point.

> **[Writing Kibana Plugins – Custom applications](https://www.timroes.de/2016/02/21/writing-kibana-plugins-custom-applications/)**
>
> This tutorial series explains how to create own Kibana plugins. This part explains
> how to create a custom application plugin.

Is there a way to call spark-submit from the Kibana server?  
Or is there any api's in Spark which I can call from Kibana server?

Thanks,  
Yu Watanabe

---

<div class="post-metadata">

**Author:** ![lukas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas/32/6812_2.png) [@lukas](https://discuss.elastic.co/u/lukas)\
**Post date:** [August 2, 2016, 4:10pm UTC](https://discuss.elastic.co/t/submitting-jobs-to-apache-spark/57014/2 "2016-08-02T16:10:16Z")

</div>

I'm not too sure about what exists on the Apache Spark side, but you should be able to create your own server APIs which invoke the Spark APIs.

If I'm understanding correctly, you'd like to submit jobs to Spark _every_ time there is a query to Elasticsearch?

---

<div class="post-metadata">

**Author:** ![YuWatanabe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuwatanabe/32/13259_2.png) [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\
**Post date:** [August 2, 2016, 11:17pm UTC](https://discuss.elastic.co/t/submitting-jobs-to-apache-spark/57014/3 "2016-08-02T23:17:54Z")

</div>

Hello @lukas

Yes . Your understanding is correct. I am planning to connect to spark whenever there is a  
search executed from Kibana interface and accessed to elasticsearch.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:42pm UTC](https://discuss.elastic.co/t/submitting-jobs-to-apache-spark/57014/4 "2017-07-06T13:42:20Z")

</div>


