# Is it possible to make a ingest-plugin which processor is able to make an HTTP request **URL:** \ **Category:** Elasticsearch\ **Created:** [April 24, 2019, 6:08pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311 "2019-04-24T18:08:41Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Borwick\_5576](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borwick_5576/32/44759_2.png) [@Borwick\_5576](https://discuss.elastic.co/u/Borwick_5576)\ **Post date:** [April 24, 2019, 6:08pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/1 "2019-04-24T18:08:41Z")
The goal here is being able to send the data of documents to a service my companie has and then index the response as an anotation. Im having trouble opening a connection from the processor. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 24, 2019, 6:34pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/2 "2019-04-24T18:34:59Z")
I started a similar thing 2 years ago and stopped it as it was not a very good idea at the end of the day because this is dramatically slowing down the index operations. Instead it's better to use Logstash to do external lookups like this and then send the final document to the indexation process which will be fast then and won't block any indexing threads... Coming back to your question. I think you can. > I'm having trouble opening a connection from the processor. May be it's because of the security manager. What kind of error do you have? ---
**Author:** ![Borwick\_5576](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borwick_5576/32/44759_2.png) [@Borwick\_5576](https://discuss.elastic.co/u/Borwick_5576)\ **Post date:** [April 25, 2019, 8:16am UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/3 "2019-04-25T08:16:46Z")
Hey, thanks for the quick reply. The error is the following: > Task :forbiddenApisMain FAILED > Forbidden method invocation: java.net.URLConnection#getInputStream() [**Don't open socket connections**] > in org.elasticsearch.plugin.ingest.library.LibraryProcessor (LibraryProcessor.java:61) > Forbidden method invocation: java.io.InputStreamReader#(java.io.InputStream) [**Uses default charset**] > in org.elasticsearch.plugin.ingest.library.LibraryProcessor (LibraryProcessor.java:61) > Scanned 3 class file(s) for forbidden API invocations (in 0.36s), 2 error(s). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 25, 2019, 8:33am UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/4 "2019-04-25T08:33:58Z")
Not sure if this is the problem here but I'd explore [https://www.elastic.co/guide/en/elasticsearch/plugins/current/plugin-authors.html#plugin-authors-jsm](https://www.elastic.co/guide/en/elasticsearch/plugins/current/plugin-authors.html#plugin-authors-jsm) ---
**Author:** ![Borwick\_5576](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borwick_5576/32/44759_2.png) [@Borwick\_5576](https://discuss.elastic.co/u/Borwick_5576)\ **Post date:** [April 29, 2019, 2:05pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/5 "2019-04-29T14:05:35Z")
I've wrapped my code in a Lambda function inside like this: ``` SecurityManager sm = System.getSecurityManager(); if (sm != null) { // unprivileged code such as scripts do not have SpecialPermission sm.checkPermission(new SpecialPermission()); } AccessController.doPrivileged((PrivilegedAction) () -> { try { InputStream is = con.getInputStream(); BufferedReader in = new BufferedReader(new InputStreamReader(is,StandardCharsets.UTF_8)); StringBuffer response = new StringBuffer(); String inputLine; while ((inputLine = in.readLine()) != null) { response.append(inputLine); } in.close(); //Read JSON response and print JSONObject myResponse = new JSONObject(response.toString()); ingestDocument.setFieldValue(targetField, myResponse.getString("occupation")); return null; }catch(IOException ex){ return null; } } ); ``` and also granted socket permissions in the plugin-security.policy: ``` grant { permission java.io.FilePermission "*", "read,write"; permission java.net.SocketPermission "*", "connect,accept"; }; ``` But the problem still remains. Output of the Gradle clean check: > Configure project : > ======================================= > Elasticsearch Build Hamster says Hello! > Gradle Version : 5.2.1 > OS Info : Linux 4.15.0-47-generic (amd64) > JDK Version : 12 (Oracle Corporation 12 [Java HotSpot(TM) 64-Bit Server VM 12+33]) > JAVA\_HOME : /usr/lib/jvm/jdk-12 > Random Testing Seed : C4F42EF220CB8540 > ======================================= > Task :forbiddenApisMain FAILED > **Forbidden method invocation: java.net.URLConnection#getInputStream() [Don't open socket connections]** > in org.elasticsearch.plugin.ingest.library.LibraryProcessor (LibraryProcessor.java:83) > Scanned 3 class file(s) for forbidden API invocations (in 0.04s), 1 error(s). > ``` > FAILURE: Build failed with an exception. > > * What went wrong: > Execution failed for task ':forbiddenApisMain'. > > de.thetaphi.forbiddenapis.ForbiddenApiException: Check for forbidden API calls failed, see log. > > * Try: > Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. > > * Get more help at https://help.gradle.org > > BUILD FAILED in 2s > 11 actionable tasks: 11 executed > > ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 29, 2019, 3:34pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/6 "2019-04-29T15:34:04Z")
I don't know. May be @rjernst knows? ---
**Author:** ![Borwick\_5576](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/borwick_5576/32/44759_2.png) [@Borwick\_5576](https://discuss.elastic.co/u/Borwick_5576)\ **Post date:** [May 6, 2019, 8:29am UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/7 "2019-05-06T08:29:03Z")
Sorry for pining you here but I'm quite lost on how to do this @rjernst ---
**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\ **Post date:** [May 6, 2019, 10:25pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/8 "2019-05-06T22:25:39Z")
You'll need to look at `@SuppressForbidden`. Additionally, some of those forbidden uses you should just correct, for example with not providing a charset. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 3, 2019, 10:25pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-ingest-plugin-which-processor-is-able-to-make-an-http-request/178311/9 "2019-06-03T22:25:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.