# Access to Elasticsearch data from an ingest plugin

**URL:** <https://discuss.elastic.co/t/access-to-elasticsearch-data-from-an-ingest-plugin/113019>\
**Category:** Elasticsearch\
**Created:** [December 22, 2017, 3:15pm UTC](https://discuss.elastic.co/t/access-to-elasticsearch-data-from-an-ingest-plugin/113019 "2017-12-22T15:15:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![uderline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/uderline/32/26245_2.png) [@uderline](https://discuss.elastic.co/u/uderline)\
**Post date:** [December 22, 2017, 3:15pm UTC](https://discuss.elastic.co/t/access-to-elasticsearch-data-from-an-ingest-plugin/113019/1 "2017-12-22T15:15:55Z")

</div>

Hello,

Using @dadoonet's article on how to make an ingest plugin in Elasticsearch 5.6, I want to be able to use [Duke](https://github.com/larsga/Duke) in a ES plugin.  
Basically, I integrated Duke-core to the ingest plugin I am programming. The main goal is to link external data with the indexed ES data.

To link the data, I import a portion of the ES data using an ES RestClient then I link it with the new data.

When I am working with my IDE (IntelliJ) and doing tests with the ESIntegTestCase class, I have no problems but when I install it on a ES server, this error appears:

```
[2017-12-22T15:30:20,912][INFO][o.e.c.r.a.AllocationService] [HO5QnH2] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[miniduke][1], [.kibana][0]] ...]).
[2017-12-22T15:30:30,655][INFO][o.e.c.m.MetaDataDeleteIndexService] [HO5QnH2] [miniduke/AtTI84rKQ22aCVPwo67vlQ] deleting index

[2017-12-22T15:30:48,313][ERROR][MiniDuke Processor] java.security.AccessControlException: access denied ("java.net.NetPermission" "getProxySelector")

[2017-12-22T15:30:48,343][INFO][o.e.c.m.MetaDataCreateIndexService] [HO5QnH2] [miniduke] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings []
[2017-12-22T15:30:48,448][INFO][o.e.c.m.MetaDataMappingService] [HO5QnH2] [miniduke/74bgexIoR4q8d4UT7x9xhQ] create_mapping [hello]

```

My first thought is that I can't access data from a RestClient because it's sort of an "external connection".

What do you think ?  
Is there another way of accessing ES's data within a plugin ?

Regards,

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 22, 2017, 4:05pm UTC](https://discuss.elastic.co/t/access-to-elasticsearch-data-from-an-ingest-plugin/113019/2 "2017-12-22T16:05:01Z")

</div>

Here is what I did:

In plugin-security.policy, I added:

```auto
grant {
    permission java.net.SocketPermission "*", "accept,connect,resolve";
    permission java.net.NetPermission "getProxySelector";
};

```

I also created this class:

```auto
    static class ClientSecurityManager {

        static <T> T doPrivilegedException(PrivilegedExceptionAction<T> operation) throws Exception {
            SpecialPermission.check();
            try {
                return AccessController.doPrivileged(operation);
            } catch (PrivilegedActionException e) {
                throw (Exception) e.getCause();
            }
        }

    }

```

When I want to call my elasticsearch Rest client from Ingest processor, I'm doing:

```auto
SearchResponse response = doPrivilegedException(() -> client.search(request));

```

Hope this helps.

---

<div class="post-metadata">

**Author:** ![uderline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/uderline/32/26245_2.png) [@uderline](https://discuss.elastic.co/u/uderline)\
**Post date:** [December 29, 2017, 10:56am UTC](https://discuss.elastic.co/t/access-to-elasticsearch-data-from-an-ingest-plugin/113019/3 "2017-12-29T10:56:50Z")

</div>

Great ! Thank you very much ! 🙂

To complete your answer:  
I upgraded to 5.6.5 - the latest version of ES 5.x.  
`SpecialPermission.check()` doesn't exist in 5.6 so I just copied the method from the v6.x.  
The class becomes:

```
static class ClientSecurityManager {
    public static final SpecialPermission INSTANCE = new SpecialPermission();

    static <T> T doPrivilegedException(PrivilegedExceptionAction<T> operation) throws Exception {
        check();
        try {
            return AccessController.doPrivileged(operation);
        } catch (PrivilegedActionException e) {
            throw (Exception) e.getCause();
        }
    }

    // Stolen from the SpecialPermission class (ES v6.x)
    public static void check() {
        SecurityManager sm = System.getSecurityManager();
        if (sm != null) {
            sm.checkPermission(INSTANCE);
        }
    }

```

`client` is a `RestHighLevelClient` object and `request` is a `SearchRequest` object

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 26, 2018, 10:57am UTC](https://discuss.elastic.co/t/access-to-elasticsearch-data-from-an-ingest-plugin/113019/4 "2018-01-26T10:57:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.