Access to Elasticsearch data from an ingest plugin

Elastic Stack Elasticsearch
1

Hello,

Using @dadoonet's article on how to make an ingest plugin in Elasticsearch 5.6, I want to be able to use Duke in a ES plugin.
Basically, I integrated Duke-core to the ingest plugin I am programming. The main goal is to link external data with the indexed ES data.

To link the data, I import a portion of the ES data using an ES RestClient then I link it with the new data.

When I am working with my IDE (IntelliJ) and doing tests with the ESIntegTestCase class, I have no problems but when I install it on a ES server, this error appears:

[2017-12-22T15:30:20,912][INFO ][o.e.c.r.a.AllocationService] [HO5QnH2] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[miniduke][1], [.kibana][0]] ...]).
[2017-12-22T15:30:30,655][INFO ][o.e.c.m.MetaDataDeleteIndexService] [HO5QnH2]    [miniduke/AtTI84rKQ22aCVPwo67vlQ] deleting index

[2017-12-22T15:30:48,313][ERROR][MiniDuke Processor       ]             java.security.AccessControlException: access denied ("java.net.NetPermission"    "getProxySelector")

[2017-12-22T15:30:48,343][INFO ][o.e.c.m.MetaDataCreateIndexService] [HO5QnH2] [miniduke] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings []
[2017-12-22T15:30:48,448][INFO ][o.e.c.m.MetaDataMappingService] [HO5QnH2] [miniduke/74bgexIoR4q8d4UT7x9xhQ] create_mapping [hello]

My first thought is that I can't access data from a RestClient because it's sort of an "external connection".

What do you think ?
Is there another way of accessing ES's data within a plugin ?

Regards,

2

Here is what I did:

In plugin-security.policy, I added:

grant {
    permission java.net.SocketPermission "*", "accept,connect,resolve";
    permission java.net.NetPermission "getProxySelector";
};

I also created this class:

    static class ClientSecurityManager {

        static <T> T doPrivilegedException(PrivilegedExceptionAction<T> operation) throws Exception {
            SpecialPermission.check();
            try {
                return AccessController.doPrivileged(operation);
            } catch (PrivilegedActionException e) {
                throw (Exception) e.getCause();
            }
        }

    }

When I want to call my elasticsearch Rest client from Ingest processor, I'm doing:

SearchResponse response = doPrivilegedException(() -> client.search(request));

Hope this helps.

1 Like
3

Great ! Thank you very much ! :slight_smile:

To complete your answer:
I upgraded to 5.6.5 - the latest version of ES 5.x.
SpecialPermission.check() doesn't exist in 5.6 so I just copied the method from the v6.x.
The class becomes:

static class ClientSecurityManager {
    public static final SpecialPermission INSTANCE = new SpecialPermission();

    static <T> T doPrivilegedException(PrivilegedExceptionAction<T> operation) throws Exception {
        check();
        try {
            return AccessController.doPrivileged(operation);
        } catch (PrivilegedActionException e) {
            throw (Exception) e.getCause();
        }
    }

    // Stolen from the SpecialPermission class (ES v6.x)
    public static void check() {
        SecurityManager sm = System.getSecurityManager();
        if (sm != null) {
            sm.checkPermission(INSTANCE);
        }
    }

client is a RestHighLevelClient object and request is a SearchRequest object

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.