Plugin development 5.3 - Rest changes

Elastic Stack Elasticsearch
1

Hi all,

I'm converting my 5.2 ES plugin to 5.3 and couldn't help but notice the drastic change to the getRestHandlers method inside the ActionPlugin class. Are these changes documented somewhere or is it up to the developer to find this out?

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-5.3.html is not covering any of the plugin changes

Since the current documentation around plugins is already so minimalistic, it would be nice to get an overview of breaking changes in API with each new version regarding plugin development.

2

There is not really a stable API for plugins yet.

But there is a lot of code in elasticsearch repo to see how to implement things. That's what I am doing every time it breaks.

3

Hey David ,

Your personal blogs helped me a lot getting through the maze of plugin development in ES (so thanks a lot for that!), but I'm kinda stuck right now. I went through the example plugins in order to find some code that could point me in the right direction, but so far no luck.

Before 5.3 I could extend BaseRestHandlerin my custom RestHandler classes so that they would be picked up in the AbstractComponentLifeCycle, now with 5.3 I need to implement the RestHandler interface. This results in me not being able to inject my services into my controllers. I'm quite dependent on the guice injections on those services.

Have your tried converting your 'Adding a New REST Endpoint to Elasticsearch' example to 5.3? Or do you know a way to inject dependencies into those RestHandler classes?

4

Thank you! I appreciate the feedback and knowing that this actually helps!

I did. But there was not that many changes I saw (for my use case actually).

I'm quite dependent on the guice injections on those services.

Yeah. I think we are removing more and more guice so I believe you need to create your instances yourself and passing them to your REST classes.
No more hidden magic.

But I'm not the expert here so may be @rjernst can also help?
I believe you'll need to share more code though.

5

I'll try and share some of the code to further explain my problems:

Inside my: ExamplePlugin

The old code:

public List<Class<? extends org.elasticsearch.rest.RestHandler>> getRestHandlers() {
        List<Class<? extends org.elasticsearch.rest.RestHandler>> restHandlers = new ArrayList<>();
        restHandlers.add(MainRestHandler.class);
        restHandlers.add(StubRestHandler.class);
        restHandlers.add(StubTwoRestHandler.class);
        return restHandlers;
}

Since I had an @Inject on my RestHandlers, the two services I created (one which extends AbstractLifecycleComponent and one that extends AbstractComponent) were automatically injected. With the new version, I have to instantiate those services myself but those services are reliant on other AbstractLifecycleComponents like ClusterService

My current 5.3 version:

@Override
public List<org.elasticsearch.rest.RestHandler> getRestHandlers(Settings settings, RestController restController, ClusterSettings clusterSettings, IndexScopedSettings indexScopedSettings, SettingsFilter settingsFilter, IndexNameExpressionResolver indexNameExpressionResolver, Supplier<DiscoveryNodes> nodesInCluster) {
    return Arrays.asList(
            new MainRestHandler(settings, restController),
            new StubRestHandler(settings, restController, null, null),
            new StubTwoRestHandler(settings, restController)
    );
}

Now I need to add my services on the spots where it currently says null, but in order to do so, I need to instantiate them myself. They require Client, ClusterService but they're not available inside my ExamplePlugin class and instantiating those myself doesn't seem like the way to go.

I can't disclose much more about the functionality of the plugin, but I hope this gives a better view of what I'm trying to achieve. I'm not sure whether I'm overlooking something

6

Client is available when the rest handlers do their thing so we don't expect it to be required.

ClusterService is a thing that we don't expect REST to interact with at all - interactions with that are a thing you'd do in a transport action, hopefully.

Part of the point of providing minimal dependencies to the REST layer is to solidify our vision for what the REST layer is supposed to do.

7

It's not so much that the REST handlers/layer need the ClusterService or the Client, but the services I built need them in order to persists and check several things acrross the cluster. Maybe the previous plugin structure allowed me to come up with a design that's not really according to the standard you're proposing, but it's hard to figure out now what that structure should be.

Flow of plugin (in a nutshell):

1: User send a file to a new custom REST endpoint.
2: REST layer calls service where the Clusterservice is used to figure out what the cluster-topology is and create a specific index-mapping for the cluster (that could be replaced by the Supplier<DiscoveryNodes> nodesInCluster paramater I just noticed)
3: Service creates the index and stores the file (Client is used)
4: Send a message to all nodes in cluster that we just stored a new version of the file in our index, and order them to update their locally cached file (file can be really large and wouldn't want to send that over the wire to all nodes)
5: File is used for creation of new type of TokenFilter.

@nik9000 Would your suggestion be to change step 2&3 to create a TransportAction and handle the creation of the index & storage of the file in that handler?

8

That is very much things I'd do in a transport action, yes.

Elasticsearch needs to do all the complex stuff in transport actions so that the actions can be exposed over the transport client. This requirement will hopefully fade when the transport client goes, but that isn't for several more major releases. So REST handlers are limited to simple translation layers from json/yaml/whatever into internal requests and shipping off the request. Or at lest they should be limited to that. Or, REST handlers that we make have to be that way.

If you must do complicated stuff in the REST handler you can use createComponents to create some service and throw it into the rest handlers when they are built.

9

Thanks for the tips @nik9000

I'll have to check whether rewriting the whole structure of the plugin takes more time than using your proposed example, but as the plugin needs to be up-to-date with the latest version of Elasticsearch I'm afraid this will mean a partial re-write. Thanks again @nik9000 and @dadoonet for your inputs

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.