I am preparing to write an ES plugin for Complex Event Processing using Esper (http://www.espertech.com). I've searched for resources regarding writing plugins but found only basic tutorials.
I have some doubts whether it is possible to have some features using plugin API at all, so before I start going trough code I'd like to ask someone with experience whether it is possible to do some things.
- The most value added is in the Pattern functionality and yielding composite events - aggregations can be realized with native ES func so there is lesser priority to have them.
- Pattern is stateful, needs recovery and routing, parallel execution can be achieved using Context (session, device, etc.) and routing, otherwise with global patterns one Node has to be appointed to be a 'pattern executor'.
- Patterns have to be possible to recover after Node reboot or replicated for durability but only using standard ES architecture.
- As with any synthetic data it is easy to flood storage, so additional settings have to be available as max target index size which triggers cleanup.
So, can I do the following with only a plugin:
- attaching custom objects to Shard or having them in shard scope and lifecycle;
- attaching custom objects to Node or having them in node scope and lifecycle;
- custom routing (to Node or Shard);
- putting document (event) into index without any request, triggered by timer;
- executing custom function on node failure/recovery/reboot.
Any materials regarding architecture or coding for Elastic will be appreciated.
Also, thank you for this great tool!