I want to test my custom plugin for ES and I found Rally. It's a really cool tool for testing ES. I also found an article about the plugins inside the Rally documentation. But I couldn't add my custom plugin for testing. In my case I need to install 4 plugins. 3 of them are the default plugins from ES and they are easily installed when I use --elasticsearch-plugins="discovery-ec2,repository-s3,analysis-icu", but I don't know how to add my custom plugin.
Firstly, I tried to download the repository with my plugin and specify three mandatory parameters inside the ~/.rally/rally.ini:
plugin.my-plugin.src.subdir
plugin.my-plugin.build.command
plugin.my-plugin.build.artifact.subdir
But it doesn't help me. It says that I haven't this plugin.
Secondly, I would like to try to test with optional parameter (plugin.my-plugin.remote.repo.url) when we can add the url to the repository in GitHub, but I have a private repository. I also found in the Rally documentation that you supported a privated repository and I need to setup my credentials, but I didn't find where I need to setup it.
For the current moment, it sound for me like a challenge . Maybe Rally has a way to install a plugin from the .zip file as ES. (bin/elasticsearch-plugin install repository-s3). But I also didn't find any information about that.
Could you please help me with this issue? Maybe you have some ideas how to easily add my custom plugin inside the test?
To which local path did you clone your plugin? Rally expects that plugin.my-plugin.src.subdir is below src.root.dir. Ideally, you have Rally clone the repo by specifying plugin.my-plugin.remote.repo.url (which brings us to your next question).
Rally assumes that your machine is setup for passwordless access to private repos. If you use the SSH URL (e.g. git@github.com:example-org/my-plugin.git) instead of the HTTPS URL (e.g. https://github.com/example-org/my-plugin.git), git uses your private key to authenticate.
Using plugins is definitely not a feature that we use everyday ourselves so I am sure there is room for improvement and I'm glad about your feedback.
This is not directly possible; Rally does this itself after it has build your plugin. There are three supported scenarios:
You build your plugin alongside Elasticsearch from sources (see docs)
Your build your plugin but it is based on a released version of Elasticsearch (see docs)
You have published a released plugin artifact. Then you can add the download URL in rally.ini (see docs)
If all that does not work for you, there is always the possibility to setup Elasticsearch including all plugins outside the control of Rally and use Rally only as a load generator by using the pipeline benchmark-only.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
. Maybe Rally has a way to install a plugin from the .zip file as ES. (bin/elasticsearch-plugin install repository-s3). But I also didn't find any information about that.