Perl API call to new() is slow

Elastic Stack Elasticsearch
1

I've written a number of Perl clients using the Perl API and have done some profiling. I've found that the call to Search::Elasticsearch::new() can take more than 50% of the total execution time. I'd like to speed this up some how. Any suggestions?

One thought I had was to have a proxy application that keeps an instance of an ES object and then some how pass that object to my clients so they don't have to instantiate their own. Is that possible some how? Or maybe there's a better solution.

Thanks,

2

Yes, you must create a Search::Elasticsearch->new() instance only once.

See the tests in https://github.com/elastic/elasticsearch-perl/tree/master/t for examples.

3

The Perl client uses Moo, which provides a lot of flexibility but does have a small start up cost. Typically most applications would create an instance once then reuse it many times. The same approach should be applied for pretty much all other Perl modules as well. Why are you creating new instances frequently?

4

I have a rather large and complex bash script which is used in one of our production systems. The bash script make many calls to the perl scripts. While I could reorganize the bash script and perl scripts so that only one call to a single perl script is made, this would be a significant change to our configuration and would move most of the functionality out of our current bash script. I was hoping there was some way to "save" the ES instance (may via Storable) so that each perl script doesn't have to instantiate their own. Or maybe have a proxy perl script daemon which can serve the ES instance to the individual scripts.

5

You're launching multiple Perl processes from bash and you're concerned about performance? :smile:

Loading Perl and creating a new Search::Elasticsearch instance takes all of 208ms on my laptop:

> time perl -MSearch::Elasticsearch -e '$a=Search::Elasticsearch->new'

real	0m0.208s
user	0m0.171s
sys	0m0.020s

If that is really too long for you, try Elastijk instead

6

Hmm, well thanks for your input anyways. And thanks for your work with the API, it's really useful.