How to setup unit testing for elastissearch configuration

Elastic Stack Elasticsearch
1

Hi

I'm looking into elasticsearch and do I since I do not understand all the
aspects of it yet, I want to have as short feedback loops as possible.
So I want to setup UT where I test my assumptions of ES. This part looks
ok. But if I use the java api to do this changes, and when I want to have
this configuration on a test or a production env, where I want ES to be a
standalone application. How do I do this the best way? I see that on
elasticsearch.org the common way suggesting is to have the configuration
in a elasticsearch.yml file.
I was thinking that it was better to have all the configuration in the
"elasticsearch-config" app, and when I run this application it deploys the
config I have tested to a clean elasticsearch server.
I tried to google and look in the google groups to see if someone has done
this before, but I can't see anyone that has thought the same as me.

Do this sound like a bad idea, or do there exist some kind of best
practice on how to do this? (UT the config code and use the same config in
a standalone ES instance)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I actually start Elasticsearch embedded and configure it using
System.setProperty. Anything you can define in the yml, you can also
override with -D or using system properties (you have to prepend "es.:
though.

This way, you don't need any pre-existing infrastructure/configuration to
run your tests. Using the system properties, I set it to use in memory
storage, use a custom node name, and disable clustering, etc.

In my unit tests, I actually make sure that Elasticsearch is started only
once and my tests use randomized data (ids etc.) so I don't have to wipe
the index in between tests. I'd recommend anyone to do the same since this
speeds things up quite a bit.

Here's my launcher class if you are interested.

I actually use this launcher in production as well but with different
system properties. The nice thing about this is that the jars are already
there because they are bundled with my application and I don't need a
separate Elasticsearch package. Also, I can control which version of
Elasticsearch rolls out using my application pom file.

Jilles

On Tuesday, March 5, 2013 4:27:20 PM UTC+1, Sindre Svendby wrote:

Hi

I'm looking into elasticsearch and do I since I do not understand all the
aspects of it yet, I want to have as short feedback loops as possible.
So I want to setup UT where I test my assumptions of ES. This part looks
ok. But if I use the java api to do this changes, and when I want to have
this configuration on a test or a production env, where I want ES to be a
standalone application. How do I do this the best way? I see that on
elasticsearch.org the common way suggesting is to have the configuration
in a elasticsearch.yml file.
I was thinking that it was better to have all the configuration in the
"elasticsearch-config" app, and when I run this application it deploys the
config I have tested to a clean elasticsearch server.
I tried to google and look in the google groups to see if someone has done
this before, but I can't see anyone that has thought the same as me.

Do this sound like a bad idea, or do there exist some kind of best
practice on how to do this? (UT the config code and use the same config in
a standalone ES instance)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

I work with Python and pyes and I actually managed to have a quite nice UT
system to test my assumptions about ES.

I think you almost never need to use elasticsearch.yml, because you can
still set up everything at run time with newly created indexes.
This for example

class TempIndex:
def init(self, name=None, settings=None, empty=False):
self.name = name or "index" + uuid4().hex
self.settings = settings or {}
self.empty = empty

def __enter__(self):
    ES_CON.create_index(self.name, settings=self.settings)
    if not self.empty:
        populate(ES_CON, self.name)

    return self.name

def __exit__(self, type, value, traceback):
    ES_CON.delete_index(self.name)

creates a temporary fresh index, to which I can pass some settings and
mappings and populates it.
with TempIndex(settings={'analyzer': 'xyz'}) as index:
es_con.search(StringQuery(...

and there check all my assumptions.
Works pretty well and does not depend on any specific machine configuration.

On Tuesday, March 5, 2013 3:27:20 PM UTC, Sindre Svendby wrote:

Hi

I'm looking into elasticsearch and do I since I do not understand all the
aspects of it yet, I want to have as short feedback loops as possible.
So I want to setup UT where I test my assumptions of ES. This part looks
ok. But if I use the java api to do this changes, and when I want to have
this configuration on a test or a production env, where I want ES to be a
standalone application. How do I do this the best way? I see that on
elasticsearch.org the common way suggesting is to have the configuration
in a elasticsearch.yml file.
I was thinking that it was better to have all the configuration in the
"elasticsearch-config" app, and when I run this application it deploys the
config I have tested to a clean elasticsearch server.
I tried to google and look in the google groups to see if someone has done
this before, but I can't see anyone that has thought the same as me.

Do this sound like a bad idea, or do there exist some kind of best
practice on how to do this? (UT the config code and use the same config in
a standalone ES instance)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.