Best practices: testing, cluster configuration, data analysis integrity, query results

Elastic Stack Elasticsearch
1

Hi,

I would like to ask how do people do testing of
Elasticsearch setup/configuration, especially in Java.

For example, some search operations can be quite sensitive to even little
mappings and analysis changes, like suggestions for example. Are there any
common practices how to best validate setup configurations in an automated
way. Are there already any tools that can help in testing process as
opposed to home-grown solution? Extensions to common Java testing
frameworks?

For example, starting two or three node cluster, creating indices, setting
up mappings and analysis, indexing sample data and then running queries and
validating results. This can be a lot of work, are there already any tools
that can make this more automated/declarative/less tedious? Preferably
maven friendly.

Thanks,
Lukas

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I use TestNG and adapted the ES test code that is provided in the ES source.
+1

Right now I'm stuck with indexing tests, not query tests, but I'd like to.

In the JDBC river I have added TestNG XML parameter files and I use
Maven Profiles to test against different database drivers, including
starting the river, fetching data to ES, and counting the results.

My ideas of a cool test framework are

  • a collection of URLs to downloadable free test data sets (packed
    JSON?) plus pre-defined settings/mappings (and their plugin dependencies)
  • a collection of test / workload scenarios to select from
  • a test tool for the ES 'bin' directory to execute tests, with results
    also in JSON
  • option to upload test results to the web, for ranking/comparison (to
    assist in detecting performance regressions)
  • graphs, graphs, graphs...

Jörg

Am 22.03.13 17:35, schrieb Lukáš Vlček:

Hi,

I would like to ask how do people do testing of
Elasticsearch setup/configuration, especially in Java.

For example, some search operations can be quite sensitive to even
little mappings and analysis changes, like suggestions for example.
Are there any common practices how to best validate setup
configurations in an automated way. Are there already any tools that
can help in testing process as opposed to home-grown solution?
Extensions to common Java testing frameworks?

For example, starting two or three node cluster, creating indices,
setting up mappings and analysis, indexing sample data and then
running queries and validating results. This can be a lot of work, are
there already any tools that can make this more
automated/declarative/less tedious? Preferably maven friendly.

Thanks,
Lukas

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Lukas,

Same as Jorg, for specific tests I also adapt the code provided in ES
source.

For most common cases (create/delete indices or mapping, bulk indexing
data, etc) , I use a small test framework I developed:
GitHub - tlrx/elasticsearch-test: elasticsearch-test, a framework that makes elasticsearch unit testing a breeze. . Works fine with JUnit and
TestNG, you can see how it is used here:
https://github.com/tlrx/elasticsearch-test/blob/master/src/test/java/com/github/tlrx/elasticsearch/test/EsSetupTest.java

Hope this helps,

-- Tanguy

Le vendredi 22 mars 2013 17:35:22 UTC+1, Lukáš Vlček a écrit :

Hi,

I would like to ask how do people do testing of
Elasticsearch setup/configuration, especially in Java.

For example, some search operations can be quite sensitive to even little
mappings and analysis changes, like suggestions for example. Are there any
common practices how to best validate setup configurations in an automated
way. Are there already any tools that can help in testing process as
opposed to home-grown solution? Extensions to common Java testing
frameworks?

For example, starting two or three node cluster, creating indices, setting
up mappings and analysis, indexing sample data and then running queries and
validating results. This can be a lot of work, are there already any tools
that can make this more automated/declarative/less tedious? Preferably
maven friendly.

Thanks,
Lukas

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks,

Tanguy, your test tool looks handy. I will have a look.

Regards,
Lukas

On Mon, Mar 25, 2013 at 10:40 AM, Tanguy tlrx.dev@gmail.com wrote:

Hi Lukas,

Same as Jorg, for specific tests I also adapt the code provided in ES
source.

For most common cases (create/delete indices or mapping, bulk indexing
data, etc) , I use a small test framework I developed:
GitHub - tlrx/elasticsearch-test: elasticsearch-test, a framework that makes elasticsearch unit testing a breeze. . Works fine with JUnit and
TestNG, you can see how it is used here:

https://github.com/tlrx/elasticsearch-test/blob/master/src/test/java/com/github/tlrx/elasticsearch/test/EsSetupTest.java

Hope this helps,

-- Tanguy

Le vendredi 22 mars 2013 17:35:22 UTC+1, Lukáš Vlček a écrit :

Hi,

I would like to ask how do people do testing of Elasticsearch setup/**configuration,
especially in Java.

For example, some search operations can be quite sensitive to even little
mappings and analysis changes, like suggestions for example. Are there any
common practices how to best validate setup configurations in an automated
way. Are there already any tools that can help in testing process as
opposed to home-grown solution? Extensions to common Java testing
frameworks?

For example, starting two or three node cluster, creating indices,
setting up mappings and analysis, indexing sample data and then running
queries and validating results. This can be a lot of work, are there
already any tools that can make this more automated/declarative/less
tedious? Preferably maven friendly.

Thanks,
Lukas

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Basically, you can call the Elasticsearch main class and configure it
using system properties (e.g. System.setProperty(name,value).

        NodeBuilder nodeBuilder = NodeBuilder.nodeBuilder().settings(

EMPTY_SETTINGS).loadConfigSettings(false);
node = nodeBuilder.build();
// register a shutdown hook
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
node.close();
}
});

        node.start();

I have an ESLauncherClass that does this and an corresponding initializing
spring bean that uses it after Spring finishes loading. This setup allows
me to inject a running es node into my spring test configuration and use it
over the network. Apparently you can have several embedded nodes if you
want to run a cluster in your test setup.

Jilles

On Friday, March 22, 2013 5:35:22 PM UTC+1, Lukáš Vlček wrote:

Hi,

I would like to ask how do people do testing of
Elasticsearch setup/configuration, especially in Java.

For example, some search operations can be quite sensitive to even little
mappings and analysis changes, like suggestions for example. Are there any
common practices how to best validate setup configurations in an automated
way. Are there already any tools that can help in testing process as
opposed to home-grown solution? Extensions to common Java testing
frameworks?

For example, starting two or three node cluster, creating indices, setting
up mappings and analysis, indexing sample data and then running queries and
validating results. This can be a lot of work, are there already any tools
that can make this more automated/declarative/less tedious? Preferably
maven friendly.

Thanks,
Lukas

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.