Testing Query Builders (Java API)

Elastic Stack Elasticsearch
1

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

2

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?
On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

3

ah, ok. Thought this method had another usecase. How would I use that
method to produce a string?

On 10 Jan., 19:58, Shay Banon shay.ba...@elasticsearch.com wrote:

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?

On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

4

Create a builder, something like XContentFactor.jsonBuilder(), call the method:

builder = XContentFactor.jsonBuilder();
something.toXContent(builder, ToXContent.EMPTY_PARAMS);

And then call:

builder.string()

To get the json string value.
On Monday, January 10, 2011 at 9:23 PM, Ptr wrote:

ah, ok. Thought this method had another usecase. How would I use that
method to produce a string?

On 10 Jan., 19:58, Shay Banon shay.ba...@elasticsearch.com wrote:

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?

On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

5

ah, ok. Thanks a lot!!
I had to put start + end object around otherwise I got "Can not write
a field name, expecting a value". So this worked:

TermsFilterBuilder b = (TermsFilterBuilder) builder;
XContentBuilder json = jsonBuilder().startObject();
b.doXContent(json, null);
json.endObject();
System.out.println(json.string());

BTW: for what is the second unused parameter? (You used
ToXContent.EMPTY_PARAMS)
BTW2: it would be nice if doXContent would return the XContentBuilder
to make the calls a bit shorter.

On 10 Jan., 20:30, Shay Banon shay.ba...@elasticsearch.com wrote:

Create a builder, something like XContentFactor.jsonBuilder(), call the method:

builder = XContentFactor.jsonBuilder();
something.toXContent(builder, ToXContent.EMPTY_PARAMS);

And then call:

builder.string()

To get the json string value.

On Monday, January 10, 2011 at 9:23 PM, Ptr wrote:

ah, ok. Thought this method had another usecase. How would I use that
method to produce a string?

On 10 Jan., 19:58, Shay Banon shay.ba...@elasticsearch.com wrote:

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?

On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

6

ups, do vs. toXContent ... so no start + end object necessary ...

On 10 Jan., 20:41, Ptr tableyourt...@googlemail.com wrote:

ah, ok. Thanks a lot!!
I had to put start + end object around otherwise I got "Can not write
a field name, expecting a value". So this worked:

TermsFilterBuilder b = (TermsFilterBuilder) builder;
XContentBuilder json = jsonBuilder().startObject();
b.doXContent(json, null);
json.endObject();
System.out.println(json.string());

BTW: for what is the second unused parameter? (You used
ToXContent.EMPTY_PARAMS)
BTW2: it would be nice if doXContent would return the XContentBuilder
to make the calls a bit shorter.

On 10 Jan., 20:30, Shay Banon shay.ba...@elasticsearch.com wrote:

Create a builder, something like XContentFactor.jsonBuilder(), call the method:

builder = XContentFactor.jsonBuilder();
something.toXContent(builder, ToXContent.EMPTY_PARAMS);

And then call:

builder.string()

To get the json string value.

On Monday, January 10, 2011 at 9:23 PM, Ptr wrote:

ah, ok. Thought this method had another usecase. How would I use that
method to produce a string?

On 10 Jan., 19:58, Shay Banon shay.ba...@elasticsearch.com wrote:

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?

On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

7

Just pushed support for returning the builder from the toXContent method.

BTW: for what is the second unused parameter? (You used
ToXContent.EMPTY_PARAMS)

the idea is to pass external parameters that might control how it gets serialized. Currently, it is not used.

BTW2: it would be nice if doXContent would return the XContentBuilder
to make the calls a bit shorter.

You nailed that one, its toXContent
On Monday, January 10, 2011 at 9:45 PM, Ptr wrote:

ups, do vs. toXContent ... so no start + end object necessary ...

On 10 Jan., 20:41, Ptr tableyourt...@googlemail.com wrote:

ah, ok. Thanks a lot!!
I had to put start + end object around otherwise I got "Can not write
a field name, expecting a value". So this worked:

TermsFilterBuilder b = (TermsFilterBuilder) builder;
XContentBuilder json = jsonBuilder().startObject();
b.doXContent(json, null);
json.endObject();
System.out.println(json.string());

BTW: for what is the second unused parameter? (You used
ToXContent.EMPTY_PARAMS)
BTW2: it would be nice if doXContent would return the XContentBuilder
to make the calls a bit shorter.

On 10 Jan., 20:30, Shay Banon shay.ba...@elasticsearch.com wrote:

Create a builder, something like XContentFactor.jsonBuilder(), call the method:

builder = XContentFactor.jsonBuilder();
something.toXContent(builder, ToXContent.EMPTY_PARAMS);

And then call:

builder.string()

To get the json string value.

On Monday, January 10, 2011 at 9:23 PM, Ptr wrote:

ah, ok. Thought this method had another usecase. How would I use that
method to produce a string?

On 10 Jan., 19:58, Shay Banon shay.ba...@elasticsearch.com wrote:

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?

On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

8

wow. awesome reply time. awesome fix time. Thanks again Shay :slight_smile: !

On 10 Jan., 21:21, Shay Banon shay.ba...@elasticsearch.com wrote:

Just pushed support for returning the builder from the toXContent method.

BTW: for what is the second unused parameter? (You used
ToXContent.EMPTY_PARAMS)

the idea is to pass external parameters that might control how it gets serialized. Currently, it is not used.

BTW2: it would be nice if doXContent would return the XContentBuilder
to make the calls a bit shorter.

You nailed that one, its toXContent

On Monday, January 10, 2011 at 9:45 PM, Ptr wrote:

ups, do vs. toXContent ... so no start + end object necessary ...

On 10 Jan., 20:41, Ptr tableyourt...@googlemail.com wrote:

ah, ok. Thanks a lot!!
I had to put start + end object around otherwise I got "Can not write
a field name, expecting a value". So this worked:

TermsFilterBuilder b = (TermsFilterBuilder) builder;
XContentBuilder json = jsonBuilder().startObject();
b.doXContent(json, null);
json.endObject();
System.out.println(json.string());

BTW: for what is the second unused parameter? (You used
ToXContent.EMPTY_PARAMS)
BTW2: it would be nice if doXContent would return the XContentBuilder
to make the calls a bit shorter.

On 10 Jan., 20:30, Shay Banon shay.ba...@elasticsearch.com wrote:

Create a builder, something like XContentFactor.jsonBuilder(), call the method:

builder = XContentFactor.jsonBuilder();
something.toXContent(builder, ToXContent.EMPTY_PARAMS);

And then call:

builder.string()

To get the json string value.

On Monday, January 10, 2011 at 9:23 PM, Ptr wrote:

ah, ok. Thought this method had another usecase. How would I use that
method to produce a string?

On 10 Jan., 19:58, Shay Banon shay.ba...@elasticsearch.com wrote:

All Filters and Query Builders implement ToXContent, so you can generate a string representation of it by calling toXContent method. Then you can create the expected query as well, and match on strings.

Adding getters to each builder is certainly possible, its just that its going to add to the API and might make it more complex. What do you think, is the string thingy above make sense?

On Monday, January 10, 2011 at 8:54 PM, Ptr wrote:

I have a rather complex query creation and I need to unit-test it. But
at the moment this is not possible IMO.

Example:

XContentFilterBuilder builder = createXYQuery(input);
// that's ok:
assertTrue(builder instanceof TermsFilterBuilder);
TermsFilterBuilder b = (TermsFilterBuilder) builder;
// but now I need these checks:
assertEquals(1, b.terms.size());
assertEquals("test", b.terms.get(0));

Would this be possible? E.g transforming the query into a string and
check this string?

Or would I need to hack into the Java API myself and extend it (for
every queryBuilder type)?

Regards,
Peter.

9

hey shay,
seems there is a problem on dynamic mapping,after using index template,the dynamic mapping doesn't seems to work,and here is ther recreation,can you pls have a look at.

curl -XPUT http://localhost:9200/_template/temp12 -d'{"mappings":{"mytype":{"properties":{"identity":{"precision_step":4,"type":"float","boost":1.0,"include_in_all":true,"index":"analyzed","store":"no"},"datetime":{"precision_step":4,"type":"date","boost":1.0,"include_in_all":true,"index":"analyzed","store":"no"}}}},"template":"tte*","order":0,"settings":{"number_of_shards":3,"number_of_replicas":2}}'

curl -XPUT http://localhost:9200/tte124 -d'{"settings":{"number_of_shards":3,"number_of_replicas":2}}'
echo " //that works"

curl -XPUT http://localhost:9200/tte123/type/123 -d'{"a":543}'
echo "//time_out with dynamic type and data ，data with 123 will missing"

and i’ve created a issue track it,here:https://github.com/elasticsearch/elasticsearch/issues/issue/619

10

Heya,

You did not wait enough :), by default, the index timeout is 1 minute, and then you would have gotten an UnavailableShards failure. This happens because you start a single node, and create an index with 2 replicas. By default, an index request will be executed only if there is a quorum of shards active (in our case, 2). But, since we have only 1 node, we have 1 shard and no replicas allocated.

You can control the timeout by passing the timeout parameter. You can control the consistency level as well on writes.

If this make sense, and you verify it with testing (I did locally), you can close the issue. This is the expected behavior :slight_smile:

-shay.banon
On Tuesday, January 11, 2011 at 4:53 PM, medcl2000@gmail.com wrote:

hey shay,
seems there is a problem on dynamic mapping,after using index template,the dynamic mapping doesn't seems to work,and here is ther recreation,can you pls have a look at.

curl -XPUT http://localhost:9200/_template/temp12 -d'{"mappings":{"mytype":{"properties":{"identity":{"precision_step":4,"type":"float","boost":1.0,"include_in_all":true,"index":"analyzed","store":"no"},"datetime":{"precision_step":4,"type":"date","boost":1.0,"include_in_all":true,"index":"analyzed","store":"no"}}}},"template":"tte*","order":0,"settings":{"number_of_shards":3,"number_of_replicas":2}}'

curl -XPUT http://localhost:9200/tte124 -d'{"settings":{"number_of_shards":3,"number_of_replicas":2}}'
echo " //that works"

curl -XPUT http://localhost:9200/tte123/type/123 -d'{"a":543}'
echo "//time_out with dynamic type and data ，data with 123 will missing"

and i’ve created a issue track it,here:dynamic mapping doesn't work after using index template · Issue #619 · elastic/elasticsearch · GitHub