Creating an Index via Java API and JSON String

Elastic Stack Elasticsearch
1

Hi guys,

I'm trying to use the Java API to help embed ElasticSearch in my
tests. I'm seem to be getting stuck with setting the "settings" on the
index. I have a JSON String:

{
"settings":{
"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}
}
}

And I'm basically calling:

ciReq = createIndexRequest(index.name)
.settings(index.settings) //JSON String as shown above

return idxClient.create(ciReq).actionGet().acknowledged()

I've tried the JSON in a number of formats based on examples that I've
seen. But nothing seems to have an effect on the created index. The
logs always come back with something like this:

... [Oddball] creating Index [myindex], shards [5]/[1]

(I'm expecting the number in brackets to be [1]/[0], is that
correct?) And the index is not in-memory.

Any help would be much appreciated.
Thanks.

2

Turns out that wrapping it all in "settings" wasn't helping. This
seems to work:

"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}

Though, I can't tell now whether or not it's really a "memory-only"
index. For tests I would prefer it if the embedded cluster didn't
write to disk at all... Is there a way to make that happen?

Thanks.

On Oct 29, 6:05 pm, David P patri....@gmail.com wrote:

Hi guys,

I'm trying to use the Java API to help embed Elasticsearch in my
tests. I'm seem to be getting stuck with setting the "settings" on the
index. I have a JSON String:

{
"settings":{
"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}
}

}

And I'm basically calling:

ciReq = createIndexRequest(index.name)
.settings(index.settings) //JSON String as shown above

return idxClient.create(ciReq).actionGet().acknowledged()

I've tried the JSON in a number of formats based on examples that I've
seen. But nothing seems to have an effect on the created index. The
logs always come back with something like this:

... [Oddball] creating Index [myindex], shards [5]/[1]

(I'm expecting the number in brackets to be [1]/[0], is that
correct?) And the index is not in-memory.

Any help would be much appreciated.
Thanks.

3

I think that ES write few things on disk when your cluster node starts (before knowing anything about your indexes).

I use maven for my project so I set up ES to write in target/estest dir.

Not sure you can fully disable disk writing.

HTH
David :wink:

Le 30 oct. 2011 à 00:34, David P patri.dlp@gmail.com a écrit :

Turns out that wrapping it all in "settings" wasn't helping. This
seems to work:

"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}

Though, I can't tell now whether or not it's really a "memory-only"
index. For tests I would prefer it if the embedded cluster didn't
write to disk at all... Is there a way to make that happen?

Thanks.

On Oct 29, 6:05 pm, David P patri....@gmail.com wrote:

Hi guys,

I'm trying to use the Java API to help embed Elasticsearch in my
tests. I'm seem to be getting stuck with setting the "settings" on the
index. I have a JSON String:

{
"settings":{
"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}
}

}

And I'm basically calling:

ciReq = createIndexRequest(index.name)
.settings(index.settings) //JSON String as shown above

return idxClient.create(ciReq).actionGet().acknowledged()

I've tried the JSON in a number of formats based on examples that I've
seen. But nothing seems to have an effect on the created index. The
logs always come back with something like this:

... [Oddball] creating Index [myindex], shards [5]/[1]

(I'm expecting the number in brackets to be [1]/[0], is that
correct?) And the index is not in-memory.

Any help would be much appreciated.
Thanks.

4

Thanks David.

I ending up doing something similar in my SBT by configuring the
Node's paths. The only other thing I can think of to do would be to
test for the existence of the "test/data" directory and blow it away
as tests startup.

A fun note, when you use SBT for testing, it will, by default, launch
tests in parallel. Each of my test cases start up a node on "setup"
and closes it on "tearDown" (BeforeClass, AfterClass in JUnit 4
terms). So these nodes would find each other and form a cluster.
Amazingly everything was working fine, but I decided to turn off
parallel execution just in case.

  • David P.

On Oct 29, 6:49 pm, David Pilato da...@pilato.fr wrote:

I think that ES write few things on disk when your cluster node starts (before knowing anything about your indexes).

I use maven for my project so I set up ES to write in target/estest dir.

Not sure you can fully disable disk writing.

HTH
David :wink:

Le 30 oct. 2011 à 00:34, David P patri....@gmail.com a écrit :

Turns out that wrapping it all in "settings" wasn't helping. This
seems to work:

"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}

Though, I can't tell now whether or not it's really a "memory-only"
index. For tests I would prefer it if the embedded cluster didn't
write to disk at all... Is there a way to make that happen?

Thanks.

On Oct 29, 6:05 pm, David P patri....@gmail.com wrote:

Hi guys,

I'm trying to use the Java API to help embed Elasticsearch in my
tests. I'm seem to be getting stuck with setting the "settings" on the
index. I have a JSON String:

{
"settings":{
"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}
}

}

And I'm basically calling:

ciReq = createIndexRequest(index.name)
.settings(index.settings) //JSON String as shown above

return idxClient.create(ciReq).actionGet().acknowledged()

I've tried the JSON in a number of formats based on examples that I've
seen. But nothing seems to have an effect on the created index. The
logs always come back with something like this:

... [Oddball] creating Index [myindex], shards [5]/[1]

(I'm expecting the number in brackets to be [1]/[0], is that
correct?) And the index is not in-memory.

Any help would be much appreciated.
Thanks.

5

Heya,

If you want to disable the "recovery" aspect (i.e.: stop a node, start it
again, and it will recover the indices created and so on), you can the
gateway.type setting to none. It will still write to "disk", but will not
use it.

You can use different cluster names to separate tests. Even something
like UUID can work (per test). Also, to make things run faster, you can use
"local" nodes, which don't do multicast discovery / use TCP for
communication (all done "local" to the VM).

On Sun, Oct 30, 2011 at 1:22 AM, David P patri.dlp@gmail.com wrote:

Thanks David.

I ending up doing something similar in my SBT by configuring the
Node's paths. The only other thing I can think of to do would be to
test for the existence of the "test/data" directory and blow it away
as tests startup.

A fun note, when you use SBT for testing, it will, by default, launch
tests in parallel. Each of my test cases start up a node on "setup"
and closes it on "tearDown" (BeforeClass, AfterClass in JUnit 4
terms). So these nodes would find each other and form a cluster.
Amazingly everything was working fine, but I decided to turn off
parallel execution just in case.

  • David P.

On Oct 29, 6:49 pm, David Pilato da...@pilato.fr wrote:

I think that ES write few things on disk when your cluster node starts
(before knowing anything about your indexes).

I use maven for my project so I set up ES to write in target/estest dir.

Not sure you can fully disable disk writing.

HTH
David :wink:

Le 30 oct. 2011 à 00:34, David P patri....@gmail.com a écrit :

Turns out that wrapping it all in "settings" wasn't helping. This
seems to work:

"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}

Though, I can't tell now whether or not it's really a "memory-only"
index. For tests I would prefer it if the embedded cluster didn't
write to disk at all... Is there a way to make that happen?

Thanks.

On Oct 29, 6:05 pm, David P patri....@gmail.com wrote:

Hi guys,

I'm trying to use the Java API to help embed Elasticsearch in my
tests. I'm seem to be getting stuck with setting the "settings" on the
index. I have a JSON String:

{
"settings":{
"index":{
"number_of_replicas":0,
"number_of_shards":1,
"store.fs.memory.enabled":true,
"store.type":"memory"
}
}

}

And I'm basically calling:

ciReq = createIndexRequest(index.name)
.settings(index.settings) //JSON String as shown above

return idxClient.create(ciReq).actionGet().acknowledged()

I've tried the JSON in a number of formats based on examples that I've
seen. But nothing seems to have an effect on the created index. The
logs always come back with something like this:

... [Oddball] creating Index [myindex], shards [5]/[1]

(I'm expecting the number in brackets to be [1]/[0], is that
correct?) And the index is not in-memory.

Any help would be much appreciated.
Thanks.