Java API and controlling node data directory

Elastic Stack Elasticsearch
1

Hello.

I'm using Elastic Search Java API so that Elastic Search nodes can be
embedded in my application.
And I have some questions:

  1. How to change default data directory, where node stores its data?
    It is always created in current working directory as "data". Is there
    a setting to control this behaviour?
  2. How to disable gateway/translogs/data storing completely (using
    java API), so that Elastic Search won't create any directories on hard
    drive and keep everything in memory only?
  3. Is there a possibility of defining recursive data mapping? I'd like
    to store in index an object, which can contain other objects of the
    same type as properties, and they also can contain these objects as
    properties etc.

Regards
Wojciech Durczyński

2

In general, to further configure ES when embedded in Java App is either by passing settings programmatically to the NodeBuilder, or having an elasticsearch.(yml|json) configuration within the classpath under / or /config.

  1. In order to change the data directory location, set the path.data setting to the location you want it to be stored at.

  2. In order to have the index be stored in memory (and not survive restarts), set index.store.type to memory.

  3. You can define template mappings.
    On Tuesday, December 14, 2010 at 6:05 PM, Wojciech Durczyński wrote:

Hello.

I'm using Elastic Search Java API so that Elastic Search nodes can be
embedded in my application.
And I have some questions:

  1. How to change default data directory, where node stores its data?
    It is always created in current working directory as "data". Is there
    a setting to control this behaviour?
  2. How to disable gateway/translogs/data storing completely (using
    java API), so that Elastic Search won't create any directories on hard
    drive and keep everything in memory only?
  3. Is there a possibility of defining recursive data mapping? I'd like
    to store in index an object, which can contain other objects of the
    same type as properties, and they also can contain these objects as
    properties etc.

Regards
Wojciech Durczyński

3

Thank you for quick answer.

I set "index.store.type" to "memory" in context of whole node and
additionally during creation of particular index.
And the problem is that directories are still created, and they
contain files translog-* and node.lock.
Is there a possibility to disable creation of those files also?

I set also "gateway.type" to "none" and sometimes despite of this
setting (and index.store.type=memory) I have an exception:
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(ClusterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

On Dec 15, 5:06 am, Shay Banon shay.ba...@elasticsearch.com wrote:

In general, to further configure ES when embedded in Java App is either by passing settings programmatically to the NodeBuilder, or having an elasticsearch.(yml|json) configuration within the classpath under / or /config.

  1. In order to change the data directory location, set the path.data setting to the location you want it to be stored at.

  2. In order to have the index be stored in memory (and not survive restarts), set index.store.type to memory.

  3. You can define template mappings.

On Tuesday, December 14, 2010 at 6:05 PM, Wojciech Durczyński wrote:

Hello.

I'm using Elastic Search Java API so that Elastic Search nodes can be
embedded in my application.
And I have some questions:

  1. How to change default data directory, where node stores its data?
    It is always created in current working directory as "data". Is there
    a setting to control this behaviour?
  2. How to disable gateway/translogs/data storing completely (using
    java API), so that Elastic Search won't create any directories on hard
    drive and keep everything in memory only?
  3. Is there a possibility of defining recursive data mapping? I'd like
    to store in index an object, which can contain other objects of the
    same type as properties, and they also can contain these objects as
    properties etc.

Regards
Wojciech Durczyński

4

No, those files will always get created.

2010/12/15 Wojciech Durczyński wojciech.durczynski@comarch.com

Thank you for quick answer.

I set "index.store.type" to "memory" in context of whole node and
additionally during creation of particular index.
And the problem is that directories are still created, and they
contain files translog-* and node.lock.
Is there a possibility to disable creation of those files also?

I set also "gateway.type" to "none" and sometimes despite of this
setting (and index.store.type=memory) I have an exception:
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at

org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(ClusterBlocks.java:
112)
at

org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:
65)
at

org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

On Dec 15, 5:06 am, Shay Banon shay.ba...@elasticsearch.com wrote:

In general, to further configure ES when embedded in Java App is either
by passing settings programmatically to the NodeBuilder, or having an
elasticsearch.(yml|json) configuration within the classpath under / or
/config.

  1. In order to change the data directory location, set the path.data
    setting to the location you want it to be stored at.

  2. In order to have the index be stored in memory (and not survive
    restarts), set index.store.type to memory.

  3. You can define template mappings.

On Tuesday, December 14, 2010 at 6:05 PM, Wojciech Durczyński wrote:

Hello.

I'm using Elastic Search Java API so that Elastic Search nodes can be
embedded in my application.
And I have some questions:

  1. How to change default data directory, where node stores its data?
    It is always created in current working directory as "data". Is there
    a setting to control this behaviour?
  2. How to disable gateway/translogs/data storing completely (using
    java API), so that Elastic Search won't create any directories on hard
    drive and keep everything in memory only?
  3. Is there a possibility of defining recursive data mapping? I'd like
    to store in index an object, which can contain other objects of the
    same type as properties, and they also can contain these objects as
    properties etc.

Regards
Wojciech Durczyński