Failed to index a document after increasing the number of replicas from 0 to 2. UnavailableShardsException was encountered

Here are my codes:


private Client client;
...

Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name", clusterName)
.put("path.conf", homePath)
.put("path.data", homePath + "/data")
.put("path.work", homePath + "/work")
.put("path.logs", homePath + "/logs")
.build();

node =
NodeBuilder.nodeBuilder().settings(settings).local(true).node();
client = node.client();

Settings indexSettings = ImmutableSettings.settingsBuilder()
.put(IndexMetaData.SETTING_NUMBER_OF_SHARDS,
shards)
.put(IndexMetaData.SETTING_NUMBER_OF_REPLICAS,
replicas)
.build();

CreateIndexRequest cir = new CreateIndexRequest(dbName);
cir.settings(indexSettings);

AdminClient adminClient = client.admin();
adminClient.indices().create(cir).actionGet();
adminClient.cluster().health(new
ClusterHealthRequest(dbName).waitForGreenStatus()).actionGet();

The setting values of shards=1 and replicas=0 works as shown in the
results below:
Search 'contact.address:B*': took 82ms, shards 1:0, hits 14
metadata.title: Albert Brown
metadata.desc: Contacts person document containing details of
Albert Brown
metadata.title: Mike Rivera
metadata.desc: Contacts person document containing details of Mike
Rivera
metadata.title: Kristina Rhoades
metadata.desc: Contacts person document containing details of
Kristina Rhoades
metadata.title: Juan Robert
metadata.desc: Contacts person document containing details of Juan
Robert
metadata.title: Alyssa Michel
metadata.desc: Contacts person document containing details of
Alyssa Michel
metadata.title: Bradley Grant
metadata.desc: Contacts person document containing details of
Bradley Grant
metadata.title: Lori Mcgrath
metadata.desc: Contacts person document containing details of Lori
Mcgrath
metadata.title: Ingrid Thigpen
metadata.desc: Contacts person document containing details of
Ingrid Thigpen
metadata.title: David Ruvalcaba
metadata.desc: Contacts person document containing details of
David Ruvalcaba
metadata.title: Heather Hogan
metadata.desc: Contacts person document containing details of
Heather Hogan
getResult = {"matches":14,"results":[{"title":"Albert
Brown","desc":"Contacts person document containing details of Albert
Brown","path":"/test_db1/UbaCrmPerson/
W2.10","type":"UbaCrmPerson","version":1},{"title":"Mike
Rivera","desc":"Contacts person document containing details of Mike
Rivera","path":"/test_db1/UbaCrmPerson/
W2.11","type":"UbaCrmPerson","version":1},{"title":"Kristina
Rhoades","desc":"Contacts person document containing details of
Kristina Rhoades","path":"/test_db1/UbaCrmPerson/
W2.20","type":"UbaCrmPerson","version":1},{"title":"Juan
Robert","desc":"Contacts person document containing details of Juan
Robert","path":"/test_db1/UbaCrmPerson/
W2.36","type":"UbaCrmPerson","version":1},{"title":"Alyssa
Michel","desc":"Contacts person document containing details of Alyssa
Michel","path":"/test_db1/UbaCrmPerson/
W2.43","type":"UbaCrmPerson","version":1},{"title":"Bradley
Grant","desc":"Contacts person document containing details of Bradley
Grant","path":"/test_db1/UbaCrmPerson/
W2.45","type":"UbaCrmPerson","version":1},{"title":"Lori
Mcgrath","desc":"Contacts person document containing details of Lori
Mcgrath","path":"/test_db1/UbaCrmPerson/
W2.54","type":"UbaCrmPerson","version":1},{"title":"Ingrid
Thigpen","desc":"Contacts person document containing details of Ingrid
Thigpen","path":"/test_db1/UbaCrmPerson/
W2.68","type":"UbaCrmPerson","version":1},{"title":"David
Ruvalcaba","desc":"Contacts person document containing details of
David Ruvalcaba","path":"/test_db1/UbaCrmPerson/
W2.70","type":"UbaCrmPerson","version":1},{"title":"Heather
Hogan","desc":"Contacts person document containing details of Heather
Hogan","path":"/test_db1/UbaCrmPerson/
W2.77","type":"UbaCrmPerson","version":1}],"count":20,"start":
0,"query":"contact.address:B*","queryId":
1333338919603,"filter":"{"must":{"contact.address":"NSW"}}"}

But when I changed the setting values of shards=1 and replicas=2
indexing fails as shown in the logs below:

Testcase: com.ubacrm.webapp.database.T008_4_SearchServlet_Test: Caused
an ERROR
UbaEnv Error: Failed saving to 'system/template/config' - [system][0]
[3] shardIt, [1] active : Timeout waiting for [1m], request: index
{[system][template][config], source[{"subsections":{"configs":{"ui":
{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"Configs","type":"string"}}},"versions":
{"ui":{"caption":"Versions","type":"Versions"},"items":-1,"fields":
{}},"system":{"ui":{"type":"GridForm"},"items":1,"fields":
{"jaasConfig":{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"authRequest":
{"macro":null,"index":null,"dv":true,"caption":"Authenticate
Requests","type":"boolean"},"userJAAS":
{"macro":null,"index":null,"dv":false,"caption":"User
JAAS","type":"boolean"},"sessionTimeout":
{"macro":null,"index":null,"dv":30,"caption":"Seeion
Timeout","type":"integer"}}},"documents":{"ui":
{"type":"GridForm"},"items":0,"fields":{"path":
{"macro":null,"index":null,"dv":"","caption":"Documents","type":"string"}}},"email":
{"ui":{"type":"GridForm"},"items":1,"fields":{"smtpHost":
{"macro":null,"index":null,"dv":"","caption":"SMTP
Host","type":"string"},"userId":
{"macro":null,"index":null,"dv":"","caption":"User
Name","type":"string"},"password":
{"macro":null,"index":null,"dv":"","caption":"Password","type":"string"}}},"roles":
{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"write":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"read":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"}}},"portfolios":{"ui":
{"type":"GridForm"},"items":0,"fields":{"system":
{"macro":null,"index":null,"dv":"","caption":"System","type":"string"},"desc":
{"macro":null,"index":null,"dv":"","caption":"Description","type":"string"},"name":
{"macro":null,"index":null,"dv":"","caption":"Name","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"Digest","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"database":
{"macro":null,"index":null,"dv":"","caption":"Database","type":"string"},"size":
{"macro":null,"index":null,"dv":
0,"caption":"Size","type":"integer"}}},"sms":{"ui":
{"type":"GridForm"},"items":1,"fields":{"message":
{"macro":null,"index":null,"dv":"","caption":"Message","type":"string"},"to":
{"macro":null,"index":null,"dv":"","caption":"To","type":"string"},"subject":
{"macro":null,"index":null,"dv":"","caption":"Subject","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"url":
{"macro":null,"index":null,"dv":"","caption":"URL","type":"string"}}},"metadata":
{"ui":{"caption":"Metadata","type":"Metadata"},"items":-1,"fields":
{}}},"macros":{},"sections":{"settings":{"subsections":
["system","email","sms"],"min":1,"sort":1,"max":
1,"caption":"Settings"},"security":{"subsections":
["portfolios","roles"],"min":1,"sort":2,"max":
1,"caption":"Security"},"depends":{"subsections":
["configs","documents"],"min":1,"sort":3,"max":
1,"caption":"Dependencies"},"info":{"subsections":
["metadata","versions"],"min":1,"sort":4,"max":
1,"caption":"Info"}},"styles":[],"lists":{},"metadata":
{"id":"config","versions":[{"by":"system","ver":
1,"date":"2012-04-02T03:26:27Z"}],"title":"config
template","system":"UbaEnv03","desc":"config
template","portfolio":"sys","owner":"system","type":"template","database":"system"}}]}
com.ubacrm.webapp.exceptions.E510_UbaEnvError_Exception: UbaEnv Error:
Failed saving to 'system/template/config' - [system][0] [3] shardIt,
[1] active : Timeout waiting for [1m], request: index {[system]
[template][config], source[{"subsections":{"configs":{"ui":
{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"Configs","type":"string"}}},"versions":
{"ui":{"caption":"Versions","type":"Versions"},"items":-1,"fields":
{}},"system":{"ui":{"type":"GridForm"},"items":1,"fields":
{"jaasConfig":{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"authRequest":
{"macro":null,"index":null,"dv":true,"caption":"Authenticate
Requests","type":"boolean"},"userJAAS":
{"macro":null,"index":null,"dv":false,"caption":"User
JAAS","type":"boolean"},"sessionTimeout":
{"macro":null,"index":null,"dv":30,"caption":"Seeion
Timeout","type":"integer"}}},"documents":{"ui":
{"type":"GridForm"},"items":0,"fields":{"path":
{"macro":null,"index":null,"dv":"","caption":"Documents","type":"string"}}},"email":
{"ui":{"type":"GridForm"},"items":1,"fields":{"smtpHost":
{"macro":null,"index":null,"dv":"","caption":"SMTP
Host","type":"string"},"userId":
{"macro":null,"index":null,"dv":"","caption":"User
Name","type":"string"},"password":
{"macro":null,"index":null,"dv":"","caption":"Password","type":"string"}}},"roles":
{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"write":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"read":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"}}},"portfolios":{"ui":
{"type":"GridForm"},"items":0,"fields":{"system":
{"macro":null,"index":null,"dv":"","caption":"System","type":"string"},"desc":
{"macro":null,"index":null,"dv":"","caption":"Description","type":"string"},"name":
{"macro":null,"index":null,"dv":"","caption":"Name","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"Digest","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"database":
{"macro":null,"index":null,"dv":"","caption":"Database","type":"string"},"size":
{"macro":null,"index":null,"dv":
0,"caption":"Size","type":"integer"}}},"sms":{"ui":
{"type":"GridForm"},"items":1,"fields":{"message":
{"macro":null,"index":null,"dv":"","caption":"Message","type":"string"},"to":
{"macro":null,"index":null,"dv":"","caption":"To","type":"string"},"subject":
{"macro":null,"index":null,"dv":"","caption":"Subject","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"url":
{"macro":null,"index":null,"dv":"","caption":"URL","type":"string"}}},"metadata":
{"ui":{"caption":"Metadata","type":"Metadata"},"items":-1,"fields":
{}}},"macros":{},"sections":{"settings":{"subsections":
["system","email","sms"],"min":1,"sort":1,"max":
1,"caption":"Settings"},"security":{"subsections":
["portfolios","roles"],"min":1,"sort":2,"max":
1,"caption":"Security"},"depends":{"subsections":
["configs","documents"],"min":1,"sort":3,"max":
1,"caption":"Dependencies"},"info":{"subsections":
["metadata","versions"],"min":1,"sort":4,"max":
1,"caption":"Info"}},"styles":[],"lists":{},"metadata":
{"id":"config","versions":[{"by":"system","ver":
1,"date":"2012-04-02T03:26:27Z"}],"title":"config
template","system":"UbaEnv03","desc":"config
template","portfolio":"sys","owner":"system","type":"template","database":"system"}}]}
at com.ubacrm.webapp.Document.save(Document.java:397)
at com.ubacrm.webapp.Template.save(Template.java:172)
at com.ubacrm.webapp.UbaEnv.createDatabase(UbaEnv.java:393)
at com.ubacrm.webapp.UbaEnv.getSystemDatabase(UbaEnv.java:351)
at
com.ubacrm.webapp.database.T008_4_SearchServlet_Test.setUpClass(T008_4_SearchServlet_Test.java:
47)
Caused by: org.elasticsearch.action.UnavailableShardsException:
[system][0] [3] shardIt, [1] active : Timeout waiting for [1m],
request: index {[system][template][config], source[{"subsections":
{"configs":{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"Configs","type":"string"}}},"versions":
{"ui":{"caption":"Versions","type":"Versions"},"items":-1,"fields":
{}},"system":{"ui":{"type":"GridForm"},"items":1,"fields":
{"jaasConfig":{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"authRequest":
{"macro":null,"index":null,"dv":true,"caption":"Authenticate
Requests","type":"boolean"},"userJAAS":
{"macro":null,"index":null,"dv":false,"caption":"User
JAAS","type":"boolean"},"sessionTimeout":
{"macro":null,"index":null,"dv":30,"caption":"Seeion
Timeout","type":"integer"}}},"documents":{"ui":
{"type":"GridForm"},"items":0,"fields":{"path":
{"macro":null,"index":null,"dv":"","caption":"Documents","type":"string"}}},"email":
{"ui":{"type":"GridForm"},"items":1,"fields":{"smtpHost":
{"macro":null,"index":null,"dv":"","caption":"SMTP
Host","type":"string"},"userId":
{"macro":null,"index":null,"dv":"","caption":"User
Name","type":"string"},"password":
{"macro":null,"index":null,"dv":"","caption":"Password","type":"string"}}},"roles":
{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"write":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"read":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"}}},"portfolios":{"ui":
{"type":"GridForm"},"items":0,"fields":{"system":
{"macro":null,"index":null,"dv":"","caption":"System","type":"string"},"desc":
{"macro":null,"index":null,"dv":"","caption":"Description","type":"string"},"name":
{"macro":null,"index":null,"dv":"","caption":"Name","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"Digest","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"database":
{"macro":null,"index":null,"dv":"","caption":"Database","type":"string"},"size":
{"macro":null,"index":null,"dv":
0,"caption":"Size","type":"integer"}}},"sms":{"ui":
{"type":"GridForm"},"items":1,"fields":{"message":
{"macro":null,"index":null,"dv":"","caption":"Message","type":"string"},"to":
{"macro":null,"index":null,"dv":"","caption":"To","type":"string"},"subject":
{"macro":null,"index":null,"dv":"","caption":"Subject","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"url":
{"macro":null,"index":null,"dv":"","caption":"URL","type":"string"}}},"metadata":
{"ui":{"caption":"Metadata","type":"Metadata"},"items":-1,"fields":
{}}},"macros":{},"sections":{"settings":{"subsections":
["system","email","sms"],"min":1,"sort":1,"max":
1,"caption":"Settings"},"security":{"subsections":
["portfolios","roles"],"min":1,"sort":2,"max":
1,"caption":"Security"},"depends":{"subsections":
["configs","documents"],"min":1,"sort":3,"max":
1,"caption":"Dependencies"},"info":{"subsections":
["metadata","versions"],"min":1,"sort":4,"max":
1,"caption":"Info"}},"styles":[],"lists":{},"metadata":
{"id":"config","versions":[{"by":"system","ver":
1,"date":"2012-04-02T03:26:27Z"}],"title":"config
template","system":"UbaEnv03","desc":"config
template","portfolio":"sys","owner":"system","type":"template","database":"system"}}]}
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction
$AsyncShardOperationAction
$3.onTimeout(TransportShardReplicationOperationAction.java:518)
at org.elasticsearch.cluster.service.InternalClusterService
$NotifyTimeout.run(InternalClusterService.java:356)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

I'm running on ES version 0.19.1. I was hoping that
waitForGreenStatus() would resolve this as I've read in other post,
but that wasn't the case for me. I'm I missing any settings/
configuration?

Regards,
Ariel

Please discard my post. I have it working now.

On Mon, Apr 2, 2012 at 12:17 PM, Ariel Mendoza remote.arielm@gmail.comwrote:

Here are my codes:


private Client client;
...

Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name", clusterName)
.put("path.conf", homePath)
.put("path.data", homePath + "/data")
.put("path.work", homePath + "/work")
.put("path.logs", homePath + "/logs")
.build();

node =
NodeBuilder.nodeBuilder().settings(settings).local(true).node();
client = node.client();

Settings indexSettings = ImmutableSettings.settingsBuilder()
.put(IndexMetaData.SETTING_NUMBER_OF_SHARDS,
shards)
.put(IndexMetaData.SETTING_NUMBER_OF_REPLICAS,
replicas)
.build();

CreateIndexRequest cir = new CreateIndexRequest(dbName);
cir.settings(indexSettings);

AdminClient adminClient = client.admin();
adminClient.indices().create(cir).actionGet();
adminClient.cluster().health(new
ClusterHealthRequest(dbName).waitForGreenStatus()).actionGet();


The setting values of shards=1 and replicas=0 works as shown in the
results below:
Search 'contact.address:B*': took 82ms, shards 1:0, hits 14
metadata.title: Albert Brown
metadata.desc: Contacts person document containing details of
Albert Brown
metadata.title: Mike Rivera
metadata.desc: Contacts person document containing details of Mike
Rivera
metadata.title: Kristina Rhoades
metadata.desc: Contacts person document containing details of
Kristina Rhoades
metadata.title: Juan Robert
metadata.desc: Contacts person document containing details of Juan
Robert
metadata.title: Alyssa Michel
metadata.desc: Contacts person document containing details of
Alyssa Michel
metadata.title: Bradley Grant
metadata.desc: Contacts person document containing details of
Bradley Grant
metadata.title: Lori Mcgrath
metadata.desc: Contacts person document containing details of Lori
Mcgrath
metadata.title: Ingrid Thigpen
metadata.desc: Contacts person document containing details of
Ingrid Thigpen
metadata.title: David Ruvalcaba
metadata.desc: Contacts person document containing details of
David Ruvalcaba
metadata.title: Heather Hogan
metadata.desc: Contacts person document containing details of
Heather Hogan
getResult = {"matches":14,"results":[{"title":"Albert
Brown","desc":"Contacts person document containing details of Albert
Brown","path":"/test_db1/UbaCrmPerson/
W2.10","type":"UbaCrmPerson","version":1},{"title":"Mike
Rivera","desc":"Contacts person document containing details of Mike
Rivera","path":"/test_db1/UbaCrmPerson/
W2.11","type":"UbaCrmPerson","version":1},{"title":"Kristina
Rhoades","desc":"Contacts person document containing details of
Kristina Rhoades","path":"/test_db1/UbaCrmPerson/
W2.20","type":"UbaCrmPerson","version":1},{"title":"Juan
Robert","desc":"Contacts person document containing details of Juan
Robert","path":"/test_db1/UbaCrmPerson/
W2.36","type":"UbaCrmPerson","version":1},{"title":"Alyssa
Michel","desc":"Contacts person document containing details of Alyssa
Michel","path":"/test_db1/UbaCrmPerson/
W2.43","type":"UbaCrmPerson","version":1},{"title":"Bradley
Grant","desc":"Contacts person document containing details of Bradley
Grant","path":"/test_db1/UbaCrmPerson/
W2.45","type":"UbaCrmPerson","version":1},{"title":"Lori
Mcgrath","desc":"Contacts person document containing details of Lori
Mcgrath","path":"/test_db1/UbaCrmPerson/
W2.54","type":"UbaCrmPerson","version":1},{"title":"Ingrid
Thigpen","desc":"Contacts person document containing details of Ingrid
Thigpen","path":"/test_db1/UbaCrmPerson/
W2.68","type":"UbaCrmPerson","version":1},{"title":"David
Ruvalcaba","desc":"Contacts person document containing details of
David Ruvalcaba","path":"/test_db1/UbaCrmPerson/
W2.70","type":"UbaCrmPerson","version":1},{"title":"Heather
Hogan","desc":"Contacts person document containing details of Heather
Hogan","path":"/test_db1/UbaCrmPerson/
W2.77","type":"UbaCrmPerson","version":1}],"count":20,"start":
0,"query":"contact.address:B*","queryId":
1333338919603,"filter":"{"must":{"contact.address":"NSW"}}"}

But when I changed the setting values of shards=1 and replicas=2
indexing fails as shown in the logs below:

Testcase: com.ubacrm.webapp.database.T008_4_SearchServlet_Test: Caused
an ERROR
UbaEnv Error: Failed saving to 'system/template/config' - [system][0]
[3] shardIt, [1] active : Timeout waiting for [1m], request: index
{[system][template][config], source[{"subsections":{"configs":{"ui":
{"type":"GridForm"},"items":0,"fields":{"name":

{"macro":null,"index":null,"dv":"","caption":"Configs","type":"string"}}},"versions":
{"ui":{"caption":"Versions","type":"Versions"},"items":-1,"fields":
{}},"system":{"ui":{"type":"GridForm"},"items":1,"fields":
{"jaasConfig":{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"authRequest":
{"macro":null,"index":null,"dv":true,"caption":"Authenticate
Requests","type":"boolean"},"userJAAS":
{"macro":null,"index":null,"dv":false,"caption":"User
JAAS","type":"boolean"},"sessionTimeout":
{"macro":null,"index":null,"dv":30,"caption":"Seeion
Timeout","type":"integer"}}},"documents":{"ui":
{"type":"GridForm"},"items":0,"fields":{"path":

{"macro":null,"index":null,"dv":"","caption":"Documents","type":"string"}}},"email":
{"ui":{"type":"GridForm"},"items":1,"fields":{"smtpHost":
{"macro":null,"index":null,"dv":"","caption":"SMTP
Host","type":"string"},"userId":
{"macro":null,"index":null,"dv":"","caption":"User
Name","type":"string"},"password":

{"macro":null,"index":null,"dv":"","caption":"Password","type":"string"}}},"roles":
{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"write":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"read":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"}}},"portfolios":{"ui":
{"type":"GridForm"},"items":0,"fields":{"system":

{"macro":null,"index":null,"dv":"","caption":"System","type":"string"},"desc":

{"macro":null,"index":null,"dv":"","caption":"Description","type":"string"},"name":

{"macro":null,"index":null,"dv":"","caption":"Name","type":"string"},"digest":

{"macro":null,"index":null,"dv":"","caption":"Digest","type":"string"},"type":

{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"database":

{"macro":null,"index":null,"dv":"","caption":"Database","type":"string"},"size":
{"macro":null,"index":null,"dv":
0,"caption":"Size","type":"integer"}}},"sms":{"ui":
{"type":"GridForm"},"items":1,"fields":{"message":

{"macro":null,"index":null,"dv":"","caption":"Message","type":"string"},"to":

{"macro":null,"index":null,"dv":"","caption":"To","type":"string"},"subject":

{"macro":null,"index":null,"dv":"","caption":"Subject","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"url":

{"macro":null,"index":null,"dv":"","caption":"URL","type":"string"}}},"metadata":
{"ui":{"caption":"Metadata","type":"Metadata"},"items":-1,"fields":
{}}},"macros":{},"sections":{"settings":{"subsections":
["system","email","sms"],"min":1,"sort":1,"max":
1,"caption":"Settings"},"security":{"subsections":
["portfolios","roles"],"min":1,"sort":2,"max":
1,"caption":"Security"},"depends":{"subsections":
["configs","documents"],"min":1,"sort":3,"max":
1,"caption":"Dependencies"},"info":{"subsections":
["metadata","versions"],"min":1,"sort":4,"max":
1,"caption":"Info"}},"styles":,"lists":{},"metadata":
{"id":"config","versions":[{"by":"system","ver":
1,"date":"2012-04-02T03:26:27Z"}],"title":"config
template","system":"UbaEnv03","desc":"config

template","portfolio":"sys","owner":"system","type":"template","database":"system"}}]}
com.ubacrm.webapp.exceptions.E510_UbaEnvError_Exception: UbaEnv Error:
Failed saving to 'system/template/config' - [system][0] [3] shardIt,
[1] active : Timeout waiting for [1m], request: index {[system]
[template][config], source[{"subsections":{"configs":{"ui":
{"type":"GridForm"},"items":0,"fields":{"name":

{"macro":null,"index":null,"dv":"","caption":"Configs","type":"string"}}},"versions":
{"ui":{"caption":"Versions","type":"Versions"},"items":-1,"fields":
{}},"system":{"ui":{"type":"GridForm"},"items":1,"fields":
{"jaasConfig":{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"authRequest":
{"macro":null,"index":null,"dv":true,"caption":"Authenticate
Requests","type":"boolean"},"userJAAS":
{"macro":null,"index":null,"dv":false,"caption":"User
JAAS","type":"boolean"},"sessionTimeout":
{"macro":null,"index":null,"dv":30,"caption":"Seeion
Timeout","type":"integer"}}},"documents":{"ui":
{"type":"GridForm"},"items":0,"fields":{"path":

{"macro":null,"index":null,"dv":"","caption":"Documents","type":"string"}}},"email":
{"ui":{"type":"GridForm"},"items":1,"fields":{"smtpHost":
{"macro":null,"index":null,"dv":"","caption":"SMTP
Host","type":"string"},"userId":
{"macro":null,"index":null,"dv":"","caption":"User
Name","type":"string"},"password":

{"macro":null,"index":null,"dv":"","caption":"Password","type":"string"}}},"roles":
{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"write":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"read":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"}}},"portfolios":{"ui":
{"type":"GridForm"},"items":0,"fields":{"system":

{"macro":null,"index":null,"dv":"","caption":"System","type":"string"},"desc":

{"macro":null,"index":null,"dv":"","caption":"Description","type":"string"},"name":

{"macro":null,"index":null,"dv":"","caption":"Name","type":"string"},"digest":

{"macro":null,"index":null,"dv":"","caption":"Digest","type":"string"},"type":

{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"database":

{"macro":null,"index":null,"dv":"","caption":"Database","type":"string"},"size":
{"macro":null,"index":null,"dv":
0,"caption":"Size","type":"integer"}}},"sms":{"ui":
{"type":"GridForm"},"items":1,"fields":{"message":

{"macro":null,"index":null,"dv":"","caption":"Message","type":"string"},"to":

{"macro":null,"index":null,"dv":"","caption":"To","type":"string"},"subject":

{"macro":null,"index":null,"dv":"","caption":"Subject","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"url":

{"macro":null,"index":null,"dv":"","caption":"URL","type":"string"}}},"metadata":
{"ui":{"caption":"Metadata","type":"Metadata"},"items":-1,"fields":
{}}},"macros":{},"sections":{"settings":{"subsections":
["system","email","sms"],"min":1,"sort":1,"max":
1,"caption":"Settings"},"security":{"subsections":
["portfolios","roles"],"min":1,"sort":2,"max":
1,"caption":"Security"},"depends":{"subsections":
["configs","documents"],"min":1,"sort":3,"max":
1,"caption":"Dependencies"},"info":{"subsections":
["metadata","versions"],"min":1,"sort":4,"max":
1,"caption":"Info"}},"styles":,"lists":{},"metadata":
{"id":"config","versions":[{"by":"system","ver":
1,"date":"2012-04-02T03:26:27Z"}],"title":"config
template","system":"UbaEnv03","desc":"config

template","portfolio":"sys","owner":"system","type":"template","database":"system"}}]}
at com.ubacrm.webapp.Document.save(Document.java:397)
at com.ubacrm.webapp.Template.save(Template.java:172)
at com.ubacrm.webapp.UbaEnv.createDatabase(UbaEnv.java:393)
at com.ubacrm.webapp.UbaEnv.getSystemDatabase(UbaEnv.java:351)
at

com.ubacrm.webapp.database.T008_4_SearchServlet_Test.setUpClass(T008_4_SearchServlet_Test.java:
47)
Caused by: org.elasticsearch.action.UnavailableShardsException:
[system][0] [3] shardIt, [1] active : Timeout waiting for [1m],
request: index {[system][template][config], source[{"subsections":
{"configs":{"ui":{"type":"GridForm"},"items":0,"fields":{"name":

{"macro":null,"index":null,"dv":"","caption":"Configs","type":"string"}}},"versions":
{"ui":{"caption":"Versions","type":"Versions"},"items":-1,"fields":
{}},"system":{"ui":{"type":"GridForm"},"items":1,"fields":
{"jaasConfig":{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"authRequest":
{"macro":null,"index":null,"dv":true,"caption":"Authenticate
Requests","type":"boolean"},"userJAAS":
{"macro":null,"index":null,"dv":false,"caption":"User
JAAS","type":"boolean"},"sessionTimeout":
{"macro":null,"index":null,"dv":30,"caption":"Seeion
Timeout","type":"integer"}}},"documents":{"ui":
{"type":"GridForm"},"items":0,"fields":{"path":

{"macro":null,"index":null,"dv":"","caption":"Documents","type":"string"}}},"email":
{"ui":{"type":"GridForm"},"items":1,"fields":{"smtpHost":
{"macro":null,"index":null,"dv":"","caption":"SMTP
Host","type":"string"},"userId":
{"macro":null,"index":null,"dv":"","caption":"User
Name","type":"string"},"password":

{"macro":null,"index":null,"dv":"","caption":"Password","type":"string"}}},"roles":
{"ui":{"type":"GridForm"},"items":0,"fields":{"name":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"},"write":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"read":
{"macro":null,"index":null,"dv":"pub","caption":"JAAS
Config","type":"string"},"digest":
{"macro":null,"index":null,"dv":"","caption":"JAAS
Config","type":"string"}}},"portfolios":{"ui":
{"type":"GridForm"},"items":0,"fields":{"system":

{"macro":null,"index":null,"dv":"","caption":"System","type":"string"},"desc":

{"macro":null,"index":null,"dv":"","caption":"Description","type":"string"},"name":

{"macro":null,"index":null,"dv":"","caption":"Name","type":"string"},"digest":

{"macro":null,"index":null,"dv":"","caption":"Digest","type":"string"},"type":

{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"database":

{"macro":null,"index":null,"dv":"","caption":"Database","type":"string"},"size":
{"macro":null,"index":null,"dv":
0,"caption":"Size","type":"integer"}}},"sms":{"ui":
{"type":"GridForm"},"items":1,"fields":{"message":

{"macro":null,"index":null,"dv":"","caption":"Message","type":"string"},"to":

{"macro":null,"index":null,"dv":"","caption":"To","type":"string"},"subject":

{"macro":null,"index":null,"dv":"","caption":"Subject","type":"string"},"type":
{"macro":null,"index":null,"dv":"","caption":"Type","type":"string"},"url":

{"macro":null,"index":null,"dv":"","caption":"URL","type":"string"}}},"metadata":
{"ui":{"caption":"Metadata","type":"Metadata"},"items":-1,"fields":
{}}},"macros":{},"sections":{"settings":{"subsections":
["system","email","sms"],"min":1,"sort":1,"max":
1,"caption":"Settings"},"security":{"subsections":
["portfolios","roles"],"min":1,"sort":2,"max":
1,"caption":"Security"},"depends":{"subsections":
["configs","documents"],"min":1,"sort":3,"max":
1,"caption":"Dependencies"},"info":{"subsections":
["metadata","versions"],"min":1,"sort":4,"max":
1,"caption":"Info"}},"styles":,"lists":{},"metadata":
{"id":"config","versions":[{"by":"system","ver":
1,"date":"2012-04-02T03:26:27Z"}],"title":"config
template","system":"UbaEnv03","desc":"config

template","portfolio":"sys","owner":"system","type":"template","database":"system"}}]}
at

org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction
$AsyncShardOperationAction
$3.onTimeout(TransportShardReplicationOperationAction.java:518)
at org.elasticsearch.cluster.service.InternalClusterService
$NotifyTimeout.run(InternalClusterService.java:356)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

I'm running on ES version 0.19.1. I was hoping that
waitForGreenStatus() would resolve this as I've read in other post,
but that wasn't the case for me. I'm I missing any settings/
configuration?

Regards,
Ariel