UnavailableShardsException with couchdb river

Elastic Stack Elasticsearch
1

Hi,

as per the subject I have an UnavailableShardsException with couchdb river.

this command

curl -XPUT -d @elasticsearch.json 'localhost:9200/_river/copycat_content/_meta'

with the contents of elasticsearch.json being

{
"type" : "couchdb",
"couchdb" : {
"host" : "localhost",
"port" : 5984,
"db" : "copycat_content",
"filter" : null
}
}

returns this

{"error":"UnavailableShardsException[[_river][0] [2] shardIt, [0]
active : Timeout waiting for [1m], request: index
{[_river][copycat_content][_meta], source[{ "type" : "couchdb",
"couchdb" : { "host" : "localhost", "port" :
5984, "db" : "copycat_content", "filter" : null
}}]}]","status":503}

I don't understand the sharding replication aspects yet, and haven't
had to worry much about them because I had a very small application
working with one db.

But now I have two dbs that I would like to use couchdb river with, I
suppose I have to do something to create a new index or something.

I am using elastichq, my current index info is:

Docs: 13635
Primary Size#: 11.2mb
Shards#: 5
Replicas: 1
Status: open

What should I do to be able to run these two separate dbs?

Thanks,
Bryan Rasmussen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

as per the subject I have an UnavailableShardsException with couchdb river.

I think you should first create the mapping (don't alter mapping as it
will basically works only for new fields and not on existing ones):

curl -XPUT 'http://localhost:9200/articleindex/cardiopathy/_mapping' -d '
{
"cardiopathy" : {
"file" : {
"properties" : {
"Authors" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Media" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Organization" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Keywords" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Title" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"ISSN" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Pages" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Abstracts" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"FileContent" : {
"type" : "attachment",
"fields" : {
"file" : { "indexAnalyzer": "ik","searchAnalyzer": "ik","store" : "yes",
"index" : "analyzed" },
"date" : { "store" : "yes" },
"author" : { "store" : "yes" },
"keywords" : { "store" : "yes" },
"content_type" : { "store" : "yes" },
"title" : { "store" : "yes" }
}
}
}
}
}
}'

So I guess I can just do dynamic mapping of my db, like

{
"copycat_content" : {

}
}

the reason being that I am just starting on this particular db, and
not sure on everything that it will require.

By the way, can one do something like a star field

{
"copycat_content" : {
"meta_*" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
}
}

so that meta_os and whatever other meta_ data I want to add gets indexed?

Thanks,
Bryan Rasmussen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Well I tried to define the mapping beforehand, which didn't really
seem reasonable to me since if I am getting an
unavailableShardsException
doesn't that mean I don't have an index - wouldn't you create an index
before you create a mapping for it?

Because when I do

curl -XPUT -d @elasticsearch_mapping.json
'localhost:9200/copycat_content/copycat_content/_mapping'

I get

{"error":"IndexMissingException[[copycat_content] missing]","status":404}

Thanks,
Bryan Rasmussen

On Tue, Jul 9, 2013 at 10:54 AM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

as per the subject I have an UnavailableShardsException with couchdb river.

I think you should first create the mapping (don't alter mapping as it
will basically works only for new fields and not on existing ones):

curl -XPUT 'http://localhost:9200/articleindex/cardiopathy/_mapping' -d '
{
"cardiopathy" : {
"file" : {
"properties" : {
"Authors" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Media" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Organization" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Keywords" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Title" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"ISSN" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Pages" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Abstracts" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"FileContent" : {
"type" : "attachment",
"fields" : {
"file" : { "indexAnalyzer": "ik","searchAnalyzer": "ik","store" : "yes",
"index" : "analyzed" },
"date" : { "store" : "yes" },
"author" : { "store" : "yes" },
"keywords" : { "store" : "yes" },
"content_type" : { "store" : "yes" },
"title" : { "store" : "yes" }
}
}
}
}
}
}'

So I guess I can just do dynamic mapping of my db, like

{
"copycat_content" : {

}
}

the reason being that I am just starting on this particular db, and
not sure on everything that it will require.

By the way, can one do something like a star field

{
"copycat_content" : {
"meta_*" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
}
}

so that meta_os and whatever other meta_ data I want to add gets indexed?

Thanks,
Bryan Rasmussen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Since I don't have an unavaliableShardsException on my currently
running db that is being indexed with couchdb river I'm guessing it is
a simple configuration I have to do somewhere that creates a second
shard? Since this is just my development machine I dont have a lot of
requirements, there are only 13 thousand small documents in one index
of 11.2mb, probably won't get more than 30, and this index that I want
now shouldn't end up with more than 10 thousand documents, probably
only 6mb given how small the document size is.

Thanks,
Bryan Rasmussen

On Wed, Jul 10, 2013 at 12:06 PM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

Well I tried to define the mapping beforehand, which didn't really
seem reasonable to me since if I am getting an
unavailableShardsException
doesn't that mean I don't have an index - wouldn't you create an index
before you create a mapping for it?

Because when I do

curl -XPUT -d @elasticsearch_mapping.json
'localhost:9200/copycat_content/copycat_content/_mapping'

I get

{"error":"IndexMissingException[[copycat_content] missing]","status":404}

Thanks,
Bryan Rasmussen

On Tue, Jul 9, 2013 at 10:54 AM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

as per the subject I have an UnavailableShardsException with couchdb river.

I think you should first create the mapping (don't alter mapping as it
will basically works only for new fields and not on existing ones):

curl -XPUT 'http://localhost:9200/articleindex/cardiopathy/_mapping' -d '
{
"cardiopathy" : {
"file" : {
"properties" : {
"Authors" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Media" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Organization" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Keywords" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Title" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"ISSN" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Pages" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Abstracts" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"FileContent" : {
"type" : "attachment",
"fields" : {
"file" : { "indexAnalyzer": "ik","searchAnalyzer": "ik","store" : "yes",
"index" : "analyzed" },
"date" : { "store" : "yes" },
"author" : { "store" : "yes" },
"keywords" : { "store" : "yes" },
"content_type" : { "store" : "yes" },
"title" : { "store" : "yes" }
}
}
}
}
}
}'

So I guess I can just do dynamic mapping of my db, like

{
"copycat_content" : {

}
}

the reason being that I am just starting on this particular db, and
not sure on everything that it will require.

By the way, can one do something like a star field

{
"copycat_content" : {
"meta_*" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
}
}

so that meta_os and whatever other meta_ data I want to add gets indexed?

Thanks,
Bryan Rasmussen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Also, I supposed that the use of rivers basically hid such configuration issues?
I mean that if you are able to create multiple rivers for multiple dbs
then I guess that stuff should just be taken care in the river
creation process.

Thanks,
Bryan Rasmussen

On Wed, Jul 10, 2013 at 12:16 PM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

Since I don't have an unavaliableShardsException on my currently
running db that is being indexed with couchdb river I'm guessing it is
a simple configuration I have to do somewhere that creates a second
shard? Since this is just my development machine I dont have a lot of
requirements, there are only 13 thousand small documents in one index
of 11.2mb, probably won't get more than 30, and this index that I want
now shouldn't end up with more than 10 thousand documents, probably
only 6mb given how small the document size is.

Thanks,
Bryan Rasmussen

On Wed, Jul 10, 2013 at 12:06 PM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

Well I tried to define the mapping beforehand, which didn't really
seem reasonable to me since if I am getting an
unavailableShardsException
doesn't that mean I don't have an index - wouldn't you create an index
before you create a mapping for it?

Because when I do

curl -XPUT -d @elasticsearch_mapping.json
'localhost:9200/copycat_content/copycat_content/_mapping'

I get

{"error":"IndexMissingException[[copycat_content] missing]","status":404}

Thanks,
Bryan Rasmussen

On Tue, Jul 9, 2013 at 10:54 AM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

as per the subject I have an UnavailableShardsException with couchdb river.

I think you should first create the mapping (don't alter mapping as it
will basically works only for new fields and not on existing ones):

curl -XPUT 'http://localhost:9200/articleindex/cardiopathy/_mapping' -d '
{
"cardiopathy" : {
"file" : {
"properties" : {
"Authors" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Media" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Organization" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Keywords" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Title" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"ISSN" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Pages" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Abstracts" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"FileContent" : {
"type" : "attachment",
"fields" : {
"file" : { "indexAnalyzer": "ik","searchAnalyzer": "ik","store" : "yes",
"index" : "analyzed" },
"date" : { "store" : "yes" },
"author" : { "store" : "yes" },
"keywords" : { "store" : "yes" },
"content_type" : { "store" : "yes" },
"title" : { "store" : "yes" }
}
}
}
}
}
}'

So I guess I can just do dynamic mapping of my db, like

{
"copycat_content" : {

}
}

the reason being that I am just starting on this particular db, and
not sure on everything that it will require.

By the way, can one do something like a star field

{
"copycat_content" : {
"meta_*" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
}
}

so that meta_os and whatever other meta_ data I want to add gets indexed?

Thanks,
Bryan Rasmussen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

so, thinking it over it makes most sense to do a complete deletion of
everything, and then redo it, given how small my data is?

Is there a guide somewhere on how to do a complete delete, so I won't
have anything left for a clean install?

Thanks,
Bryan Rasmussen

On Wed, Jul 10, 2013 at 12:34 PM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

Also, I supposed that the use of rivers basically hid such configuration issues?
I mean that if you are able to create multiple rivers for multiple dbs
then I guess that stuff should just be taken care in the river
creation process.

Thanks,
Bryan Rasmussen

On Wed, Jul 10, 2013 at 12:16 PM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

Since I don't have an unavaliableShardsException on my currently
running db that is being indexed with couchdb river I'm guessing it is
a simple configuration I have to do somewhere that creates a second
shard? Since this is just my development machine I dont have a lot of
requirements, there are only 13 thousand small documents in one index
of 11.2mb, probably won't get more than 30, and this index that I want
now shouldn't end up with more than 10 thousand documents, probably
only 6mb given how small the document size is.

Thanks,
Bryan Rasmussen

On Wed, Jul 10, 2013 at 12:06 PM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

Well I tried to define the mapping beforehand, which didn't really
seem reasonable to me since if I am getting an
unavailableShardsException
doesn't that mean I don't have an index - wouldn't you create an index
before you create a mapping for it?

Because when I do

curl -XPUT -d @elasticsearch_mapping.json
'localhost:9200/copycat_content/copycat_content/_mapping'

I get

{"error":"IndexMissingException[[copycat_content] missing]","status":404}

Thanks,
Bryan Rasmussen

On Tue, Jul 9, 2013 at 10:54 AM, bryan rasmussen
rasmussen.bryan@gmail.com wrote:

as per the subject I have an UnavailableShardsException with couchdb river.

I think you should first create the mapping (don't alter mapping as it
will basically works only for new fields and not on existing ones):

curl -XPUT 'http://localhost:9200/articleindex/cardiopathy/_mapping' -d '
{
"cardiopathy" : {
"file" : {
"properties" : {
"Authors" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Media" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Organization" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Keywords" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Title" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"ISSN" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Pages" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"Abstracts" : { "type" : "string" ,"indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
"FileContent" : {
"type" : "attachment",
"fields" : {
"file" : { "indexAnalyzer": "ik","searchAnalyzer": "ik","store" : "yes",
"index" : "analyzed" },
"date" : { "store" : "yes" },
"author" : { "store" : "yes" },
"keywords" : { "store" : "yes" },
"content_type" : { "store" : "yes" },
"title" : { "store" : "yes" }
}
}
}
}
}
}'

So I guess I can just do dynamic mapping of my db, like

{
"copycat_content" : {

}
}

the reason being that I am just starting on this particular db, and
not sure on everything that it will require.

By the way, can one do something like a star field

{
"copycat_content" : {
"meta_*" : {"type": "string","indexAnalyzer": "ik","searchAnalyzer":
"ik","store" : "yes"},
}
}

so that meta_os and whatever other meta_ data I want to add gets indexed?

Thanks,
Bryan Rasmussen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.