Different objects under same index = IndexAlreadyExistsException[

Elastic Stack Elasticsearch
1

Hi all.

Sorry if this looks like a silly question or error of concept.

Scenario:
I've created an index /directorio/usuario and now I want to create/directorio/grupo(and maybe to keep extending
/directorio later on).

I've generated the directorio index and populated it with usuario objects.
Now I'm trying to create the mapping for the grupo objects

{
"mappings": {
"grupo":{
"properties":{
"allow_sex":{
"type":"string", "index" : "not_analyzed"
},
"avatar":{
"type":"string", "index" : "not_analyzed"
},
"description":{
"type":"string", "index" : "not_analyzed"
},
"id":{
"type":"integer"
},
"is_open":{
"type":"boolean"
},
"members_qty":{
"type":"integer"
},
"name":{
"type":"string"
},
"user_owner":{
"type":"string", "index" : "not_analyzed"
}
}
}
}
}

That throws "error":"IndexAlreadyExistsException[[directorio] Already
exists]",
*Questions: *

  • Is it possible to create the mapping for another object on an existing
    index? - Is it fine to have two different objects under the same index?
    I've chosen that structure just for the sake of having a "directory" which
    has "users", "groups", etc... - Currently we have a bit over 1 million
    users and growing around 2K/day - Groups are just a bunch so far, but
    eventually will start growing fast.
    Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

If the index already exists and you want to make changes, you must update
the mappings, and also update the settings if you need additional analyzers
and such for the new mappings.

Mappings cannot conflict (in general; I've never wanted to override that
warning), making it easy to add new types and to add new fields to existing
types.

The API for updating an existing index is a little different than for
creating it: When creating the index I generated the complete
settings+mappings as one JSON document and submitted it (via the Java API,
but the curl API also accepts it). For updating, I had to submit the index
settings and the document mappings as two separate JSON documents. It took
a little fiddling to get it right, but it works very nicely.

On Tuesday, April 30, 2013 4:13:17 PM UTC-4, Fernando Javier Martin wrote:

Hi all.

Sorry if this looks like a silly question or error of concept.

Scenario:
I've created an index /directorio/usuario and now I want to create/directorio/grupo(and maybe to keep extending
/directorio later on).

I've generated the directorio index and populated it with usuario objects.
Now I'm trying to create the mapping for the grupo objects

{
"mappings": {
"grupo":{
"properties":{
"allow_sex":{
"type":"string", "index" : "not_analyzed"
},
"avatar":{
"type":"string", "index" : "not_analyzed"
},
"description":{
"type":"string", "index" : "not_analyzed"
},
"id":{
"type":"integer"
},
"is_open":{
"type":"boolean"
},
"members_qty":{
"type":"integer"
},
"name":{
"type":"string"
},
"user_owner":{
"type":"string", "index" : "not_analyzed"
}
}
}
}
}

That throws "error":"IndexAlreadyExistsException[[directorio] Already
exists]",
*Questions: *

  • Is it possible to create the mapping for another object on an existing
    index? - Is it fine to have two different objects under the same index?
    I've chosen that structure just for the sake of having a "directory" which
    has "users", "groups", etc... - Currently we have a bit over 1 million
    users and growing around 2K/day - Groups are just a bunch so far, but
    eventually will start growing fast.
    Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Thanks InquiringMind.
I got what you said but I'm still unable to make it work.
You'll see my current and desired mappings in this
gist: gist:5502715 · GitHub

These are pretty basic indices and not for complex text parsing. As you can
guess these indices will have litttle to nothing text analysis but huge
number of updates and searches.
Anyway, I'm now in doubt if the huge number of records could slow down the
updates and retrieval.
Maybe having completely differente indices like /userdir/user and
/groupdir/group and possibly later /whateverdir/whatever

BTW, I can't find specific documentation on the PUT Mapping API.
This one (
Elasticsearch Platform — Find real-time answers at scale | Elastic)
doesn't provides much information or I'm missing something else.
Could you please give me some guidance on this?

Thanks in advance.
Fernando

El martes, 30 de abril de 2013 17:20:48 UTC-3, InquiringMind escribió:

If the index already exists and you want to make changes, you must update
the mappings, and also update the settings if you need additional analyzers
and such for the new mappings.

Mappings cannot conflict (in general; I've never wanted to override that
warning), making it easy to add new types and to add new fields to existing
types.

The API for updating an existing index is a little different than for
creating it: When creating the index I generated the complete
settings+mappings as one JSON document and submitted it (via the Java API,
but the curl API also accepts it). For updating, I had to submit the index
settings and the document mappings as two separate JSON documents. It took
a little fiddling to get it right, but it works very nicely.

On Tuesday, April 30, 2013 4:13:17 PM UTC-4, Fernando Javier Martin wrote:

Hi all.

Sorry if this looks like a silly question or error of concept.

Scenario:
I've created an index /directorio/usuario and now I want to create/directorio/grupo(and maybe to keep extending
/directorio later on).

I've generated the directorio index and populated it with usuario
objects.
Now I'm trying to create the mapping for the grupo objects

{
"mappings": {
"grupo":{
"properties":{
"allow_sex":{
"type":"string", "index" : "not_analyzed"
},
"avatar":{
"type":"string", "index" : "not_analyzed"
},
"description":{
"type":"string", "index" : "not_analyzed"
},
"id":{
"type":"integer"
},
"is_open":{
"type":"boolean"
},
"members_qty":{
"type":"integer"
},
"name":{
"type":"string"
},
"user_owner":{
"type":"string", "index" : "not_analyzed"
}
}
}
}
}

That throws "error":"IndexAlreadyExistsException[[directorio]
Already exists]",
*Questions: *

  • Is it possible to create the mapping for another object on an existing
    index? - Is it fine to have two different objects under the same index?
    I've chosen that structure just for the sake of having a "directory" which
    has "users", "groups", etc... - Currently we have a bit over 1 million
    users and growing around 2K/day - Groups are just a bunch so far, but
    eventually will start growing fast.
    Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Fernando,

Here's what I've learned, based on some excellent help from this newsgroup
and some additional experimentation of my own. Note that I pass the
settings and mappings into the Java API as JSON strings. The trick, I
found, was to get the JSON structure correct.

The full JSON settings and mappings as passed to the
CreateIndexRequestBuilder.setSource(String) method:

{
"settings" : {
"index" : {
"number_of_shards" : 1,
"refresh_interval" : "1s",
"analysis" : {
"char_filter" : { },
"filter" : {
"english_snowball_filter" : {
"type" : "snowball",
"language" : "English"
}
},
"analyzer" : {
"english_stemming_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding",
"english_snowball_filter" ]
},
"english_standard_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding" ]
}
}
}
}
},
"mappings" : {
"ghost" : {
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true,
"default" : "1.9m"
},
"properties" : {
"cn" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer"
},
"text" : {
"type" : "multi_field",
"fields" : {
"text" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer",
"position_offset_gap" : 4
},
"std" : {
"type" : "string",
"analyzer" : "english_standard_analyzer",
"position_offset_gap" : 4
}
}
}
}
},
"elf" : {
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true
},
"properties" : {
"cn" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer"
},
"text" : {
"type" : "multi_field",
"fields" : {
"text" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer",
"position_offset_gap" : 4
},
"std" : {
"type" : "string",
"analyzer" : "english_standard_analyzer",
"position_offset_gap" : 4
}
}
}
}
}
}
}

The JSON settings as passed to the UpdateSettingsRequestBuilder.setSettings(String)
method:

{
"index" : {
"refresh_interval" : "1s",
"analysis" : {
"char_filter" : { },
"filter" : {
"english_snowball_filter" : {
"type" : "snowball",
"language" : "English"
}
},
"analyzer" : {
"english_stemming_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding",
"english_snowball_filter" ]
},
"english_standard_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding" ]
}
}
}
}
}

The JSON mapping as passed to the PutMappingRequestBuilder.setSource(String)method for the specified document type name.

{
"elf" : {
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true
},
"properties" : {
"cn" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer"
},
"text" : {
"type" : "multi_field",
"fields" : {
"text" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer",
"position_offset_gap" : 4
},
"std" : {
"type" : "string",
"analyzer" : "english_standard_analyzer",
"position_offset_gap" : 4
}
}
}
}
}
}

I hope this helps. (Sorry for the delay: The Elasticsearch 0.90 live
webinar took place from noon to 1PM EST, followed by lunch. It was an
excellent presentation!)

On Thursday, May 2, 2013 11:04:30 AM UTC-4, Fernando Javier Martin wrote:

Thanks InquiringMind.
I got what you said but I'm still unable to make it work.
You'll see my current and desired mappings in this gist:
gist:5502715 · GitHub

These are pretty basic indices and not for complex text parsing. As you
can guess these indices will have litttle to nothing text analysis but huge
number of updates and searches.
Anyway, I'm now in doubt if the huge number of records could slow down the
updates and retrieval.
Maybe having completely differente indices like /userdir/user and
/groupdir/group and possibly later /whateverdir/whatever

BTW, I can't find specific documentation on the PUT Mapping API.
This one (
Elasticsearch Platform — Find real-time answers at scale | Elastic)
doesn't provides much information or I'm missing something else.
Could you please give me some guidance on this?

Thanks in advance.
Fernando

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hi InquiringMind,
I'm glad you enjoyed the webinar.

Unfortunately I was unable to get it. I keep getting the IndexAlreadyExists
error.
I tried with this:
{
"grupo":{
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true
},
"properties":{
"allow_sex":{
"type":"string", "index" : "not_analyzed"
},
"avatar":{
"type":"string", "index" : "not_analyzed"
},
"description":{
"type":"string", "index" : "not_analyzed"
},
"id":{
"type":"integer"
},
"is_open":{
"type":"boolean"
},
"members_qty":{
"type":"integer"
},
"name":{
"type":"string"
},
"user_owner":{
"type":"string", "index" : "not_analyzed"
}
}
}
}

And this:
{
"mappings": {
"usuario":{
"properties":{
"avatar":{
"type":"string", "index" : "not_analyzed"
},
"friends":{
"type":"string", "index" : "not_analyzed"
},
"friends_qty":{
"type":"integer"
},
"iduser":{
"type":"integer"
},
"username":{
"type":"string", "index" : "not_analyzed"
}
}
},
"grupo":{
"properties":{
"allow_sex":{
"type":"string", "index" : "not_analyzed"
},
"avatar":{
"type":"string", "index" : "not_analyzed"
},
"description":{
"type":"string", "index" : "not_analyzed"
},
"id":{
"type":"integer"
},
"is_open":{
"type":"boolean"
},
"members_qty":{
"type":"integer"
},
"name":{
"type":"string"
},
"user_owner":{
"type":"string", "index" : "not_analyzed"
}
}
}
}
}

Neither worked.

I must to move on. I 'm stuck and quite late; I can handle it in another
way by using separate indexes as I stated above.
Anyway, thanks for your time.

Hopefully sooner or later I'll get the book or any other kind of course.
Thanks for your time.

El jueves, 2 de mayo de 2013 15:15:12 UTC-3, InquiringMind escribió:

Fernando,

Here's what I've learned, based on some excellent help from this newsgroup
and some additional experimentation of my own. Note that I pass the
settings and mappings into the Java API as JSON strings. The trick, I
found, was to get the JSON structure correct.

The full JSON settings and mappings as passed to the
CreateIndexRequestBuilder.setSource(String) method:

{
"settings" : {
"index" : {
"number_of_shards" : 1,
"refresh_interval" : "1s",
"analysis" : {
"char_filter" : { },
"filter" : {
"english_snowball_filter" : {
"type" : "snowball",
"language" : "English"
}
},
"analyzer" : {
"english_stemming_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding",
"english_snowball_filter" ]
},
"english_standard_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding" ]
}
}
}
}
},
"mappings" : {
"ghost" : {
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true,
"default" : "1.9m"
},
"properties" : {
"cn" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer"
},
"text" : {
"type" : "multi_field",
"fields" : {
"text" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer",
"position_offset_gap" : 4
},
"std" : {
"type" : "string",
"analyzer" : "english_standard_analyzer",
"position_offset_gap" : 4
}
}
}
}
},
"elf" : {
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true
},
"properties" : {
"cn" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer"
},
"text" : {
"type" : "multi_field",
"fields" : {
"text" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer",
"position_offset_gap" : 4
},
"std" : {
"type" : "string",
"analyzer" : "english_standard_analyzer",
"position_offset_gap" : 4
}
}
}
}
}
}
}

The JSON settings as passed to the UpdateSettingsRequestBuilder.setSettings(String)
method:

{
"index" : {
"refresh_interval" : "1s",
"analysis" : {
"char_filter" : { },
"filter" : {
"english_snowball_filter" : {
"type" : "snowball",
"language" : "English"
}
},
"analyzer" : {
"english_stemming_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding",
"english_snowball_filter" ]
},
"english_standard_analyzer" : {
"type" : "custom",
"tokenizer" : "standard",
"filter" : [ "standard", "lowercase", "asciifolding" ]
}
}
}
}
}

The JSON mapping as passed to the
PutMappingRequestBuilder.setSource(String) method for the specified
document type name.

{
"elf" : {
"_all" : {
"enabled" : false
},
"_ttl" : {
"enabled" : true
},
"properties" : {
"cn" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer"
},
"text" : {
"type" : "multi_field",
"fields" : {
"text" : {
"type" : "string",
"analyzer" : "english_stemming_analyzer",
"position_offset_gap" : 4
},
"std" : {
"type" : "string",
"analyzer" : "english_standard_analyzer",
"position_offset_gap" : 4
}
}
}
}
}
}

I hope this helps. (Sorry for the delay: The Elasticsearch 0.90 live
webinar took place from noon to 1PM EST, followed by lunch. It was an
excellent presentation!)

On Thursday, May 2, 2013 11:04:30 AM UTC-4, Fernando Javier Martin wrote:

Thanks InquiringMind.
I got what you said but I'm still unable to make it work.
You'll see my current and desired mappings in this gist:
gist:5502715 · GitHub

These are pretty basic indices and not for complex text parsing. As you
can guess these indices will have litttle to nothing text analysis but huge
number of updates and searches.
Anyway, I'm now in doubt if the huge number of records could slow down
the updates and retrieval.
Maybe having completely differente indices like /userdir/user and
/groupdir/group and possibly later /whateverdir/whatever

BTW, I can't find specific documentation on the PUT Mapping API.
This one (
Elasticsearch Platform — Find real-time answers at scale | Elastic)
doesn't provides much information or I'm missing something else.
Could you please give me some guidance on this?

Thanks in advance.
Fernando

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Of the 3 JSON documents I showed you, the first one contains the settings
and mappings as passed to a CreateIndexRequestBuilder.

The other two are passed into the two methods of an
UpdateIndexRequestBuilder.

I can't tell from your post if you are passing the second two JSON formats
to a create index command, but I suspect that's the case from your error.

If you update an existing index, but your mappings are wrong because of
invalid structure then they'll just be silently ignored (usually); if
mappings are wrong because an existing string field is now a long, then
Elasticsearch will complain (rightly so) in that case.

Again, be sure and check to see which Java object
(CreateIndexRequestBuilder or UpdateIndexRequestBuilder) you are using in
each case.

A create action won't update settings or mappings. An update is needed if
the index already exists. I don't know the HTTP REST API for this; I just
know the Java API.

Hope this helps.

On Friday, May 3, 2013 9:30:57 AM UTC-4, Fernando Javier Martin wrote:

Hi InquiringMind,
I'm glad you enjoyed the webinar.

Unfortunately I was unable to get it. I keep getting the
IndexAlreadyExists error.
I tried with this:

Neither worked.

I must to move on. I 'm stuck and quite late; I can handle it in another
way by using separate indexes as I stated above.
Anyway, thanks for your time.

Hopefully sooner or later I'll get the book or any other kind of course.
Thanks for your time.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

I guess I should stated initially that I'm using the HTTP REST API. I don't
even know JAVA :frowning:
Sorry, my fault.

I was creating the indexes and everything else via REST.
The changes to the index(es) before this error were done via PUT verb over
REST.
That's why I showed the link to the PUT API docs.

Maybe someone else has something to add up.
Anyway, as I said, I moved on by creating separate indices.

Thanks!

El viernes, 3 de mayo de 2013 12:03:20 UTC-3, InquiringMind escribió:

Of the 3 JSON documents I showed you, the first one contains the settings
and mappings as passed to a CreateIndexRequestBuilder.

The other two are passed into the two methods of an
UpdateIndexRequestBuilder.

I can't tell from your post if you are passing the second two JSON formats
to a create index command, but I suspect that's the case from your error.

If you update an existing index, but your mappings are wrong because of
invalid structure then they'll just be silently ignored (usually); if
mappings are wrong because an existing string field is now a long, then
Elasticsearch will complain (rightly so) in that case.

Again, be sure and check to see which Java object
(CreateIndexRequestBuilder or UpdateIndexRequestBuilder) you are using in
each case.

A create action won't update settings or mappings. An update is needed if
the index already exists. I don't know the HTTP REST API for this; I just
know the Java API.

Hope this helps.

On Friday, May 3, 2013 9:30:57 AM UTC-4, Fernando Javier Martin wrote:

Hi InquiringMind,
I'm glad you enjoyed the webinar.

Unfortunately I was unable to get it. I keep getting the
IndexAlreadyExists error.
I tried with this:

Neither worked.

I must to move on. I 'm stuck and quite late; I can handle it in another
way by using separate indexes as I stated above.
Anyway, thanks for your time.

Hopefully sooner or later I'll get the book or any other kind of course.
Thanks for your time.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

8

Fernando,

If using the HTTP REST API, then the documents I posted are the same.
However, the following links describe the differences in the URLs.

For example, the following is what you're using (I believe) to create an
index with settings and mappings:

The following should be used to update settings (and analyzers, I presume)
for an existing index. The URL endpoint is different:

And the following describes how to update existing mappings or add new
mappings for an existing index:

Hope this helps!

On Friday, May 3, 2013 11:59:06 AM UTC-4, Fernando Javier Martin wrote:

I guess I should stated initially that I'm using the HTTP REST API. I
don't even know JAVA :frowning:
Sorry, my fault.

I was creating the indexes and everything else via REST.
The changes to the index(es) before this error were done via PUT verb over
REST.
That's why I showed the link to the PUT API docs.

Maybe someone else has something to add up.
Anyway, as I said, I moved on by creating separate indices.

Thanks!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.