# Need help architecting an index/schema (migrating data from MySQL)

**URL:** <https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541>\
**Category:** Elasticsearch\
**Created:** [April 11, 2013, 2:17am UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541 "2013-04-11T02:17:54Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![mmattax](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mmattax/32/2395_2.png) [@mmattax](https://discuss.elastic.co/u/mmattax)\
**Post date:** [April 11, 2013, 2:17am UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/1 "2013-04-11T02:17:54Z")

</div>

Hi,

I'm just getting started with elastic search and need some advice. We  
have 3 tables in MySQL that we'd like to combine into a simpler schema  
with elasticsearch:

+---------------------+  
| form +  
+---------------------+  
| id |  
| name |  
+---------------------+

+---------------------+  
| entry +  
+---------------------+  
| id |  
| form\_id |  
| timestamp |  
+---------------------+

+---------------------+  
| entry\_data +  
+---------------------+  
| id |  
| entry\_id |  
| key |  
| value |  
+---------------------+

Constraints as follows: A form can have many entries, an entry can  
have many entry\_data's. Each form has it's own schema. Our DB  
currently has 250,000 forms and 47,000,000 entries.

A few ways that I think we can accomplish this within elasticsearch:

1. Keep the same structure we have in MySQL: create an entry index,  
the "form\_id" will be a part of the entry document.
2. Have a entry index, each form becomes a type, entry is the  
document.
3. Each form is an index.

My main questions:

Is one of the schemes above better than the other? I'm thinking that  
#2 will be the simplest. Is there a performance cost to having that  
many types (250k)?

Would you architect this differently?

Thanks,

Michael

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 11, 2013, 8:24am UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/2 "2013-04-11T08:24:24Z")

</div>

IMHO, the first question should be always user centric and not technical centric.  
I mean that you should ask yourself: "what my users want to find?"

I don't think that they want to search for entries or for data entries, do they?  
Probably, you have behind this flexible design a real use case that you could share with us.

I would probably try to create real business documents instead of technical ones in my application and then send to Elasticsearch _real_ documents.  
That said, I don't have any idea of your use case and I might be wrong.

My 2 cents

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 11 avr. 2013 à 04:17, mmattax [mattax.m@gmail.com](mailto:mattax.m@gmail.com) a écrit :

> Hi,
> 
> I'm just getting started with Elasticsearch and need some advice. We  
> have 3 tables in MySQL that we'd like to combine into a simpler schema  
> with elasticsearch:
> 
> +---------------------+  
> | form +  
> +---------------------+  
> | id |  
> | name |  
> +---------------------+
> 
> +---------------------+  
> | entry +  
> +---------------------+  
> | id |  
> | form\_id |  
> | timestamp |  
> +---------------------+
> 
> +---------------------+  
> | entry\_data +  
> +---------------------+  
> | id |  
> | entry\_id |  
> | key |  
> | value |  
> +---------------------+
> 
> Constraints as follows: A form can have many entries, an entry can  
> have many entry\_data's. Each form has it's own schema. Our DB  
> currently has 250,000 forms and 47,000,000 entries.
> 
> A few ways that I think we can accomplish this within elasticsearch:
> 
> 1. Keep the same structure we have in MySQL: create an entry index,  
> the "form\_id" will be a part of the entry document.
> 2. Have a entry index, each form becomes a type, entry is the  
> document.
> 3. Each form is an index.
> 
> My main questions:
> 
> Is one of the schemes above better than the other? I'm thinking that  
> #2 will be the simplest. Is there a performance cost to having that  
> many types (250k)?
> 
> Would you architect this differently?
> 
> Thanks,
> 
> Michael
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mmattax](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mmattax/32/2395_2.png) [@mmattax](https://discuss.elastic.co/u/mmattax)\
**Post date:** [April 11, 2013, 1:54pm UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/3 "2013-04-11T13:54:56Z")

</div>

The entry and entry\_data tables represent form responses that customers  
have built (I work for Formstack [http://formstack.com](http://formstack.com)). The data can  
represent anything from a survey, event registration forms, order forms,  
etc. Ideally everything in entry\_data becomes searchable and sortable:

entry\_data might have a few rows in MySQL:

id entry\_id key value  
1 100 name first=Michael&last=Mattax  
2 100 email [michael@example.com](mailto:michael@example.com)  
3 100 sex male  
4 100 birthday 1985-08-15  
.. there could be hundreds more ...

Using Elasticsearch we want to make all of this data searchable so the  
document might look like

curl -XPUT '[http://localhost:9200/entries/](http://localhost:9200/entries/)\<FORM\_ID\>/100' -d '{

```
name : { first : 'Michael', last : 'Mattax' },
email : 'michael@example.com',
sex : 'male',
'birthday' : '1985-08-15'

```

}'

That way we can use Elasticsearch power and not do any joins (which we do  
now for complex searches). Note that each form has a different schema  
(besides some common shared metadata).

Does that clarify things at all?

On Thursday, April 11, 2013 4:24:24 AM UTC-4, David Pilato wrote:

> IMHO, the first question should be always user centric and not technical  
> centric.  
> I mean that you should ask yourself: "what my users want to find?"
> 
> I don't think that they want to search for entries or for data entries, do  
> they?  
> Probably, you have behind this flexible design a real use case that you  
> could share with us.
> 
> I would probably try to create real business documents instead of  
> technical ones in my application and then send to Elasticsearch _real_  
> documents.  
> That said, I don't have any idea of your use case and I might be wrong.
> 
> My 2 cents
> 
> --  
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)  
> | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> Le 11 avr. 2013 à 04:17, mmattax \<[matt...@gmail.com](mailto:matt...@gmail.com) \<javascript:\>\> a  
> écrit :
> 
> Hi,
> 
> I'm just getting started with Elasticsearch and need some advice. We  
> have 3 tables in MySQL that we'd like to combine into a simpler schema  
> with elasticsearch:
> 
> +---------------------+  
> | form +  
> +---------------------+  
> | id |  
> | name |  
> +---------------------+
> 
> +---------------------+  
> | entry +  
> +---------------------+  
> | id |  
> | form\_id |  
> | timestamp |  
> +---------------------+
> 
> +---------------------+  
> | entry\_data +  
> +---------------------+  
> | id |  
> | entry\_id |  
> | key |  
> | value |  
> +---------------------+
> 
> Constraints as follows: A form can have many entries, an entry can  
> have many entry\_data's. Each form has it's own schema. Our DB  
> currently has 250,000 forms and 47,000,000 entries.
> 
> A few ways that I think we can accomplish this within elasticsearch:
> 
> 1. Keep the same structure we have in MySQL: create an entry index,  
> the "form\_id" will be a part of the entry document.
> 2. Have a entry index, each form becomes a type, entry is the  
> document.
> 3. Each form is an index.
> 
> My main questions:
> 
> Is one of the schemes above better than the other? I'm thinking that  
> #2 will be the simplest. Is there a performance cost to having that  
> many types (250k)?
> 
> Would you architect this differently?
> 
> Thanks,
> 
> Michael
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 11, 2013, 2:13pm UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/4 "2013-04-11T14:13:57Z")

</div>

Definitely!

It's a cool idea.  
So you agree that you will have to create a document on your side based on entry\_data.

You should be aware that you will probably need to define some things using mapping. Otherwise, the default analysis process could give you undesired results.

For example, for email, you should define an email analyzer.  
Be careful of field names and types. I would try to avoid to have a field name for example answer1 and answer1 in type1 is a Number and answer1 in type2 is a String.

Regarding to your first question: I also think that #2 is one of the best option here.  
I also like #3 as I can imagine that after some months, you may want to remove old surveys? It's really more efficient to remove a full index than removing documents.  
Also, you can think of mixing #2 and #3 and have a fine tuning about the number of shards needed for each index. Let's say that you have a very big Survey with 10 000 000 of answers. You probably want to index it in its own index. Let's say that you have 240 000 _small surveys_. You probably want to share the same index…

Make sense?

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 11 avr. 2013 à 15:54, mmattax [mattax.m@gmail.com](mailto:mattax.m@gmail.com) a écrit :

> The entry and entry\_data tables represent form responses that customers have built (I work for Formstack). The data can represent anything from a survey, event registration forms, order forms, etc. Ideally everything in entry\_data becomes searchable and sortable:
> 
> entry\_data might have a few rows in MySQL:
> 
> id entry\_id key value  
> 1 100 name first=Michael&last=Mattax  
> 2 100 email [michael@example.com](mailto:michael@example.com)  
> 3 100 sex male  
> 4 100 birthday 1985-08-15  
> .. there could be hundreds more ...
> 
> Using Elasticsearch we want to make all of this data searchable so the document might look like
> 
> curl -XPUT '[http://localhost:9200/entries/](http://localhost:9200/entries/)\<FORM\_ID\>/100' -d '{  
> name : { first : 'Michael', last : 'Mattax' },  
> email : 'michael@example.com',  
> sex : 'male',  
> 'birthday' : '1985-08-15'  
> }'
> 
> That way we can use Elasticsearch power and not do any joins (which we do now for complex searches). Note that each form has a different schema (besides some common shared metadata).
> 
> Does that clarify things at all?
> 
> On Thursday, April 11, 2013 4:24:24 AM UTC-4, David Pilato wrote:  
> IMHO, the first question should be always user centric and not technical centric.  
> I mean that you should ask yourself: "what my users want to find?"
> 
> I don't think that they want to search for entries or for data entries, do they?  
> Probably, you have behind this flexible design a real use case that you could share with us.
> 
> I would probably try to create real business documents instead of technical ones in my application and then send to Elasticsearch _real_ documents.  
> That said, I don't have any idea of your use case and I might be wrong.
> 
> My 2 cents
> 
> --  
> David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
> @dadoonet | @elasticsearchfr | @scrutmydocs
> 
> Le 11 avr. 2013 à 04:17, mmattax [matt...@gmail.com](mailto:matt...@gmail.com) a écrit :
> 
> > Hi,
> > 
> > I'm just getting started with Elasticsearch and need some advice. We  
> > have 3 tables in MySQL that we'd like to combine into a simpler schema  
> > with elasticsearch:
> > 
> > +---------------------+  
> > | form +  
> > +---------------------+  
> > | id |  
> > | name |  
> > +---------------------+
> > 
> > +---------------------+  
> > | entry +  
> > +---------------------+  
> > | id |  
> > | form\_id |  
> > | timestamp |  
> > +---------------------+
> > 
> > +---------------------+  
> > | entry\_data +  
> > +---------------------+  
> > | id |  
> > | entry\_id |  
> > | key |  
> > | value |  
> > +---------------------+
> > 
> > Constraints as follows: A form can have many entries, an entry can  
> > have many entry\_data's. Each form has it's own schema. Our DB  
> > currently has 250,000 forms and 47,000,000 entries.
> > 
> > A few ways that I think we can accomplish this within elasticsearch:
> > 
> > 1. Keep the same structure we have in MySQL: create an entry index,  
> > the "form\_id" will be a part of the entry document.
> > 2. Have a entry index, each form becomes a type, entry is the  
> > document.
> > 3. Each form is an index.
> > 
> > My main questions:
> > 
> > Is one of the schemes above better than the other? I'm thinking that  
> > #2 will be the simplest. Is there a performance cost to having that  
> > many types (250k)?
> > 
> > Would you architect this differently?
> > 
> > Thanks,
> > 
> > Michael
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mmattax](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mmattax/32/2395_2.png) [@mmattax](https://discuss.elastic.co/u/mmattax)\
**Post date:** [April 11, 2013, 2:32pm UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/5 "2013-04-11T14:32:48Z")

</div>

A few follow up questions, which might help me pick between #2 and #3.

- Can we delete (in bulk) a type out of an index (each form being it's  
own type and it's own maping)?
- Is there any pitfalls with having a large number of types (and  
mapping), for example 250k mapping/types for the index?
- Is there a point where we're going to need to worry about open file  
descriptors (with 250k indexes)?

On Thursday, April 11, 2013 10:13:57 AM UTC-4, David Pilato wrote:

> Definitely!
> 
> It's a cool idea.  
> So you agree that you will have to create a document on your side based on  
> entry\_data.
> 
> You should be aware that you will probably need to define some things  
> using mapping. Otherwise, the default analysis process could give you  
> undesired results.
> 
> For example, for email, you should define an email analyzer.  
> Be careful of field names and types. I would try to avoid to have a field  
> name for example answer1 and answer1 in type1 is a Number and answer1 in  
> type2 is a String.
> 
> Regarding to your first question: I also think that #2 is one of the best  
> option here.  
> I also like #3 as I can imagine that after some months, you may want to  
> remove old surveys? It's really more efficient to remove a full index than  
> removing documents.  
> Also, you can think of mixing #2 and #3 and have a fine tuning about the  
> number of shards needed for each index. Let's say that you have a very big  
> Survey with 10 000 000 of answers. You probably want to index it in its own  
> index. Let's say that you have 240 000 _small surveys_. You probably want  
> to share the same index…
> 
> Make sense?
> 
> --  
> _David Pilato_ | _Technical Advocate_ | _[Elasticsearch.com](http://Elasticsearch.com)_  
> @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)  
> | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> 
> Le 11 avr. 2013 à 15:54, mmattax \<[matt...@gmail.com](mailto:matt...@gmail.com) \<javascript:\>\> a  
> écrit :
> 
> The entry and entry\_data tables represent form responses that customers  
> have built (I work for Formstack [http://formstack.com/](http://formstack.com/)). The data can  
> represent anything from a survey, event registration forms, order forms,  
> etc. Ideally everything in entry\_data becomes searchable and sortable:
> 
> entry\_data might have a few rows in MySQL:
> 
> id entry\_id key value  
> 1 100 name first=Michael&last=Mattax  
> 2 100 email [mic...@example.com](mailto:mic...@example.com) \<javascript:\>  
> 3 100 sex male  
> 4 100 birthday 1985-08-15  
> .. there could be hundreds more ...
> 
> Using Elasticsearch we want to make all of this data searchable so the  
> document might look like
> 
> curl -XPUT '[http://localhost:9200/entries/](http://localhost:9200/entries/)\<FORM\_ID\>/100' -d '{
> 
> ```
> name : { first : 'Michael', last : 'Mattax' },
> email : 'mic...@example.com <javascript:>',
> sex : 'male',
> 'birthday' : '1985-08-15'
> 
> ```
> 
> }'
> 
> That way we can use Elasticsearch power and not do any joins (which we do  
> now for complex searches). Note that each form has a different schema  
> (besides some common shared metadata).
> 
> Does that clarify things at all?
> 
> On Thursday, April 11, 2013 4:24:24 AM UTC-4, David Pilato wrote:
> 
> > IMHO, the first question should be always user centric and not technical  
> > centric.  
> > I mean that you should ask yourself: "what my users want to find?"
> > 
> > I don't think that they want to search for entries or for data entries,  
> > do they?  
> > Probably, you have behind this flexible design a real use case that you  
> > could share with us.
> > 
> > I would probably try to create real business documents instead of  
> > technical ones in my application and then send to Elasticsearch _real_  
> > documents.  
> > That said, I don't have any idea of your use case and I might be wrong.
> > 
> > My 2 cents
> > 
> > --  
> > _David Pilato_ | _Technical Advocate_ | \*[Elasticsearch.com](http://Elasticsearch.com)[http://elasticsearch.com/](http://elasticsearch.com/)  
> > \*  
> > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr[https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr)  
> > | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs)
> > 
> > Le 11 avr. 2013 à 04:17, mmattax [matt...@gmail.com](mailto:matt...@gmail.com) a écrit :
> > 
> > Hi,
> > 
> > I'm just getting started with Elasticsearch and need some advice. We  
> > have 3 tables in MySQL that we'd like to combine into a simpler schema  
> > with elasticsearch:
> > 
> > +---------------------+  
> > | form +  
> > +---------------------+  
> > | id |  
> > | name |  
> > +---------------------+
> > 
> > +---------------------+  
> > | entry +  
> > +---------------------+  
> > | id |  
> > | form\_id |  
> > | timestamp |  
> > +---------------------+
> > 
> > +---------------------+  
> > | entry\_data +  
> > +---------------------+  
> > | id |  
> > | entry\_id |  
> > | key |  
> > | value |  
> > +---------------------+
> > 
> > Constraints as follows: A form can have many entries, an entry can  
> > have many entry\_data's. Each form has it's own schema. Our DB  
> > currently has 250,000 forms and 47,000,000 entries.
> > 
> > A few ways that I think we can accomplish this within elasticsearch:
> > 
> > 1. Keep the same structure we have in MySQL: create an entry index,  
> > the "form\_id" will be a part of the entry document.
> > 2. Have a entry index, each form becomes a type, entry is the  
> > document.
> > 3. Each form is an index.
> > 
> > My main questions:
> > 
> > Is one of the schemes above better than the other? I'm thinking that  
> > #2 will be the simplest. Is there a performance cost to having that  
> > many types (250k)?
> > 
> > Would you architect this differently?
> > 
> > Thanks,
> > 
> > Michael
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 11, 2013, 2:42pm UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/6 "2013-04-11T14:42:01Z")

</div>

My 2 cents inline...

> A few follow up questions, which might help me pick between #2 and #3.  
> Can we delete (in bulk) a type out of an index (each form being it's own type and it's own maping)?  
> Not using bulk but using: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-delete-mapping/)  
> Is there any pitfalls with having a large number of types (and mapping), for example 250k mapping/types for the index?  
> I don't see any. That said, my concern is more about field names collision with different mapping definition.  
> Is there a point where we're going to need to worry about open file descriptors (with 250k indexes)?  
> Yes. If you need to update all indices at the same time, you will probably have issues if you run that on a small number of nodes.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![mmattax](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mmattax/32/2395_2.png) [@mmattax](https://discuss.elastic.co/u/mmattax)\
**Post date:** [April 11, 2013, 3:06pm UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/7 "2013-04-11T15:06:58Z")

</div>

I really appreciate the insight. Going to get started with Elasticsearch!

On Thursday, April 11, 2013 10:42:01 AM UTC-4, David Pilato wrote:

> My 2 cents inline...
> 
> A few follow up questions, which might help me pick between #2 and #3.
> 
> - Can we delete (in bulk) a type out of an index (each form being it's  
> own type and it's own maping)?
> 
> Not using bulk but using:  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-delete-mapping/)
> 
> - Is there any pitfalls with having a large number of types (and  
> mapping), for example 250k mapping/types for the index?
> 
> I don't see any. That said, my concern is more about field names collision  
> with different mapping definition.
> 
> - Is there a point where we're going to need to worry about open file  
> descriptors (with 250k indexes)?
> 
> Yes. If you need to update all indices at the same time, you will probably  
> have issues if you run that on a small number of nodes.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:41am UTC](https://discuss.elastic.co/t/need-help-architecting-an-index-schema-migrating-data-from-mysql/11541/8 "2017-07-06T02:41:34Z")

</div>


