# How should I deal with 10TB data with diferent languages!

**URL:** <https://discuss.elastic.co/t/how-should-i-deal-with-10tb-data-with-diferent-languages/67941>\
**Category:** Elasticsearch\
**Created:** [December 4, 2016, 6:22am UTC](https://discuss.elastic.co/t/how-should-i-deal-with-10tb-data-with-diferent-languages/67941 "2016-12-04T06:22:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![xile](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xile/32/13573_2.png) [@xile](https://discuss.elastic.co/u/xile)\
**Post date:** [December 4, 2016, 6:22am UTC](https://discuss.elastic.co/t/how-should-i-deal-with-10tb-data-with-diferent-languages/67941/1 "2016-12-04T06:22:06Z")

</div>

We have almost 10TB data of E-draft, E-draft have different languages, about 10 languages(such as zh-CN. en. Arabic Language etc.).

- How should we index and query such amounts of data with different languages.

- Should we create multiple index?

- how to set up shards?

These Examples are parial of our data :

**1. zh\_CN:**

 ![](https://us1.discourse-cdn.com/elastic/original/2X/9/9f998e6fc5506626b2d906e27d606a2be288e2fe.png)

**2. en**

 ![](https://us1.discourse-cdn.com/elastic/original/2X/0/04436753c189fcb170771ab77adead217d851816.png)

**3. ar**

 ![](https://us1.discourse-cdn.com/elastic/original/2X/0/0bdc0f578a181d852e650ce9fd96d8e415fcdf85.png)

**Thanks for answers.**

12/04/2016

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 4, 2016, 6:35am UTC](https://discuss.elastic.co/t/how-should-i-deal-with-10tb-data-with-diferent-languages/67941/2 "2016-12-04T06:35:01Z")

</div>

I'd start here: [https://www.elastic.co/guide/en/elasticsearch/guide/current/languages.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/languages.html)

---

<div class="post-metadata">

**Author:** ![xile](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xile/32/13573_2.png) [@xile](https://discuss.elastic.co/u/xile)\
**Post date:** [December 4, 2016, 7:00am UTC](https://discuss.elastic.co/t/how-should-i-deal-with-10tb-data-with-diferent-languages/67941/4 "2016-12-04T07:00:58Z")

</div>

> [@dadoonet](#):
>
> I'd start here: [Dealing with Human Language | Elasticsearch: The Definitive Guide [2.x] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/current/languages.html)

Thanks,I'll read it carefully

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 1, 2017, 7:00am UTC](https://discuss.elastic.co/t/how-should-i-deal-with-10tb-data-with-diferent-languages/67941/5 "2017-01-01T07:00:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.