# Single index for different sources

**URL:** <https://discuss.elastic.co/t/single-index-for-different-sources/42698>\
**Category:** Elasticsearch\
**Created:** [February 25, 2016, 10:55am UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698 "2016-02-25T10:55:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dinushan\_m](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinushan_m/32/8095_2.png) [@dinushan\_m](https://discuss.elastic.co/u/dinushan_m)\
**Post date:** [February 25, 2016, 10:55am UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698/1 "2016-02-25T10:55:46Z")

</div>

I have two kind of docs there fore when indexing the sources are different. Seems some fields from one type of docs are not indexed.  
Do I need to add empty fields to make equal(uniform) source from different source.  
eg:

```
doc1
{
 "id" = "/userfile1",
 "age" = "100"
}

doc2
{
  "id" = "/dir/file2",
  "name" = "bob"
}

```

problem: now the field "age" is not indexed.

should I do somthing like this ?

```
if(doc.get("age") == null)
    doc.append("age",null);
if(doc.get("name")==null)
   doc.append("name",null);

```

Or should I maintain two indixes?

---

<div class="post-metadata">

**Author:** ![crickes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crickes/32/18009_2.png) [@crickes](https://discuss.elastic.co/u/crickes)\
**Post date:** [February 25, 2016, 4:32pm UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698/2 "2016-02-25T16:32:35Z")

</div>

ES will create a index map based on the first document it indexes (and therefore creates the index), so the fields it stores will be based on that first insert. So based on which every doc type you send first, it will only have those fields indexed. To overcome this, what you need to do is create an index template to predefine all the expected fields and how they should be stored.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 25, 2016, 8:30pm UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698/3 "2016-02-25T20:30:04Z")

</div>

If you have dynamic mapping enabled it will still accept the second doc. However as @crickes mentions, you are better of defining a mapping ahead of time.

---

<div class="post-metadata">

**Author:** ![dinushan\_m](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dinushan_m/32/8095_2.png) [@dinushan\_m](https://discuss.elastic.co/u/dinushan_m)\
**Post date:** [February 26, 2016, 4:44am UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698/4 "2016-02-26T04:44:50Z")

</div>

Thanks @crickes and @warkolm for your answers. I just read about types (document types) and thought of indexing them as different types. (real documents are bit complex than shown in the example).  
Should I go ahead and index as different types ? or should I create index template?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 26, 2016, 6:48am UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698/5 "2016-02-26T06:48:51Z")

</div>

Don't use different types, see the breaking changes in 2.0 for examples of why they are not a good idea.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:13pm UTC](https://discuss.elastic.co/t/single-index-for-different-sources/42698/6 "2017-07-05T23:13:24Z")

</div>