# Index full document or subset?

**URL:** <https://discuss.elastic.co/t/index-full-document-or-subset/214144>\
**Category:** Elasticsearch\
**Created:** [January 8, 2020, 2:06am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144 "2020-01-08T02:06:36Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rvs](https://avatars.discourse-cdn.com/v4/letter/r/f4b2a3/32.png) [@Rvs](https://discuss.elastic.co/u/Rvs)\
**Post date:** [January 8, 2020, 2:06am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/1 "2020-01-08T02:06:36Z")

</div>

I Had a newbie question. Would appreciate some advice here.

Should i index my entire document or only the data (subset)within my document that i know will be searched on.

What is the philosophy of elasticsearch. Should we create indexes on only a subset of our document that will be searched, to improve performance.

If in the future there is a new field that needs to be searched, then i would have to reprocess the document to include that new field in the index.

So is indexing the entire document future proofing all searches or would performance degrade with that approach.

I am trying to see what approach elastisearch recommends.

Thank you

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 8, 2020, 8:49am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/2 "2020-01-08T08:49:30Z")

</div>

Welcome!

I'd index the full document. Specifically if you need to use some of the data to display results even though you don't search on those fields.

If you don't want the fields to be searchable, you can change the mapping and define `"index": false` (see [https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index.html)).

If you change your mind, you can always use the reindex API to reindex the documents with a new strategy for those fields. As you have the full `_source`, it will make your life easier.

Long story short: send the whole document to elasticsearch.

---

<div class="post-metadata">

**Author:** ![Rvs](https://avatars.discourse-cdn.com/v4/letter/r/f4b2a3/32.png) [@Rvs](https://discuss.elastic.co/u/Rvs)\
**Post date:** [January 8, 2020, 11:28am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/3 "2020-01-08T11:28:21Z")

</div>

Thank you for the reply. I really appreciate it.  
Would size of the document be an issue from a cost or performance perspective.

Also do i need to map all the fields in my document explicitly?  
I understand elasticsearch will index all fields by default if an explicit mapping is not provided (dynamic mapping?). Is relying on that not recommended.

Thank you again

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 8, 2020, 11:54am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/4 "2020-01-08T11:54:30Z")

</div>

> [@Rvs](#):
>
> Would size of the document be an issue from a cost or performance perspective.

Yes. If you are storing things like BASE64 blobs, then it will have an impact (disk size and performance wise). It depends. What does a typical document would look like? Could you share that?

> [@Rvs](#):
>
> Is relying on that not recommended.

I do not recommend it. It's very good to begin with elasticsearch but at the end, you want to have more control on what is happening. You can define a dynamic template: see [Dynamic templates | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-templates.html).

---

<div class="post-metadata">

**Author:** ![Rvs](https://avatars.discourse-cdn.com/v4/letter/r/f4b2a3/32.png) [@Rvs](https://discuss.elastic.co/u/Rvs)\
**Post date:** [January 8, 2020, 7:49pm UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/5 "2020-01-08T19:49:43Z")

</div>

It's not just the size of the document (which might not be a big deal for now).

The variation in the document is large too (which would make mapping all fields cumbersome).

My document is a xml document that follows commercial insurance ACORD model.

Lots of nested tags and lots of optional data tags, so we'd have to map a ton of fields and mark a lot of them to be NOT indexed

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 8, 2020, 8:19pm UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/6 "2020-01-08T20:19:35Z")

</div>

If all the optional fields are under another field like `foo` and `bar` under `meta` (`meta.foo` and `meta.bar`), then you can just disable `meta` object field and everything under will be ignored.

See [https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enabled.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/enabled.html)

---

<div class="post-metadata">

**Author:** ![Rvs](https://avatars.discourse-cdn.com/v4/letter/r/f4b2a3/32.png) [@Rvs](https://discuss.elastic.co/u/Rvs)\
**Post date:** [January 9, 2020, 1:18am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/7 "2020-01-09T01:18:31Z")

</div>

Thank you for your patience. I do have one last question.

Can we map the fields we want and configure elasticsearch to ignore unmapped fields by default, instead of applying dynamic mapping

Thank you

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 9, 2020, 2:53am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/8 "2020-01-09T02:53:44Z")

</div>

Yes. Dynamic templates will help for this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 6, 2020, 2:53am UTC](https://discuss.elastic.co/t/index-full-document-or-subset/214144/9 "2020-02-06T02:53:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
