# Limit indexed fields

**URL:** <https://discuss.elastic.co/t/limit-indexed-fields/230241>\
**Category:** APM\
**Tags:** java, server\
**Created:** [April 28, 2020, 7:21pm UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241 "2020-04-28T19:21:33Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Emanuel\_Velzi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emanuel_velzi/32/66975_2.png) [@Emanuel\_Velzi](https://discuss.elastic.co/u/Emanuel_Velzi)\
**Post date:** [April 28, 2020, 7:21pm UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/1 "2020-04-28T19:21:34Z")

</div>

Hi!

Using Elastic Stack for Application Performance Management (APM) I want to index only certain fields. For example, I do not need all fields related to "observer":

- observer.ephemeral\_id
- observer.hostname
- observer.id   
...

I have modified the **fields.yml** file, restart the apm-server and my app, but I'm still seeing all the fields indexed by default, just as before.

What am I doing wrong? Somebody has any idea?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![axw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/axw/32/28197_2.png) [@axw](https://discuss.elastic.co/u/axw)\
**Post date:** [April 29, 2020, 3:02am UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/2 "2020-04-29T03:02:51Z")

</div>

The `fields.yml` file can be modified to customise the index mappings, but that won't prevent fields from being indexed. To drop fields you can use an [ingest node](https://www.elastic.co/guide/en/apm/server/current/configuring-ingest-node.html) pipeline.

---

<div class="post-metadata">

**Author:** ![Emanuel\_Velzi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emanuel_velzi/32/66975_2.png) [@Emanuel\_Velzi](https://discuss.elastic.co/u/Emanuel_Velzi)\
**Post date:** [April 29, 2020, 3:09am UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/3 "2020-04-29T03:09:27Z")

</div>

Is there some way to prevent my java agent from collecting unused data?  
I would like to avoid unnecessary network traffic and overhead..

---

<div class="post-metadata">

**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\
**Post date:** [April 29, 2020, 7:53am UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/4 "2020-04-29T07:53:20Z")

</div>

All the metadata is sent only once from the agent to the server so dropping metadata doesn't save any overhead.

> [@Emanuel\_Velzi](#):
>
> I have modified the **fields.yml** file, restart the apm-server and my app, but I'm still seeing all the fields indexed by default, just as before.

Are you trying to avoid storing the fields or do you just don't want an index on those fields? If it's the latter, note that changes in `field.yml` only change the index template. It's not possible to change the mapping of an existing index. You'll either have to reindex, delete the index, or wait for the index to be rolled over so that a new index with the new mapping from the changed template is created.

There's currently no way to drop specific fields in the Java agent.

---

<div class="post-metadata">

**Author:** ![Emanuel\_Velzi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emanuel_velzi/32/66975_2.png) [@Emanuel\_Velzi](https://discuss.elastic.co/u/Emanuel_Velzi)\
**Post date:** [April 29, 2020, 11:37am UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/5 "2020-04-29T11:37:42Z")

</div>

> [@felixbarny](#):
>
> All the metadata is sent only once from the agent to the server

In my apm-7.6.2-transaction index, I see a lot of data related to observer and host and so on, in each document. This data is sent only once or do you mean something else?

> [@felixbarny](#):
>
> Are you trying to avoid storing the fields

Yes, I do not need the most of fields in my transaction index, I don't need the data itself.. So, I would like to take only a little document per transaction.

Is this possible?

---

<div class="post-metadata">

**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\
**Post date:** [April 29, 2020, 11:53am UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/6 "2020-04-29T11:53:24Z")

</div>

> [@Emanuel\_Velzi](#):
>
> This data is sent only once or do you mean something else?

It's sent only once from APM Agent to APM Server. However, APM Server sends to Elasticsearch in each document.

> [@Emanuel\_Velzi](#):
>
> Yes, I do not need the most of fields in my transaction index, I don't need the data itself.. So, I would like to take only a little document per transaction.
> 
> Is this possible?

As Andrew mentioned, the best way would be to use an ingest node to drop the fields.

---

<div class="post-metadata">

**Author:** ![Emanuel\_Velzi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emanuel_velzi/32/66975_2.png) [@Emanuel\_Velzi](https://discuss.elastic.co/u/Emanuel_Velzi)\
**Post date:** [April 29, 2020, 5:38pm UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/7 "2020-04-29T17:38:23Z")

</div>

Ok thank you both for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 20, 2020, 1:38pm UTC](https://discuss.elastic.co/t/limit-indexed-fields/230241/8 "2020-05-20T13:38:23Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
