# Unwanted fields in indice

**URL:** <https://discuss.elastic.co/t/unwanted-fields-in-indice/232679>\
**Category:** Elasticsearch\
**Created:** [May 14, 2020, 3:34pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679 "2020-05-14T15:34:04Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [May 14, 2020, 3:34pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/1 "2020-05-14T15:34:04Z")

</div>

One of my indice has some field which are nowhere in my logstash config.  
yes that fields are from other config file. but both config files are separate pipeline. both has different document\_id

how come they appear in this indice?  
what did I do wrong? anyone has seen such problem before?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 14, 2020, 4:04pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/2 "2020-05-14T16:04:37Z")

</div>

Have you specified the pipelines in pipelines.yml? If not, having config in different files does not make them individual pipelines as Logstash concatenates flies into a single pipeline by default.

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [May 14, 2020, 4:13pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/3 "2020-05-14T16:13:30Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> ied the pipelines

yes I have different piplelines (total 21 running at this time) happend in this one index only. don't know how. trying to figure out how to prevent this happening again.  
and second how to remove them?

I don't want to recreate index pattern as I have few dashboard already design using that.  
I can remove whole indice and recreate it but as situation has given me chance to try to remove individual field if possible.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 14, 2020, 4:19pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/4 "2020-05-14T16:19:19Z")

</div>

Can you share your config and sample events that have been incorrectly processed?

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [May 14, 2020, 4:29pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/5 "2020-05-14T16:29:32Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> that have been incorrectly process

pipelines

```
- pipeline.id: tms_change_control (data from this pipeline got in to second)
  path.config: "/etc/logstash/conf.d/tms_change_control.conf"
  pipeline.workers: 1

- pipeline.id: orion_network_inventory (extra field from top one got here)
  path.config: "/etc/logstash/conf.d/orion_network_inventory.conf"
  pipeline.workers: 1

```

they both reading from completed different database  
complete different columns

but I have "assignee, assignelocation, chage\_time, change\_date,' pretty much all field from that got in here.

don't understand it.  
both config has different indexname, and different document\_id in output sections.

possible that someone ran this config manually and pointted to wrong index?

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [May 14, 2020, 4:50pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/6 "2020-05-14T16:50:34Z")

</div>

I was just want to test it and hence I deleted index and repulled the data manually and all unwanted fields are gone.

not what I wanted to test (removing individual field) but for now I can put this to rest. Thanks Christian

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 11, 2020, 5:03pm UTC](https://discuss.elastic.co/t/unwanted-fields-in-indice/232679/7 "2020-06-11T17:03:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.