# Thousands of fields named column with a number

**URL:** https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260
**Category:** Elasticsearch
**Created:** [June 8, 2016, 10:08pm UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260 "2016-06-08T22:08:30Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![jeffkirk1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeffkirk1/32/8080_2.png) [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)
#### Post date: [June 8, 2016, 10:08pm UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260/1 "2016-06-08T22:08:30Z")

</div>

Not sure if this really belongs in Elasticsearch, or in Logstash, but since I'm using both, I'll put it in Elasticsearch.

I recently noticed our ELK cluster has thousands and thousands of fields named "column" followed by a number. I have no idea how they got there, but I want to get rid of them. I understand my only option may be a full reindex (pain) but if I'm wrong, please let me know? Also, does anyone know how these fields could've appeared in the first place?

We're feeding the cluster with Filebeats via Logstash and with data coming in via the Logstash syslog plugin.

---

<div class="post-metadata">

### Author: ![msimos](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@msimos](https://discuss.elastic.co/u/msimos)
#### Post date: [June 8, 2016, 11:26pm UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260/2 "2016-06-08T23:26:34Z")

</div>

You could use the reindex API to reindex the data and remove the field(s):

[https://www.elastic.co/guide/en/elasticsearch/reference/2.3/docs-reindex.html](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/docs-reindex.html)

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 9, 2016, 5:53am UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260/3 "2016-06-09T05:53:07Z")

</div>

I would suspect they could have been generated by a Logstash csv filter, as this by default auto-generates column names in that format whenever it comes across columns it does not have configured names for.

---

<div class="post-metadata">

### Author: ![jeffkirk1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeffkirk1/32/8080_2.png) [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)
#### Post date: [June 9, 2016, 4:49pm UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260/4 "2016-06-09T16:49:04Z")

</div>

That was exactly it. A missing application\_name field when CSV-parsing our PostgreSQL server replication logs. Thanks! That led me right to the solution.

---

<div class="post-metadata">

### Author: ![jeffkirk1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeffkirk1/32/8080_2.png) [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)
#### Post date: [June 9, 2016, 4:50pm UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260/5 "2016-06-09T16:50:33Z")

</div>

Thank you! Good idea but we have way too much data to reindex in a timely manner. I've fixed the problem with the CSV parsing (described in another post in this thread). I presume that new indices will not have these extra columns after the old indices age out?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:44pm UTC](https://discuss.elastic.co/t/thousands-of-fields-named-column-with-a-number/52260/6 "2017-07-05T22:44:57Z")

</div>


