# Too many fields in an index

**URL:** <https://discuss.elastic.co/t/too-many-fields-in-an-index/339679>\
**Category:** Elasticsearch\
**Created:** [July 31, 2023, 10:55am UTC](https://discuss.elastic.co/t/too-many-fields-in-an-index/339679 "2023-07-31T10:55:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jurrien](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jurrien/32/124111_2.png) [@Jurrien](https://discuss.elastic.co/u/Jurrien)\
**Post date:** [July 31, 2023, 10:55am UTC](https://discuss.elastic.co/t/too-many-fields-in-an-index/339679/1 "2023-07-31T10:55:28Z")

</div>

We have an index with the following structure (see below)

So basically, we have our index business\_objects with a link and no. We add objects to this index (doc\_type1, doc\_type2, ....). These objects are linked via no 5793328-03-01 for example.

The problem is that we have many doc\_types (doc\_type1, doc\_type2, …., doc\_type100).

Each doc\_type has some common fields like date, id, mime\_type but there are also some doc\_type specific fields like business\_number,…  
So at the moment this increases the fields in the index if we add a new doc\_type eg doc\_type101.  
We are well over 1000 fields now.

How could we redesign this index so we stay below the recommended 1000 fields?

We were thinking of only having one doc\_type and rely on the doc\_type.type to determine the type of the doc\_type.  
This 'generic' doc\_type would contain the sum of all fields in (doc\_type1, doc\_type2, …., doc\_type100).  
The problem is that we cannot use source filtering and security filtering like this anymore if we do that:

```auto
"except": [
"link.doc_type_1.*”,  

```

Are there any other solutions for this type of index design?

```auto
"_index" : "business_objects",
        "_type" : "_doc",
        "_id" : "8579338-3",
        "_score" : 1.0,
        "_source" : {
          "business_object_no" : "8579338-3",
          "company" : {
            "id" : "148",
            "call_sign" : “NAME”
          },
          "link" : [
            {
              "no" : "5793328-03-01",
              "doc_type_1" : [
                {
                  "date" : "20220301000000",
                  "filename" : "test1.pdf",
                  "official_number" : "2022000000013",
                  "mime_type" : "application/pdf",
                  "id" : "5009467F-0000-C91C-B281-BBBBZA",
                  "type" : "doc_type_1",
                  "business_number" : "22000766"
                }
              ],
              "doc_type_2" : [
                {
                  "date" : "20220301000000",
                  "filename" : "test2.pdf",
                  "official_number" : "RE02022000000013",
                  "mime_type" : "text/xml",
                  "id" : "800D467F-0000-C314-88D3-AAAAZE",
                  "type" : "doc_type_2"
                }
              ]
        }
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 28, 2023, 10:56am UTC](https://discuss.elastic.co/t/too-many-fields-in-an-index/339679/2 "2023-08-28T10:56:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.