Total field limit exceed even less field than limit added in index

chelawatankit · July 4, 2018, 11:33am

I have an error the the "limit of total fields [55] in index" has been exceeded. Currently in my index I have 42 fields. Now I try to add 3 more fields (all integer type) in mapping but I got total field limit exceed exception.

Expected outcome :- 3 more fields should be added in existing index.
Actual Outcome :- Exception.

Is my understanding is correct.

I am calculating fields by fetching index setting and calculating "type" in mapping.

chelawatankit · July 4, 2018, 1:54pm

Hi,

Is there any fields created by elastic by default.
In my test case I found there are 11 fields by default.

Test case Step :-

Create an index with name 'test' without any mapping.
Change total field limit 11.
Try to add a field in mapping.
it throw exception field limit exceeded [11].
Change total field limit 12 now.
Try to add a field in mapping. This time it work.

When I saw mapping it show me only one field which I added.

Is this interpretation is correct.

dadoonet · July 4, 2018, 2:17pm

Yes. I believe the default internal fields like _id, _type, _index, _source (...) are playing a role here.

chelawatankit · July 4, 2018, 2:22pm

Thanks @dadoonet for response.

How many such internal fields are there?

dadoonet · July 4, 2018, 2:44pm

It's probably 9 fields. Not 100% sure though.

Source:

github.com

elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/indices/IndicesModule.java#L149-L167


private static Map<String, MetadataFieldMapper.TypeParser> initBuiltInMetadataMappers() {
    Map<String, MetadataFieldMapper.TypeParser> builtInMetadataMappers;
    // Use a LinkedHashMap for metadataMappers because iteration order matters
    builtInMetadataMappers = new LinkedHashMap<>();
    // _ignored first so that we always load it, even if only _id is requested
    builtInMetadataMappers.put(IgnoredFieldMapper.NAME, new IgnoredFieldMapper.TypeParser());
    // ID second so it will be the first (if no ignored fields) stored field to load
    // (so will benefit from "fields: []" early termination
    builtInMetadataMappers.put(IdFieldMapper.NAME, new IdFieldMapper.TypeParser());
    builtInMetadataMappers.put(RoutingFieldMapper.NAME, new RoutingFieldMapper.TypeParser());
    builtInMetadataMappers.put(IndexFieldMapper.NAME, new IndexFieldMapper.TypeParser());
    builtInMetadataMappers.put(SourceFieldMapper.NAME, new SourceFieldMapper.TypeParser());
    builtInMetadataMappers.put(TypeFieldMapper.NAME, new TypeFieldMapper.TypeParser());
    builtInMetadataMappers.put(VersionFieldMapper.NAME, new VersionFieldMapper.TypeParser());
    builtInMetadataMappers.put(SeqNoFieldMapper.NAME, new SeqNoFieldMapper.TypeParser());
    //_field_names must be added last so that it has a chance to see all the other mappers
    builtInMetadataMappers.put(FieldNamesFieldMapper.NAME, new FieldNamesFieldMapper.TypeParser());
    return Collections.unmodifiableMap(builtInMetadataMappers);
}

A part of those fields are documented here: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html

chelawatankit · July 4, 2018, 3:45pm

Thanks @dadoonet.

It would be really helpful for me.

system · August 1, 2018, 3:45pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.