Elasticsearch aggregation OOM

[mvg (Martijn Van Groningen)]

@abhijith_reddy

Does the per field memory usage returned _cat/fielddata include global ordinals ?

Yes, it does.

@abhijith_reddy

Since doc values is disk based, what does doc_values_memory_in_bytes give ?

The data associated with doc values is always on disk. However for certain types of docvalues there are small lookups (partly) in the jvm that contain file offsets for where to start looking for the actual data. The doc_values_memory_in_bytes stat reports the size of these lookups.

@abhijith_reddy

Is there a way to measure the space consumed by global ordinals ?

Not directly. For fields with doc values enabled, the size of field data cache only includes the size of global ordinals. For fields that rely on field data (for example analyzed strings), as far as I know there is no good way to determine what the size of global ordinals are, as the field data size will include field data itself.

@abhijith_reddy

Also I see conflicting things in the documentation related to fielddata and aggregations, [1] seems to suggest that fielddata wont be loaded but [2] says other wise. Is it possible to eagerly load global originals without triggering field data load.

In case for doc values fields with eager loading, only global ordinals will be loaded. In case for fields configured to use field data (by default that are analyzed string fields), field data has to be loaded. Global ordinals in that case depend on field data, without it can't be built.

[abhijith_reddy]

@mvg Thanks for the clarifications, this clears most things.
I am still seeing a weird behaviour wrt to my last point.

When I update the mapping to

{
  'type' => 'string',
  'index' => 'not_analyzed',
  'doc_values' => true,
  'fielddata' => array(
    'format' => 'disabled',
    'loading' => 'eager_global_ordinals',
  )
}

and start indexing documents _cat/fielddata reports 0b being used even after I issue flushes periodically. And aggregations still failed stating fielddata is disabled.

However when I remove 'format' => 'disabled' and repeat the process _cat/fielddata does show memory being consumed.
If global ordinals are being built using doc values then shouldn't aggregations have worked with fielddata being disabled ?

[mvg (Martijn Van Groningen)]

By disabling field data the entire cache is disabled and thus global ordinals can't be stored either. So I guess that is why you don't see any memory being consumed. I haven't verified, but disabling field data would not make any of your search requests work even though doc values have been configured?