Reading field Values from docvalues using DoubleFieldSource

Elastic Stack Elasticsearch
#1

Hello All,

In a custom analyzer plugin I am trying to read a field value using DoubleFieldSource. The problem that i am facing is that this fails with the following error.

QueryPhaseExecutionException[Query Failed [Failed to execute main query]]; nested: IllegalStateException[unexpected docvalues type SORTED_NUMERIC for field 'fileExtensionId' (expected=NUMERIC). Use UninvertingReader or index with docvalues

We have indexed the field with doc_values and the type of the field is double. We were using the custom plugin with 1.7, and it was working fine (although we were using field data and not doc values with ES 1.7).

PS: As an additional step I created SortedDoubleFieldSource which extends from DoubleFieldSource and overrides getValues to read sorted numeric values. It gives me zero for all the docs.

My question is Why is the double field being treated as Sorted numeric and not numeric? And how can i read value of this double field?

#2

It feels wrong to me that an analyzer would try to read the index.

Indeed you should not use DoubleFieldSource which expects a different encoding from the one that elasticsearch is using. elasticsearch is using SORTED_NUMERIC because all fields can be multi-valued in elasticsearch, and NUMERIC only supports single-valued fields. The right way to read values in elasticsearch would be to use the fielddata API. You need to get an IndexNumericFieldData from IndexFielddataService and then call load().getDoubleValues().

#3

Thanks for the response Adrien. i am sorry i meant custom aggregation not custom analyzer. It was a slip while typing. I will try to use the fieldData API and will post if i face trouble.

Thanks and regards.

#4

So i tried the following

SearchContext searchContex = SearchContext.current();
IndexFieldDataService idfService = searchContext.fieldData();
IndexNumericFieldData fieldData = idfService.getForField(searchContext.smartNameFieldType(field));
final SortedNumericDoubleValues arr = fieldData.load(readerContext).getDoubleValues();

I am still seeing zero for the field for all the docs. is this correct or do i need to change something? Thanks for the help.

#5

Can you check how your field is mapped in the mappings? I am thinking that it might have been mapped as an int or a long instead of a float or double?

#6

This solved the issue.

    final SortedNumericDoubleValues arr = fieldData.load(readerContext).getDoubleValues();
    final NumericDoubleValues unwrappedArr = FieldData.unwrapSingleton(arr);
#7