Suitability of Aggregation framework for traditional OLAP style analytics


(Alex Roytman) #1

I am having an issue with aggregation framework in one particular aspect -
its not handling _missing values and _other values as they were handled in
facets (_other was not handled even in any stats facets just in terms)
traditional OLAP would slice/roll-up the same data set by various
dimensions so roll up counts/totals would come out the same even if we are
grouping on a field which may be null for some records

With aggregation framework there is no way to do it except for an
exceedingly convoluted use of "missing" aggregation (just try to have
missing values aggregated as part of the overall bucket-set when doing
multi-level aggregation)
you can find lot more details, example and my proposal here

Unfortunately it did not get any reply from the development team so I can
only assume they are not convinced or did not read it (would be nice if
they at least said so may be they missed it all together)

I do not want to replace null values with some fake values representing
null and have it bleed all over the applications consuming JSON data from
elastic. So if Elastic is not going to handle missing, what are my options
(apart from the option described in my proposal on github)?

Could I use "null_value" in my index mapping for nullable fields? Will they
be used for aggregation rather than _source. Even if they are will it work
when null value is one of objects and I am aggregating on that object
properties?
(i.e. case type is {caseNumber:123, caseType:{id:10, name:'Civil'}} and I
am aggregating on caseType.id and caseType could be null)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6ad26233-59f4-4464-a380-2832685e528b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2