Kibana array of objects visibility after _source field deprecation

[israelst (Israel Steinmetz)]

Since Kibana 7.12 - the "_source" field is no longer used by default in favor of the new field API.
This totally broke our ability to view array of objects in Discover. What used to be shown as JSON, is now being flattened into fields containing array of strings - loosing all correlation between them.
for example, what was shown as:

[
    {
        "name": "john",
        "age": 21
    },
    {
        "name": "andy",
        "age": 10
    }
]

is now:

name: john,andy
age: 21, 10

We were fully aware of the Elastic limitations regarding array of objects and not being able to have a correlation-query between these object, this was fine by us since it's not our use case. But we do need to be able to view it as a correlated data in Kibana.

I realize that there is currently a configuration allowing to switch back to using _source but it is marked as "Deprecated", how are we suppose to mitigate this issue after deprecation?

We do not want to use the "nested" type since it has its drawbacks and as said - our use case does not require correlation-query.

Thanks.

@majagrubic , I saw your blog on this topic, can you maybe assist please?

[majagrubic (Maja)]

Sorry, I was away for few days. What you are describing is indeed how fields API works. Please not that Elasticsearch doesn't have an array type as such; we indeed recommend using nested fields for this.

Could you give more details into your use-case and why nested fields wouldn't work for you?