Field not showing in "Available fields" unless I un-check "Hide missing fields"

[Jody_McIvor]

Hi Everyone! I'm newish to the forums but have been running multiple stacks for my workplace for a couple years now, although my comfort level is still pretty low when it comes to usage, I can build/deploy ELK stacks blindfolded... lol

So here's what is going on:
Existing logstash index(s) with incoming data that contains a variable called "auxiliarydevice_info" and for the life of me I can't seem to get it to actually USE it as a field.

Here's a screenshot of a record containing this field:

Here is a screenshot of it present in the currently viewed indexes mapping script:

Here's a screenshot of it in Kibana index patterns:

Here's the contents of its Elasticsearch>IndexManagement>IndexTemplates record that I originally created it in:

Not sure what I'm doing wrong, it LOOKS to me that everything is lined up and it SHOULD be picking up this field and highlighting it light blue like it does the rest, but instead it is acting like the string isn't even present in log records, despite seeing them when I manually search for auxiliarydevice_info. I've attempted to figure it out on my own for a few days but I'm defeated, lol.

thoughts?

[wylie (Wylie Conlon)]

Discover before 7.12 shows you only what is available in the _source of your document, which in your case is just auxiliarydevice_info but not auxiliarydevice_info.keyword. The .keyword field will never appear in Discover because the values are stored in docvalues, which is a separate place in Elasticsearch.

In 7.12, we have changed the default way that Discover presents fields and one of the changes addresses your question. Here is the blog post about the changes:

Discover uses the fields API in 7.12