Kibana text field in discover but missing in lens table

[chowpay]

I'm on Kibana 7.16

When I'm in discover I can see this text field:

I want the field inside of a table, but when I go to lens to try and create the table it shows up in an empty field:

In discover his is the field type:

The time and index is correct. Is it because its a multi line field?

[jughosta (Julia Rechkunova)]

Hi @chowpay ,

I would suggest to check ignore_above value in the mapping for content-head.keyword field. If the configured value is currently lower than any of content-head values then increasing ignore_above and reindexing should populate content-head.keyword with data.

[stephenb (Stephen Brown)]

Hi @chowpay

What version are you on?

Even though it says it's empty, did you try to drag the .keyword (text field won't work) into a Lens table? Say rows?

Also, is the time picker correct or do you have any other filters in lens?

Can you show a bigger / whole screenshot?

[chowpay]

Hi @stephenb
7.16

Yes I've tried dragging it over and its empty.
There is no non-keyword option

ive tried dragging it in columns or rows all the same
time picker is correct beacuse I can see the field in discover just fine

Full screen shot:

Discover shows the field data

[chowpay]

Ah I found it:

        "content-head": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },

I have many indexes broken up by time

How would I adjust the mapping for all my indexes? and is this where I should be doing it?

I took 1 of the fields:

#EXTM3U
#EXT-X-VERSION:5
#EXT-X-MEDIA-SEQUENCE:52437
#EXT-X-TARGETDURATION:8
#EXT-X-DISCONTINUITY-SEQUENCE:5859
#UPLYNK-SEGMENT: 48f6499fce4a4a44954cfd70dc8bad7b,00000000,ad
#UPLYNK-KEY:https://content-ausw1.uplynk.com/check?b=48f6499fce4a4a44954cfd70dc8bad7b&c=3324f2467c414329b3b0cc5cd987b6be&pbs=f4d8f31702a7438f87f05cdd98051deb
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2024-02-29T17:20:47.271000+00:00
#EXT-X-KEY:METHOD=AES-128,URI="https://content-ausw1.uplynk.com/check2?b=48f6499fce4a4a44954cfd70dc8bad7b&v=3324f2467c414329b3b0cc5cd987b6be&r=f&c=3324f2467c414329b3b0cc5cd987b6be&pbs=f4d8f31702a7438f87f05cdd98051deb",IV=0x00000000000000000000000000000000
#EXTINF:4.0960,
https://x-default-stgec.uplynk.com/ause/slices/48f/a4e18e29d9624114a7eac2829aac559e/48f6499fce4a4a44954cfd70dc8bad7b/F00000000.ts?pbs=f4d8f31702a7438f87f05cdd98051deb&_jt=l&chid=3324f2467c414329b3b0cc5cd987b6be&cloud=aws&cdn=eci&si=1&d=4.096

Then did a character count:
Total characters (without spaces) 912

I'm guessing "Ignore_above" count is per character?

Thanks!

[jughosta (Julia Rechkunova)]

Here are some documentation pages which can help with updating mappings:

ignore_above | Elasticsearch Guide [8.12] | Elastic

Update mapping API | Elasticsearch Guide [8.12] | Elastic

This will also require reindexing the data.

The second option is to create a new runtime field with keyword type which would be a copy of content-head text field.

[chowpay]

I tried the 2nd option since in reality I would need to join 2 fields , using this as one of the fields. The runtime is very simple:

It works with other keyword fields, but when trying to access this specific one I get this error. "class _cast_exception" . Am I doing my emit incorrectly? Thanks

[jughosta (Julia Rechkunova)]

It's most probably undefined. Try adding checks like if (doc["<field name>"].size() > 0) { ... } or if (doc.containsKey('<field name>')) { ... }.

[chowpay]

I tried both but maybe its my syntax:

if(doc['content-head.keyword'].size()>0) {emit(doc['content-head.keyword'])}

When I do this and go to discover the field doesn't appear

[jughosta (Julia Rechkunova)]

If ignore_above stays the same in the mapping and data is not reindexed, then content-head.keyword would be still empty for documents with long content-head.

Using content-head.keyword inside a runtime field script would not help. But you can use other populated fields like content-head. So start with if (doc['content-head'].size()>0) { emit(doc['content-head'].value) }.