Fields are not shown in "Available fields" in KIbana

[Manav_Chopra]

I have added fields i.e through mapping API and all the fields are shown on index management> index pattern page
But when I go to Discover tab, no new fields are shown there.
Can you please tell what is the missing part and also for visualization, something more is required to use those fields while visualization?

I have done below field addition in the index in kibana dev tools using PUT index/_mapping. I want these fields to be shown on "Available fields"
Please refer https://pastebin.com/z9YrN1wW

[majagrubic (Maja)]

If you have newly added the fields, you will need to refresh the field list in the index pattern. Go to Management > Index Patterns and select refresh option. If you had already done that (and it sounds like you have), could you go to the dev console and try to retrieve the documents from your index? Are the newly added fields shown there?

GET /<your_index>/_search
{
      "query": {
       "match_all": {}
   }
}

If so, next thing is to check in Discover itself that you have "Hide missing fields" option deselected.

[Manav_Chopra]

Thanks for your reply!!
I just unselected 'Hide missing fields' , it then appeared in available fields. But when I
hit

GET /<your_index>/_search

in dev tools, that fields are not shown there.

Also that fields are not available in the data present in "Discover results". I want them to be automatically present there with data.
I mean, I know i can add/remove the fields from available fields so the added one's will be shown in Discover results but that is appearing with a -(dash). i.e no data in the column.
So how can those newly added fields should be present there with data in it instead -
Please refer attachment

[majagrubic (Maja)]

When you select the Bytes field for display (before clicking add), do you get this message?

I assume what happens here is that your documents just don't have any value for that field set.

[Manav_Chopra]

No, I didn't get this message.
I am using logstash and afterwards I have just gone to Kibana dev tools and used PUT mapping API which I have already shared in previous post and thus, fields got shown in available fields.
Is there something I need to index them or what should be done to get value for that field set.
I am using elastic cloud-trial version, so anything needs to be done with the logs that got generated and shown on discover results??
At the end I am seeing one line:
This are the first 500 document matching your search criteria. refine your search to get more results.

[majagrubic (Maja)]

After updating the mapping, you will need to refresh the index pattern, as I already described in the first reply. And I believe you had already done that, since you're seeing the fields in Discover. Try narrowing down your search results, to make sure you are including the documents that have the new field set. I still believe that what happens here is that documents you're seeing simply don't have that value set.

[Manav_Chopra]

I have no idea about it, how can I check that document is included or not, because it is shown in the index pattern and yes I have refreshed them as well. any reference link if provided, then it will be helpful for me.

One thing I need to clarify, is something needs to be added in logstash.conf file? or filebeat.input?because i have added only input and output. No filters added or no information exist in that except input and output.

Also if document doesn't have value set, so how can I get or proceed to get value in it?

[majagrubic (Maja)]

I'm not very familar with Logstash, I need to check with my colleagues. From the Kibana side - if you've changed the ES mapping, the new field should show in Discover. But until you have no documents that have that field set, it will show exactly as it is showing for you now.