Index Pattern behavior

Elastic Stack Kibana
1

Hi..

I have a logstash index and already delete_by_query of the part of the indexes, and no more particular fields, then I've done to delete the index pattern to make sure the fields is clean.
The strange behavior of the Kibana Index pattern is still have the fields that already removed in the index.
Is it expected ?
I'm using 7.6.2

Fadjar Tandabawana

2

Kibana index patterns cache the fields in your indices. If you go to the index pattern management page, there is a reload button for the index pattern to refresh fields.

See also here: https://www.elastic.co/guide/en/kibana/current/managing-fields.html ( Refresh the index fields list)

3

Yes...

I did... several times...

following the page:
Refresh the index fields list. You can refresh the index fields list to pick up any newly-added fields. Doing so also resets the Kibana popularity counters for the fields. The popularity counters are used in Discover to sort fields in lists.

What about deleted ones?

Still no luck to cleanup the old fields

4

Did you also remove them from the mapping of your indices?

5

That's what I'm afraid..
Never done before to remove the fields in the mapping.
I'm to afraid to remove mapping in production environment...

Any suggestion?
Or if it possible, to make it more automagically in the next release... :slight_smile:

Regards,
Fadjar Tandabawana

6

Kibana index patterns use the field caps api to fetch the list of fields from Elasticsearch. It seems like you have to remove the fields from the mapping to prevent them from showing up in the index pattern (which would include re-indexing existing data).

I can think of one workaround. Go to Management > saved objects, select your index pattern and export it. This will give you a JSON file of the index pattern. Go in there and remove the fields you don't want to show up anymore - be careful to not mess up the JSON syntax (no trailing commas and the like). Now re-import the file (this will overwrite the existing index pattern). Now the field should be gone from all UIs (Visualize, Discover). If you are refreshing the index pattern, you will have to do this again.

7

Hi @flash1293,

Already exported...
Unfortunately, the JSON format is not the JSON that I recognized..:slight_smile:
It's ndjson and makes me headache to read...

image
image1698×435 154 KB

I'm eager to do it, but there's too complicated, also the result in one line, I'm afraid, even I can modify it, the result is not what i'm expected, because of the complexity.
If in the new release can change to human readable JSON with proper indentation, I'll happy to modify it...

Regards,
Fadjar Tandabawana

8

Turning the export format into human readable json is not something on the roadmap (there are a bunch of technical hurdles)

If you can upload the complete file to pastebin or something else and tell me which fields to remove I can do it for you.

9

Thank you @flash1293...
I don't want to bother you...

I'll do it in my spare time...

Regards,
Fadjar Tandabawana

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.