Bad charset encoding in field names (II)

Elastic Stack Elasticsearch
1

Hi!

I have exactly the same issue as described by Ondřej Bílka on this thread:

Since the discussion was closed I'm trying to reopen it again.

All my files are in UTF-8 and just the dinamic fields are wrong - I have one 'message' field with all the data and on that field the special characters are ok.

Thanks.

2

Hi, my only solution was to change the keys to web url-safe slugs...

3

Sorry to hear that....

I'll try to solve this issue a little longer.

This was not a problem until I updated all ELK tools from 5.5.2 to 6.2.3

4

I wish you good luck :slight_smile: and if you will figure anything out, please let me know..
On the other hand I think that "webalized" string keys are much more secured because of possible encoding issues in another parts of application.

5

Yeah... i gave up and used this on my ugly field inside of a ruby filter:

event.set(strKey.tr( "ÀÁÂÃÄÅàáâãäåAaAaAaÇçCcCcCcCcÐðDdÐdÈÉÊËèéêëEeEeEeEeEeGgGgGgGgHhHhÌÍÎÏìíîïIiIiIiIiIiJjKk?LlLlLl??LlÑñNnNnNn???ÒÓÔÕÖØòóôõöøOoOoOoRrRrRrSsSsSsŠš?TtTtTtÙÚÛÜùúûüUuUuUuUuUuUuWwÝýÿYyŸZzZzŽž", "AAAAAAaaaaaaAaAaAaCcCcCcCcCcDdDdDdEEEEeeeeEeEeEeEeEeGgGgGgGgHhHhIIIIiiiiIiIiIiIiIiJjKkkLlLlLlLlLlNnNnNnNnnNnOOOOOOooooooOoOoOoRrRrRrSsSsSsSssTtTtTtUUUUuuuuUuUuUuUuUuUuWwYyyYyYZzZzZz"), strValue.to_f)

Don't know how is the performance of this, though I dont have a high volume of data going through this code..

6

There is still a problem when I use a docker image sebp/elk 6.2.3 or 6.2.4.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.