How to serializing json object to string in ingest node

Elastic Stack Elasticsearch
1

I am trying to convert json objects with certain key pattern to string by using ingest API.

for example,

{
  "myObject": {
    "toSerialize_A": {"key_a": "a"},
    "toSerialize_B": {"key_b": "b"},
    "toSerialize_C": {"key_c": "c"},
    "toSerialize_D": {"key_d": "d"},
    "notToSerialize_A": {"key_a": "a"},
    "notToSerialize_B": {"key_b": "b"},
    "notToSerialize_C": {"key_c": "c"}
  }
}

only serialize myObject.toSerialize_* to string type but index myObject.notToSerialize_* as object type. Also, we only know the toSerialize_* or notToSerialize_* as prefix but A, B, C .. here are arbitrary

Found the following code worked fine by using convert processor, Convert processor | Elasticsearch Guide [8.11] | Elastic

{
  "description": "converts the content of the myObj.a field to an integer",
  "processors" : [
    {
      "convert" : {
        "field" : "myObject.toSerialize_A",
        "type": "string"
      }
    }
  ]
}

However, it looks the convert processor doesn't support wildcards or pattern in object keys. also try the script processor Script processor | Elasticsearch Guide [master] | Elastic but doesn't support json serializing as discussed here ,

What would be the best way to handle this, thanks!

2

@android.kc
Can you post what output are you expecting? Will toSerialize_A value is arbitrary json or a fixed structure?

3

@Vinayak_Sapre
sure

here is the expected output

{
  "myObject": {
    "toSerialize_A": "{\"key_a\":\"a\"}",
    "toSerialize_B": "{\"key_b\":\"b\"}",
    "toSerialize_C": "{\"key_c\":\"c\"}",
    "toSerialize_D": "{\"key_d\": \"d\"}",
    "notToSerialize_A": {"key_a": "a"},
    "notToSerialize_B": {"key_b": "b"},
    "notToSerialize_C": {"key_c": "c"}
  }
}

values of toSerialize_* are string type
values of notToSerialize_* remind as object ype

The values of toSerialize_* will be arbitrary json,

Thanks!

4

@android.kc

The convert processor simply calls toString which will generate"toSerialize_A" : "{key_a=a}" as value is a HashMap. This is different than what you are expecting "toSerialize_A": "{\"key_a\":\"a\"}"

If you are okay with this string, you can use script processor which will give you access to all keys of myObject. You can then filter keys you want and replace their value by value.toString()

5

Thank you for sharing superb informations. Your website is very cool. I’m impressed by the details that you have on this website.

6

Happy to help! Thanks for leaving a comment.

7

@Vinayak_Sapre thank you very much for the great help, I got it worked!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.