# How to serializing json object to string in ingest node

**URL:** <https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565>\
**Category:** Elasticsearch\
**Created:** [July 24, 2020, 8:00pm UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565 "2020-07-24T20:00:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![android.kc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/android.kc/32/58806_2.png) [@android.kc](https://discuss.elastic.co/u/android.kc)\
**Post date:** [July 24, 2020, 8:00pm UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/1 "2020-07-24T20:00:45Z")

</div>

I am trying to convert json objects **with certain key pattern** to string by using ingest API.

for example,

```auto
{
  "myObject": {
    "toSerialize_A": {"key_a": "a"},
    "toSerialize_B": {"key_b": "b"},
    "toSerialize_C": {"key_c": "c"},
    "toSerialize_D": {"key_d": "d"},
    "notToSerialize_A": {"key_a": "a"},
    "notToSerialize_B": {"key_b": "b"},
    "notToSerialize_C": {"key_c": "c"}
  }
}

```

only serialize myObject.toSerialize\_\* to string type but index myObject.notToSerialize\_\* as object type. Also, we only know the toSerialize\_\* or notToSerialize\_\* as prefix but **A, B, C**.. here are arbitrary

Found the following code worked fine by using convert processor, [Convert processor | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/convert-processor.html)

```auto
{
  "description": "converts the content of the myObj.a field to an integer",
  "processors" : [
    {
      "convert" : {
        "field" : "myObject.toSerialize_A",
        "type": "string"
      }
    }
  ]
}

```

However, it looks the convert processor doesn't support wildcards or pattern in object keys. also try the script processor [Script processor | Elasticsearch Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/script-processor.html) but doesn't support json serializing as discussed here ,

> [@Painless scripting JSON functions](https://discuss.elastic.co/t/painless-scripting-json-functions/98511):
>
> In Painless scripting how can I do JSON.parse and JSON.stringify? I have kept JSON data in the single string field, but now I want to delete some of the fields. So I will be needing JSON functions.

> <https://github.com/elastic/elasticsearch/issues/37585>
>
> \*\*Describe the feature\*\*:
> 
> The \[Painless API reference\](https://www.elastic.co…/guide/en/elasticsearch/painless/master/painless-api-reference.html) does not contain any function that deals with JSON. Would it be possible to provide basic JSON functionally in Painless? If possible the serialization should be deterministic, see below.
> 
> I am including full ES document dumps in emails send by watches. Because the \[\`toJson\` Mustache function is not deterministic\](https://github.com/elastic/elasticsearch/issues/35989) I needed to implement my own deterministic serialization function in Painless and realized that this is somewhat painful (e.g. not as it should be ;-) ).
> 
> This has been asked before: https://discuss.elastic.co/t/painless-scripting-json-functions/98511
> 
> For reference, this is my workaround. My code does not produce valid JSON and probably has bugs (not extensively tested). To work without regex support in Painless, I also put HTML indent support into this function.
> 
> \`\`\`Java
> /\* JSON-like serialization Painless {{{
> \* Ref: https://github.com/elastic/elasticsearch/issues/37585
> \* License: Public domain (because the code is too simple).
> \*/
> String get\_indent\_string(boolean html\_output, int indent) {
> if (html\_output) {
> return String.join("", Collections.nCopies(indent, "&nbsp;&nbsp;"));
> } else {
> return String.join("", Collections.nCopies(indent, " "));
> }
> }
> String serializ\_leaf(def object) {
> if (object == null) {
> return "null";
> } else if (object instanceof String) {
> return '"' + object + '"';
> } else {
> return object.toString();
> }
> }
> 
> ArrayList object\_to\_human\_readable\_string\_lines(def object) {
> return object\_to\_human\_readable\_string\_lines(object, false, 0, true);
> }
> 
> ArrayList object\_to\_human\_readable\_string\_lines(def object, boolean html\_output) {
> return object\_to\_human\_readable\_string\_lines(object, html\_output, 0, true);
> }
> 
> ArrayList object\_to\_human\_readable\_string\_lines(def object, boolean html\_output, int indent) {
> return object\_to\_human\_readable\_string\_lines(object, html\_output, indent, true);
> }
> 
> ArrayList object\_to\_human\_readable\_string\_lines(def object, boolean html\_output, int indent, boolean indent\_first\_line) {
> ArrayList serialized\_lines = new ArrayList();
> 
> if (object instanceof Collection) {
> serialized\_lines.add("\[");
> indent += 1;
> for (def nested\_object : object) {
> if (nested\_object instanceof Collection || nested\_object instanceof Map) {
> serialized\_lines.addAll(object\_to\_human\_readable\_string\_lines(nested\_object, html\_output, indent));
> } else {
> serialized\_lines.add(get\_indent\_string(html\_output, indent) + serializ\_leaf(nested\_object) + ",");
> }
> }
> indent -= 1;
> serialized\_lines.add(get\_indent\_string(html\_output, indent) + "\]" + (indent == 0 ? "" : ","));
> } else if (object instanceof Map) {
> serialized\_lines.add(get\_indent\_string(html\_output, (indent\_first\_line ? indent : 0)) + "{");
> indent += 1;
> List keys = object.keySet().stream().sorted().collect(Collectors.toList());
> for (String key : keys) {
> def nested\_object = object\[key\];
> if (nested\_object instanceof Collection || nested\_object instanceof Map) {
> ArrayList nested\_serialized\_lines = object\_to\_human\_readable\_string\_lines(nested\_object, html\_output, indent, false);
> serialized\_lines.add(get\_indent\_string(html\_output, indent) + key + ": " + nested\_serialized\_lines.remove(0));
> serialized\_lines.addAll(nested\_serialized\_lines);
> } else {
> serialized\_lines.add(get\_indent\_string(html\_output, indent) + key + ": " + serializ\_leaf(nested\_object) + ",");
> }
> }
> indent -= 1;
> serialized\_lines.add(get\_indent\_string(html\_output, indent) + "}" + (indent == 0 ? "" : ","));
> } else {
> serialized\_lines.add(serializ\_leaf(object) + ",");
> }
> 
> return serialized\_lines;
> }
> /\* }}} \*/
> \`\`\`

What would be the best way to handle this, thanks!

---

<div class="post-metadata">

**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\
**Post date:** [July 24, 2020, 8:19pm UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/2 "2020-07-24T20:19:21Z")

</div>

@android.kc  
Can you post what output are you expecting? Will toSerialize\_A value is arbitrary json or a fixed structure?

---

<div class="post-metadata">

**Author:** ![android.kc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/android.kc/32/58806_2.png) [@android.kc](https://discuss.elastic.co/u/android.kc)\
**Post date:** [July 24, 2020, 9:31pm UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/3 "2020-07-24T21:31:13Z")

</div>

@Vinayak_Sapre  
sure

here is the expected output

```auto
{
  "myObject": {
    "toSerialize_A": "{\"key_a\":\"a\"}",
    "toSerialize_B": "{\"key_b\":\"b\"}",
    "toSerialize_C": "{\"key_c\":\"c\"}",
    "toSerialize_D": "{\"key_d\": \"d\"}",
    "notToSerialize_A": {"key_a": "a"},
    "notToSerialize_B": {"key_b": "b"},
    "notToSerialize_C": {"key_c": "c"}
  }
}

```

values of toSerialize\_\* are string type  
values of notToSerialize\_\* remind as object ype

> [@Vinayak\_Sapre](#):
>
> Will toSerialize\_A value is arbitrary json or a fixed structure?

The values of toSerialize\_\* will be arbitrary json,

Thanks!

---

<div class="post-metadata">

**Author:** ![Vinayak\_Sapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinayak_sapre/32/45939_2.png) [@Vinayak\_Sapre](https://discuss.elastic.co/u/Vinayak_Sapre)\
**Post date:** [July 25, 2020, 3:46am UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/4 "2020-07-25T03:46:40Z")

</div>

@android.kc

> [@android.kc](#):
>
> Found the following code worked fine by using convert processor,

The convert processor simply calls toString which will generate`"toSerialize_A" : "{key_a=a}"` as value is a HashMap. This is different than what you are expecting `"toSerialize_A": "{\"key_a\":\"a\"}"`

If you are okay with this string, you can use script processor which will give you access to all keys of myObject. You can then filter keys you want and replace their value by value.toString()

---

<div class="post-metadata">

**Author:** ![gasilskibe](https://avatars.discourse-cdn.com/v4/letter/g/ad7895/32.png) [@gasilskibe](https://discuss.elastic.co/u/gasilskibe)\
**Post date:** [July 25, 2020, 8:10am UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/5 "2020-07-25T08:10:12Z")

</div>

Thank you for sharing superb informations. Your website is very cool. I’m impressed by the details that you have on this website.

---

<div class="post-metadata">

**Author:** ![gasilskibe](https://avatars.discourse-cdn.com/v4/letter/g/ad7895/32.png) [@gasilskibe](https://discuss.elastic.co/u/gasilskibe)\
**Post date:** [July 27, 2020, 7:13am UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/6 "2020-07-27T07:13:36Z")

</div>

Happy to help! Thanks for leaving a comment[.](https://www.mygroundbiz.online/)

---

<div class="post-metadata">

**Author:** ![android.kc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/android.kc/32/58806_2.png) [@android.kc](https://discuss.elastic.co/u/android.kc)\
**Post date:** [July 27, 2020, 9:56pm UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/7 "2020-07-27T21:56:37Z")

</div>

@Vinayak_Sapre thank you very much for the great help, I got it worked!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 24, 2020, 9:56pm UTC](https://discuss.elastic.co/t/how-to-serializing-json-object-to-string-in-ingest-node/242565/8 "2020-08-24T21:56:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.