I am trying to convert json objects with certain key pattern to string by using ingest API.
for example,
{
"myObject": {
"toSerialize_A": {"key_a": "a"},
"toSerialize_B": {"key_b": "b"},
"toSerialize_C": {"key_c": "c"},
"toSerialize_D": {"key_d": "d"},
"notToSerialize_A": {"key_a": "a"},
"notToSerialize_B": {"key_b": "b"},
"notToSerialize_C": {"key_c": "c"}
}
}
only serialize myObject.toSerialize_* to string type but index myObject.notToSerialize_* as object type. Also, we only know the toSerialize_* or notToSerialize_* as prefix but A, B, C .. here are arbitrary
Found the following code worked fine by using convert processor, Convert processor | Elasticsearch Guide [8.11] | Elastic
{
"description": "converts the content of the myObj.a field to an integer",
"processors" : [
{
"convert" : {
"field" : "myObject.toSerialize_A",
"type": "string"
}
}
]
}
However, it looks the convert processor doesn't support wildcards or pattern in object keys. also try the script processor Script processor | Elasticsearch Guide [master] | Elastic but doesn't support json serializing as discussed here ,
In Painless scripting how can I do JSON.parse and JSON.stringify?
I have kept JSON data in the single string field, but now I want to delete some of the fields. So I will be needing JSON functions.
opened 06:54PM - 17 Jan 19 UTC
:Core/Infra/Scripting
Team:Core/Infra
**Describe the feature**:
The [Painless API reference](https://www.elastic.co… /guide/en/elasticsearch/painless/master/painless-api-reference.html) does not contain any function that deals with JSON. Would it be possible to provide basic JSON functionally in Painless? If possible the serialization should be deterministic, see below.
I am including full ES document dumps in emails send by watches. Because the [`toJson` Mustache function is not deterministic](https://github.com/elastic/elasticsearch/issues/35989) I needed to implement my own deterministic serialization function in Painless and realized that this is somewhat painful (e.g. not as it should be ;-) ).
This has been asked before: https://discuss.elastic.co/t/painless-scripting-json-functions/98511
For reference, this is my workaround. My code does not produce valid JSON and probably has bugs (not extensively tested). To work without regex support in Painless, I also put HTML indent support into this function.
```Java
/* JSON-like serialization Painless {{{
* Ref: https://github.com/elastic/elasticsearch/issues/37585
* License: Public domain (because the code is too simple).
*/
String get_indent_string(boolean html_output, int indent) {
if (html_output) {
return String.join("", Collections.nCopies(indent, " "));
} else {
return String.join("", Collections.nCopies(indent, " "));
}
}
String serializ_leaf(def object) {
if (object == null) {
return "null";
} else if (object instanceof String) {
return '"' + object + '"';
} else {
return object.toString();
}
}
ArrayList object_to_human_readable_string_lines(def object) {
return object_to_human_readable_string_lines(object, false, 0, true);
}
ArrayList object_to_human_readable_string_lines(def object, boolean html_output) {
return object_to_human_readable_string_lines(object, html_output, 0, true);
}
ArrayList object_to_human_readable_string_lines(def object, boolean html_output, int indent) {
return object_to_human_readable_string_lines(object, html_output, indent, true);
}
ArrayList object_to_human_readable_string_lines(def object, boolean html_output, int indent, boolean indent_first_line) {
ArrayList serialized_lines = new ArrayList();
if (object instanceof Collection) {
serialized_lines.add("[");
indent += 1;
for (def nested_object : object) {
if (nested_object instanceof Collection || nested_object instanceof Map) {
serialized_lines.addAll(object_to_human_readable_string_lines(nested_object, html_output, indent));
} else {
serialized_lines.add(get_indent_string(html_output, indent) + serializ_leaf(nested_object) + ",");
}
}
indent -= 1;
serialized_lines.add(get_indent_string(html_output, indent) + "]" + (indent == 0 ? "" : ","));
} else if (object instanceof Map) {
serialized_lines.add(get_indent_string(html_output, (indent_first_line ? indent : 0)) + "{");
indent += 1;
List keys = object.keySet().stream().sorted().collect(Collectors.toList());
for (String key : keys) {
def nested_object = object[key];
if (nested_object instanceof Collection || nested_object instanceof Map) {
ArrayList nested_serialized_lines = object_to_human_readable_string_lines(nested_object, html_output, indent, false);
serialized_lines.add(get_indent_string(html_output, indent) + key + ": " + nested_serialized_lines.remove(0));
serialized_lines.addAll(nested_serialized_lines);
} else {
serialized_lines.add(get_indent_string(html_output, indent) + key + ": " + serializ_leaf(nested_object) + ",");
}
}
indent -= 1;
serialized_lines.add(get_indent_string(html_output, indent) + "}" + (indent == 0 ? "" : ","));
} else {
serialized_lines.add(serializ_leaf(object) + ",");
}
return serialized_lines;
}
/* }}} */
```
What would be the best way to handle this, thanks!
@android.kc
@Vinayak_Sapre
here is the expected output
{
"myObject": {
"toSerialize_A": "{\"key_a\":\"a\"}",
"toSerialize_B": "{\"key_b\":\"b\"}",
"toSerialize_C": "{\"key_c\":\"c\"}",
"toSerialize_D": "{\"key_d\": \"d\"}",
"notToSerialize_A": {"key_a": "a"},
"notToSerialize_B": {"key_b": "b"},
"notToSerialize_C": {"key_c": "c"}
}
}
values of toSerialize_* are string type
The values of toSerialize_* will be arbitrary json,
Thanks!
@android.kc
The convert processor simply calls toString which will generate"toSerialize_A" : "{key_a=a}" as value is a HashMap. This is different than what you are expecting "toSerialize_A": "{\"key_a\":\"a\"}"
If you are okay with this string, you can use script processor which will give you access to all keys of myObject. You can then filter keys you want and replace their value by value.toString()
Thank you for sharing superb informations. Your website is very cool. I’m impressed by the details that you have on this website.
Happy to help! Thanks for leaving a comment.
@Vinayak_Sapre thank you very much for the great help, I got it worked!
system
August 24, 2020, 9:56pm
8
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.