Hello there,
Using an ingest pipeline, I am trying to uppercase the first char in a fieldname, for all fields under winlog.event_data:
{
"script": {
"lang": "painless",
"source": "ctx._source.winlog.event_data = ctx._source.last.replaceFirst(/[a-z]/, m ->\n m.group().toUpperCase(Locale.ROOT))"
}
},
How do I reference the field name within Painless, rather than the field value? To be clear, there are multiple fields under winlog.event_data
** Edited to add the following relevant Issue, as this is essentially what I need to do:
opened 04:56PM - 30 Jul 20 UTC
>enhancement
good first issue
help wanted
:Core/Infra/Scripting
:Data Management/Ingest
Team:Core/Infra
Team:Data Management
Each processor can add an "if" condition that runs a painless script to evaluate… if the processor should be run. The script process can also execute a painless script. However, currently the only information available to Painless (via ctx.) is the _source and meta data from the ingest document.
The for-each processor uses a special `_ingest` meta data to help keep track of which value you are working with. The `_ingest` metadata is not exposed to painless for either the script process or the "if" condition. This makes using painless with the for-each process not really possible since you can not pull the current value from the `_ingest.value`.
We should consider exposing the `_ingest` data to painless for the script processor and "if" condition. The workaround is to for-go the for-each process and do the necessary looping and processing via a top level (i.e. not inside a for-each) script processor. Additionally using the set and remove processor to read data from `{{_ingest.value}}` and then set a temporary field in the main document (and remove at the end of the pipeline) may allow for some (awkward) usage of painless within a for-each.
stu
(Stuart Tettemer)
January 5, 2022, 4:03pm
3
Hi @DefensiveDepth , ctx
is a Map of maps (and lists, depending on the document), so you can remove keys in place or replace the map entirely. For example:
POST /_ingest/pipeline/_simulate?verbose=true
{
"pipeline": {
"processors": [
{
"script": {
"lang": "painless",
"source": """
Map eventData = ctx['winlog']['event_data'];
Map updatedEventData = new HashMap();
for (String key: eventData.keySet()) {
updatedEventData[key.substring(0,1).toUpperCase() + key.substring(1)] = eventData[key]
}
ctx['winlog']['event_data'] = updatedEventData
"""
}
}
]
},
"docs": [
{
"_source": {
"winlog": {
"event_data": {
"abc": 123,
"def": "hij"
}
}
}
}
]
}
Results:
{
"docs" : [
{
"processor_results" : [
{
"processor_type" : "script",
"status" : "success",
"doc" : {
"_index" : "_index",
"_type" : "_doc",
"_id" : "_id",
"_source" : {
"winlog" : {
"event_data" : {
"Def" : "hij",
"Abc" : 123
}
}
},
"_ingest" : {
"pipeline" : "_simulate_pipeline",
"timestamp" : "2022-01-05T16:02:13.747478961Z"
}
}
}
]
}
]
}
Fantastic, thanks so much @stu !
system
(system)
Closed
February 3, 2022, 7:56pm
5
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.