My Logstash instance will be receiving JSON records containing a wide variety of nested fields whose names start with [data][ecs].... and I need to trim the [data] prefix off of all of those field names before indexing. There will be other non-ecs [data] field names that must retain the [data] prefix.
For example, [data][ecs][source][ip] needs to change to [ecs][source][ip]
but [data][org][name] needs to stay the same.
From my research so far, it sounds like this would be a job for Ruby, and this example seems to hint at the soution:
but I'm not strong enough with Ruby to go the distance from there.
Would anyone be willing to at least get me off to a strong start on getting Ruby to enumerate through all field names in the current record that start with [data][ecs] and to trim off the [data] prefix from each of them?
I sure would appreciate a hand on this. I believe the ability to dynamically adjust the position of certain nodes inside the field tree structure of records being processed by Logstash would be of broader value to the community than just my use case. I just wish I were more experienced with ruby. Please help if you can.
I'm pretty sure that event.to_hash gives you a multidimensional data structure, so there would be a key "data", not "[data][ecs]". But I think you would only need ruby for this kind of action if your target is the root level of the event (many threads for this, e.g.: Move subarrays to document root). If there is a target field, simply renaming your field should work: