# Ruby help with array of json objects **URL:** \ **Category:** Logstash\ **Created:** [July 13, 2020, 5:58pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022 "2020-07-13T17:58:56Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 5:58pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/1 "2020-07-13T17:58:56Z")
hi sorry if its repeat question. In my document there are few fields which are array of json objects. I want to extract one of json array object's value and assign it to my new field. So for that i m using ruby code to convert an array of json objects in to flat json , copying it to tmp field and then trying to extract value from temp field but its not working. Here is my logstash conf. ``` input { beats { port => 5044 } } filter { json { source => "message" } ruby { code => ' event.set("request_http_headers_tmp",event.get("request_http_headers").to_json) ' } if "transaction-id" in [request_http_headers_tmp] { mutate{ add_field => { "transaction-id" => "%{request_http_headers_tmp.transaction-id}" } } } mutate { remove_field => ["@version","fields","tags","host","agent","log","message","version","request_http_headers"] } } output { stdout { codec => rubydebug } } ``` Here is my sample input ``` {"request_http_headers":[{"transaction-id": "1234" },{"raj":"test"}]} {"request_http_headers":[{"TraceId":"9d912b9aedf0f0d6"},{"Request-Id":"a58db2fb"},{"Sampled":"0"},{"SpanId":"109996b844d56118"},{"ParentSpanId":"afd752b39e"},{"Via":"1.1 AgAAALgHxKA-"},{"X-Client-IP":"1.1.1.1"},{"transaction-id":"ecbc4be05f0c88ff00c11f61"}]} ``` Here is my output ``` "request_http_headers_tmp" => "[{\"transaction-id\":\"1234\"},{\"raj\":\"test\"}]", "ecs" => { "version" => "1.0.1" }, "transaction-id" => "%{request_http_headers_tmp.transaction-id}", "@timestamp" => 2020-07-13T17:56:31.709Z } { "request_http_headers_tmp" => "[{\"TraceId\":\"9d912b9aedf0f0d6\"},{\"Request-Id\":\"a58db2fb\"},{\"Sampled\":\"0\"},{\"SpanId\":\"109996b844d56118\"},{\"ParentSpanId\":\"afd752b39e\"},{\"Via\":\"1.1 AgAAALgHxKA-\"},{\"X-Client-IP\":\"1.1.1.1\"},{\"transaction-id\":\"ecbc4be05f0c88ff00c11f61\"}]", "ecs" => { "version" => "1.0.1" }, "transaction-id" => "%{request_http_headers_tmp.transaction-id}", "@timestamp" => 2020-07-13T17:56:31.709Z } ``` any clue on what am i doing wrong ? ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 6:06pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/2 "2020-07-13T18:06:35Z")
i have tried %{[request\_http\_headers\_tmp][transaction-id]} , %[request\_http\_headers\_tmp][transaction-id] , %[request\_http\_headers\_tmp]%[transaction-id] but i just dont know how to make this work 😞 ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 6:15pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/3 "2020-07-13T18:15:07Z")
`%{[request_http_headers_tmp][transaction-id]}` would be the correct syntax. But you converted `request_http_headers_tmp` to JSON. So it is a string, not a hash and you can not access any fields in it. ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 6:16pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/4 "2020-07-13T18:16:07Z")
so what would be better approach here ? ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 6:17pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/5 "2020-07-13T18:17:13Z")
Either do not call to\_json or do it after extracting the information you needed. You could also get these fields from the original request\_http\_headers. I'm not sure what exactly your goal is and why you are copying that field. Your desired value is in `[request_http_headers][0][transaction_id]` If it isn't always in the first entry, you have to loop over the array with Ruby to find it. ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 6:26pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/6 "2020-07-13T18:26:46Z")
> [@Jenni](#): > > `%{[request_http_headers_tmp][transaction-id]}` ok let me try that approach. Thanks . I will update in few. ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 6:30pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/7 "2020-07-13T18:30:08Z")
But that won't work?! I think what you wanted to do is merge an array of hashes in ruby. Then you'd have the desired structure. But that's too uncomfortable to google on a smartphone. So I'll leave that up to you 🙂 ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 6:57pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/8 "2020-07-13T18:57:49Z")
Back at a computer for a minute. Sorry that I didn't give you a clear solution from the start. It took me a moment to understand your thought process. You wanted to change the structure of the header field to be able to access your ID that is in one of the array entries. `to_json` doesn't do that. It just encodes that very same structure as a JSON string. What you really want to do is this: ```auto event.set('request_http_headers', event.get('request_http_headers').reduce({}, :merge!)) ``` Then your ID should by accessible as `[request_http_headers][transaction-id]`. ```auto mutate{ copy => { "[request_http_headers][transaction-id]" => "transaction-id" } } ``` I haven't tried it. But this way it sounds logical to me 🙂 ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 7:13pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/9 "2020-07-13T19:13:23Z")
> [@Jenni](#): > > `copy => { "[request_http_headers][transaction-id]" => "transaction-id" }` so other thing is not 100% transaction-id will be there in document. so will followng do he trick ? ``` if [request_http_headers][transaction-id] { mutate{ copy => { "[request_http_headers][transaction-id]" => "transaction-id" } } ``` } ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 7:25pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/10 "2020-07-13T19:25:15Z")
Yes. You can build a condition. But copy can't do anything anyway if the field that should be copied does not exist 🙃 ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 7:26pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/11 "2020-07-13T19:26:45Z")
logstash failing on this . i m not expert at ruby 🙂 event.set('request\_http\_headers', event.get('request\_http\_headers').reduce({}, :merge!)) ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 7:32pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/12 "2020-07-13T19:32:04Z")
So what does Logstash say? (Remember to use double quotes to wrap the code if you take it the way it is because I used single quotes in the code. Or change my code accordingly. I know that you did it the opposite way around in your original configuration.) ```auto ruby { code => "event.set('request_http_headers', event.get('request_http_headers').reduce({}, :merge!))" } ``` I'll need your error message because I don't currently have a test system at hand. ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 7:34pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/13 "2020-07-13T19:34:09Z")
`:exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, } at line 14, column 35 (byte 330) after filter {\n\n` here is my config i m using ``` input { beats { port => 5044 } } filter { json { source => "message" } ruby { code => ' event.set("request_http_headers_tmp",event.get("request_http_headers").to_json) event.set('request_http_headers1', event.get('request_http_headers').reduce({}, :merge)) ' } if "transaction-id" in [request_http_headers_tmp] { mutate{ #add_field => { "transaction-id" => "%{request_http_headers_tmp.transaction-id}" } copy => { "[request_http_headers_tmp][transaction-id]" => "transaction-id" } } } mutate { remove_field => ["@version","fields","tags","host","agent","log","message","version","request_http_headers","ecs"] } } output { stdout { codec => rubydebug } } ``` ---
**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\ **Post date:** [July 13, 2020, 7:39pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/14 "2020-07-13T19:39:32Z")
Read my previous answer. That's exactly the quote problem I had predicted. It causes a syntax error. And remember for the future: The code extract in those syntax error messages in the log usually ends exactly at the positiion of the problem. So the one you posted was probably longer than what you posted and ended somewhere in the Ruby code (Right?!) ---
**Author:** ![rajsolanki](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajsolanki/32/86859_2.png) [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Post date:** [July 13, 2020, 7:42pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/15 "2020-07-13T19:42:52Z")
crap sorry i missed your previous reply. yes its quote issue how genius of me lol thanks a lot ! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 10, 2020, 7:43pm UTC](https://discuss.elastic.co/t/ruby-help-with-array-of-json-objects/241022/16 "2020-08-10T19:43:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.