Extract value from a field using painless script

[yuswanul]

Hi there,

i have a use case and i want to try it using painless. so the use case is there is a field containing a valid json and i want to extract just one field from it. basically it's a bit like using grok. the field name is requestPayload and here is the value:

{"transaction_id":"xxxxx","channel":"f0","language":"id","version":"v2","service_id_a":"xxxxx","service_id_b":"xxxxx","offer_id":"113908","adn":"133","order_type":"ACT","purchase_mode":"REDEEM","payment_method":"PHYSICALVOUCHER","payment_name":"PHYSICALVOUCHER","kabupaten":"xxxxx","service_region":"1","subscription_flag":0,"reservation_flag":0,"property_list":{"property":null,"name":null},"dyn_properties":{"dyn_property":null,"name":null},"customer_info":{"customer_name":null,"customer_email":null},"expiry":{"duration":null,"uom":null},"merchant_profile":{"merchant_signature":null,"cai":null,"cati":null,"can":null,"fund_source":null,"address":null,"post_code":null,"district":null,"store_id":null,"city":null,"coordinate":null,"delivery_channel":null,"transmission_date":null,"field1":null,"field2":null,"field3":null,"field4":null,"field5":null,"fund_type":null,"business_model":null},"admin_fee":0,"hrn":"e42612e71ca54e4394e2b2496d44ac24b03f8ae144e47b7c276b6836ae8e2907"}

the one field that i want to extract from that value is offer_id.

is it possible if i'm using painless in scripted field to achieve that?

i know i can achieve that by using logstash but there are so many logstash nodes in my cluster. it feels too troublesome. please let me know if you have any idea

Thanks

[ppisljar (Peter Pisljar)]

you can crerate a runtime field with a script like this

String clientip=grok('Returning %{NUMBER:foobar} ads').extract(params._source.message)?.foobar;
emit(clientip);

to grok parse and emit a single field

you can use composite runtime field and then you can just do:

emit(grok('....').extract(..))

[yuswanul]

Is it possible in kibana v7.17.0?

Pada tanggal Rab, 7 Jun 2023 16.48, Peter Pisljar via Discuss the Elastic Stack <notifications@elastic.discoursemail.com> menulis:

[ppisljar (Peter Pisljar)]

the first option yes, composite runtime fields are however not supported in that kibana version.
you could create multiple fields using the first option to extract more parts.

[yuswanul]

for my case. since offer_id is not in the beginning of the line, should i make a grok pattern like this?

%{GREEDYDATA}"offer_id":"%{NUMBER:offer_id}"%{GREEDYDATA}

or i just simply create like this?

"offer_id";"%{NUMBER:offer_id}"

and if you don't mind, could you make an example according to my case? i don't really understand what's the point of Returning there? is that just example or it's mandatory to apply in my scripted field? and ads also

Thanks

[yuswanul]

i've tried to make the script like this based on this documentation

unfortunately, i got this error in grok part

that's quite surprising because the error is not on pattern or extract part but right on the grok filter

i was tried to make a script based on your example before but the script has return the same error