# How to handle nil or null values within mutate add field **URL:** \ **Category:** Logstash\ **Created:** [June 8, 2020, 10:55am UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160 "2020-06-08T10:55:50Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\ **Post date:** [June 8, 2020, 10:55am UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/1 "2020-06-08T10:55:50Z")
So I have my data being parsed , transformed correctly however I have this issue where null or nil values cause me problems. "userActions.cdnBusyTime" =\> "%{[userActions][cdnBusyTime]}" output ends up looking like this "userActions.cdnBusyTime" : "%{[userActions][cdnBusyTime]}", So far I have tried several things 1. only change if value exists if [userActions][cdnResources] { mutate {add\_field =\> {"userActions.cdnBusyTime" =\> "%{[userActions][cdnBusyTime]}" } 2. remove value before if null filter { ruby { code =\> "event.to\_hash.delete\_if {|field, value| value == '' }" } } 3. remove theh field later if it contains the name if [userActions][cdnBusyTime] in "cdnBusyTime" { mutate { remove\_field =\> ["[userActions][cdnBusyTime]"] } } 4. convert and test against !\> 0 mutate { convert =\> ["[userActions][cdnBusyTime]", "integer" ] } if [userActions][cdnBusyTime] !\> 0 { mutate { remove\_field =\> ["[userActions][cdnBusyTime]" ] } } There has to be a million ways to do this. But Which one works? any suggestions? ---
**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\ **Post date:** [June 8, 2020, 11:15am UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/2 "2020-06-08T11:15:32Z")
I should mention , of course this is nested fields. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 8, 2020, 2:17pm UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/3 "2020-06-08T14:17:02Z")
> [@genehunter29009](#): > > I should mention , of course this is nested fields. Well, you think of it as a nested field, but when the substitution fails what is created is a top-level field. So a prune filter with the default blacklist will remove it. ---
**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\ **Post date:** [June 8, 2020, 4:18pm UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/4 "2020-06-08T16:18:16Z")
thank you for the advice. researching now. ---
**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\ **Post date:** [June 8, 2020, 5:59pm UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/5 "2020-06-08T17:59:10Z")
So here is my issue I am putting a value in that field regardless of if its null or has a value, so the field will either have a value = 45 or "%{[userActions][cdnBusyTime]}" I have tried to only add the field if its not null but thats not easy , as you know null does not exist in elasticsearch. So the prune would have to look for this value vs null "%{[userActions][cdnBusyTime]}" my issue is of course the quotes and % , I dont know how to format that. Where would the actual other code go? 1. put here split { field =\> "[userActions]" } 2. put here mutate { add\_field =\> { "userActions.name" =\> "%{[userActions][name]}" "userActions.domain" =\> "%{[userActions][domain]}" "userActions.targetUrl" =\> "%{[userActions][targetUrl]}" "userActions.type" =\> "%{[userActions][type]}" "userActions.startTime" =\> "%{[userActions][startTime]}" "userActions.endTime" =\> "%{[userActions][endTime]}" "userActions.duration" =\> "%{[userActions][duration]}" "userActions.application" =\> "%{[userActions][application]}" "userActions.internalApplicationId" =\> "%{[userActions][internalApplicationId]}" "userActions.internalKeyUserActionId" =\> "%{[userActions][internalKeyUserActionId]}" "userActions.speedIndex" =\> "%{[userActions][speedIndex]}" "userActions.errorCount" =\> "%{[userActions][errorCount]}" "userActions.apdexCategory" =\> "%{[userActions][apdexCategory]}" "userActions.networkTime" =\> "%{[userActions][networkTime]}" "userActions.serverTime" =\> "%{[userActions][serverTime]}" "userActions.frontendTime" =\> "%{[userActions][frontendTime]}" "userActions.documentInteractiveTime" =\> "%{[userActions][documentInteractiveTime]}" "userActions.failedImages" =\> "%{[userActions][failedImages]}" "userActions.failedXhrRequests" =\> "%{[userActions][failedXhrRequests]}" "userActions.httpRequestsWithErrors" =\> "%{[userActions][httpRequestsWithErrors]}" "userActions.thirdPartyResources" =\> "%{[userActions][thirdPartyResources]}" "userActions.thirdPartyBusyTime" =\> "%{[userActions][thirdPartyBusyTime]}" "userActions.cdnResources" =\> "%{[userActions][cdnResources]}" "userActions.cdnBusyTime" =\> "%{[userActions][cdnBusyTime]}" "userActions.firstPartyBusyTime" =\> "%{[userActions][firstPartyBusyTime]}" "userActions.domCompleteTime" =\> "%{[userActions][domCompleteTime]}" "userActions.domContentLoadedTime" =\> "%{[userActions][domContentLoadedTime]}" "userActions.loadEventStart" =\> "%{[userActions][loadEventStart]}" "userActions.loadEventEnd" =\> "%{[userActions][loadEventEnd]}" "userActions.navigationStart" =\> "%{[userActions][navigationStart]}" "userActions.requestStart" =\> "%{[userActions][requestStart]}" "userActions.responseStart" =\> "%{[userActions][responseStart]}" "userActions.responseEnd" =\> "%{[userActions][responseEnd]}" "userActions.visuallyCompleteTime" =\> "%{[userActions][visuallyCompleteTime]}" "userActions.keyUserAction" =\> "%{[userActions][keyUserAction]}" } remove\_field =\> ["[userActions]" ] } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 8, 2020, 7:01pm UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/6 "2020-06-08T19:01:26Z")
A prune filter can either whitelist (only retain items in the list) or blacklist (only remove items in the list). It can do this based on either the field name or the field value. I am suggesting that you blacklist based on field values (which can be a regexp). ``` prune { blacklist_values => ["%\\{[^}]+\\}" ] } ``` That regexp is copied from the [default value](https://github.com/logstash-plugins/logstash-filter-prune/blob/9b65b28120004ae5b612487b0712846541d43674/lib/logstash/filters/prune.rb#L66) for the blacklist\_names option. It is unclear to me why the backslashes are needed (maybe to prevent the value being sprintf'd?), so you might also want to try without them. ---
**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\ **Post date:** [June 8, 2020, 11:38pm UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/7 "2020-06-08T23:38:09Z")
This is what finally worked for me. ruby { code =\> ' event.to\_hash.each { |k, v| if v.to\_s.start\_with?("%") event.remove(k) end } ' } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2020, 11:49pm UTC](https://discuss.elastic.co/t/how-to-handle-nil-or-null-values-within-mutate-add-field/236160/8 "2020-07-06T23:49:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.