# Trouble replacing leading/trailing whitespace in nested json

**URL:** <https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129>\
**Category:** Logstash\
**Created:** [January 27, 2023, 5:26pm UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129 "2023-01-27T17:26:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mark54g](https://avatars.discourse-cdn.com/v4/letter/m/cc9497/32.png) [@mark54g](https://discuss.elastic.co/u/mark54g)\
**Post date:** [January 27, 2023, 5:26pm UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129/1 "2023-01-27T17:26:17Z")

</div>

Hey, folks

Trying to figure out a clean way to solve this problem

I have nested json coming in from an SQS queue, and I've been playing with mocking it up with a static file example and filebeat, which I know is not perfect, but has given me a chance to iterate cleanly without impacting the other pieces.

The json looks a bit like this:

> {"city": "My town ","comments": "all is [well","date":"2023-01-26T00:00:00.000Z","email":"myuser@example.com](mailto:well%22,%22date%22:%222023-01-26T00:00:00.000Z%22,%22email%22:%22myuser@example.com)","firstname":"User ","lastname":"Name"}

There may be syntax errors in the above, because I did mock this up by hand (but not in my actual tests).

Basically, the issue is, the upstream app breaks when it hits extra whitespace characters that lead/trail the fields.

In the example above, city and firstname have extra whitespace, but it could be a number of fields, and the example is not complete.

I would need the pay  
load to look like the above, but without those leading/trailing whitespace chars.

Trying to use mutate + strip doesn't work, because the quotes are part of the string, and need to remain that way, so it occurred to me that strip doesn't want to work there because there is no leading or trailing whitespace.

is there a way to hit defined fields and remove the whitespace around the string, but not within?

@Ugo_Sangiorgi - who had helped in slack

---

<div class="post-metadata">

**Author:** ![Ugo\_Sangiorgi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ugo_sangiorgi/32/146361_2.png) [@Ugo\_Sangiorgi](https://discuss.elastic.co/u/Ugo_Sangiorgi)\
**Post date:** [January 27, 2023, 6:03pm UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129/2 "2023-01-27T18:03:30Z")

</div>

Maybe a quick and dirty solution would be to scan the whole string for `\W"` and `"\W` and replace them by quotes with `gsub` ?

```auto
input {

    java_generator {
        lines => ['{"city": "My town ","comments": "all is well","date":"2023-01-26T00:00:00.000Z","email":"myuser@example.com","firstname":"User ","lastname":"Name"}'
        ]
        count => 1
    }
}

filter {
    mutate {
        gsub => ['message', ' "', '"']
        gsub => ['message', '" ', '"']
    }

    json {
        source => "message"
    }
}

output {
    #logshark
    elasticsearch {
        hosts => ["http://host.docker.internal:9200"]
    }
}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/e/de2a4ab751ff92c5ff6f28b0a528625ba492ddae.jpeg)

---

<div class="post-metadata">

**Author:** ![mark54g](https://avatars.discourse-cdn.com/v4/letter/m/cc9497/32.png) [@mark54g](https://discuss.elastic.co/u/mark54g)\
**Post date:** [January 27, 2023, 7:25pm UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129/3 "2023-01-27T19:25:05Z")

</div>

@Ugo_Sangiorgi That does not appear to work for us.

The output is not going to Elasticsearch, but another SQS upstream. It would have to maintain the same structure as output.

I'm still seeing the same spaces in my attempts

---

<div class="post-metadata">

**Author:** ![Ugo\_Sangiorgi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ugo_sangiorgi/32/146361_2.png) [@Ugo\_Sangiorgi](https://discuss.elastic.co/u/Ugo_Sangiorgi)\
**Post date:** [January 27, 2023, 7:30pm UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129/4 "2023-01-27T19:30:46Z")

</div>

The destination should not matter, as the substitution takes place in the raw message, no parsing. Do you have an example where it is not working?

---

<div class="post-metadata">

**Author:** ![rclarke](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rclarke/32/68604_2.png) [@rclarke](https://discuss.elastic.co/u/rclarke)\
**Post date:** [January 30, 2023, 9:33am UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129/5 "2023-01-30T09:33:59Z")

</div>

Hey @mark54g

I'm not sure why mutate-\>strip wouldn't work... Is the problem that you want to retain the JSON as a string, and not as fields in the document? If that's the case, then howsabout this:

```auto
input {
    generator {
        lines => ['{"city": "My town ","comments": "all is well","date":"2023-01-26T00:00:00.000Z","email":"myuser@example.com","firstname":"User ","lastname":"Name"}'
        ]
        count => 1
    }
}

filter {

    json {
        source => "message"
        target => "[@metadata][json]"
    }
    mutate {
        strip => ["[@metadata][json][city]", "[@metadata][json][firstname]" ]
    }
    json_encode {
        source => "[@metadata]"
        target => "message"
    }
}

output {
    stdout {
       codec => rubydebug { metadata => true }
    }
}

```

Cheers,  
-Robin-

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 27, 2023, 9:34am UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129/6 "2023-02-27T09:34:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.