# Remove\_Field json path on logstash

**URL:** <https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146>\
**Category:** Logstash\
**Created:** [May 6, 2022, 4:46pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146 "2022-05-06T16:46:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![adxalex](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adxalex/32/105346_2.png) [@adxalex](https://discuss.elastic.co/u/adxalex)\
**Post date:** [May 6, 2022, 4:46pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/1 "2022-05-06T16:46:40Z")

</div>

Hi comunity,

I'm trying to remove some fileds into a json. The json is:

```auto
{"recode":"VZ##","response-code":"4000","response":{"result":[{"DetailsPageURL":"idsource":"0""Attribute":[{"DISPLAYNAME":"Tiempo de respuesta server oasu","Value":"2305"}]}]}}

```

I just need the fields: DISPLAYNAME and Value. I tried with to diferents configuration files:

```auto
input {
  file {
      path => "/mnt/f/user/input/archivo.log"
      type => "json"
      codec=> "json"
      }
}

 filter {
      prune
      {
         whitelist_names => ["Attribute"]
      }
     }

output {
  stdout {
    codec => rubydebug
  }
  
}

```

i recieve just {} in blank

```auto
input {
  file {
      path => "/mnt/f/user/input/archivo.log"
      type => "json"
      codec=> "json"
      }
}

 filter {
      
      json {
        source => "result"
        remove_field => ["foo_%{DetailsPageURL}", "idsource"]
        
      }}
    

output {
  stdout {
    codec => rubydebug
  }
  
}

```

and this config dont delete anything.

Thanks for your helps.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 6, 2022, 5:00pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/2 "2022-05-06T17:00:16Z")

</div>

As the documentation says, prune only operates on top-level fields, so you cannot use it on fields inside [response][result].

Your message is not valid JSON. If I change it to be valid then I can do this:

```
input { generator { count => 1 lines => ['{"recode":"VZ##","response-code":"4000","response":{"result":[{"DetailsPageURL":"foo","idsource":"0","Attribute":[{"DISPLAYNAME":"Tiempo de respuesta server oasu","Value":"2305"}]}]}}' ] } }
filter {
    json { source => "message" remove_field => ["message"] }
    mutate {
        add_field => {
            "DISPLAYNAME" => "%{[response][result][0][Attribute][0][DISPLAYNAME]}"
            "Value" => "%{[response][result][0][Attribute][0][Value]}"
        }
        remove_field => ["response"]
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

```

which produces

```
"response-code" => "4000",
        "Value" => "2305",
       "recode" => "VZ##",
  "DISPLAYNAME" => "Tiempo de respuesta server oasu"

```

You may want to replace the remove\_field with a prune using whitelist\_names.

If you want to keep those two fields in place then you would need a custom ruby filter. [I do not think](https://discuss.elastic.co/t/ruby-filter-to-whitelist-nested-fields/298424/2) there is a generic solution to do it that way.

---

<div class="post-metadata">

**Author:** ![adxalex](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adxalex/32/105346_2.png) [@adxalex](https://discuss.elastic.co/u/adxalex)\
**Post date:** [May 6, 2022, 8:11pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/3 "2022-05-06T20:11:15Z")

</div>

Great @Badger , i apreciate your answer, thats works perfect.

Just one aditional questions:

I want to export tu csv:

```auto
output {
  stdout {
    codec => rubydebug
  }
  csv {
    csv_options => {"headers" => true}
    fields => ["LASTPOLLEDTIME", "[DISPLAYNAME]","[Value]"]
    fields => ["LASTPOLLEDTIME", "[DISPLAYNAME2]","[Value2]"]
    path => "/mnt/f/user/output/archivo.csv"
  }
}

```

I would like from once call from json, be able to insert the fields in diferents rows, i tried adding "fields" two times, but doesn't work he still add in the same raw.

How can insert into csv file diferentes fields in diferents rows?

Thanks

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 6, 2022, 8:35pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/4 "2022-05-06T20:35:42Z")

</div>

> [@adxalex](#):
>
> ```auto
> fields => ["LASTPOLLEDTIME", "[DISPLAYNAME]","[Value]"]
> fields => ["LASTPOLLEDTIME", "[DISPLAYNAME2]","[Value2]"]
> 
> ```

When set an option that expects a hash or array more than once logstash will merge them into a single hash or array.

A csv output will always produce one line per event. The only way I can think of to get multiple lines is something like

```
input { generator { count => 1 lines => ['{ "a": 1, "b":2, "c":3, "d":4 }'] codec => json } }
filter {}
output { stdout { codec => line { format => "%{a},%{b}
%{c},%{d}" } } }

```

which produces

```auto
1,2
3,4

```

Obviously you would use a file output, not a stdout output.

---

<div class="post-metadata">

**Author:** ![adxalex](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adxalex/32/105346_2.png) [@adxalex](https://discuss.elastic.co/u/adxalex)\
**Post date:** [May 9, 2022, 1:39pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/5 "2022-05-09T13:39:54Z")

</div>

Hi @Badger, thanks for your help,

I could take another path and this works:

```auto
output {
  stdout {
    codec => rubydebug
  }
  csv {
    csv_options => {"headers" => true}
    fields => ["LASTPOLLEDTIME", "[DISPLAYNAME]","[Value]"]
    path => "/mnt/f/user/output/archivo.csv"
  }

  csv {
    csv_options => {"headers" => true}
    
    fields => ["LASTPOLLEDTIME", "[DISPLAYNAME2]","[Value2]"]
    path => "/mnt/f/user/output/archivo.csv"
  }
  
}

```

Thanks for your help and let me know if you think that this path it's ok.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 9, 2022, 1:44pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/6 "2022-05-09T13:44:52Z")

</div>

I do not know if that will work. Whether two outputs writing to the same file will respect oneanother's positions in the file is really dependent on the operating system and filesystem. I would not expect them to maintain order, either one could write a batch of events first.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 6, 2022, 1:45pm UTC](https://discuss.elastic.co/t/remove-field-json-path-on-logstash/304146/7 "2022-06-06T13:45:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.