# Extract specific fields from Json using Logstash

**URL:** <https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750>\
**Category:** Logstash\
**Created:** [September 9, 2019, 4:58pm UTC](https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750 "2019-09-09T16:58:26Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![baselai](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@baselai](https://discuss.elastic.co/u/baselai)\
**Post date:** [September 9, 2019, 4:58pm UTC](https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750/1 "2019-09-09T16:58:26Z")

</div>

I have this json file

```auto
{
  "type": "resource",
  "headers": {
    "destination_channel": "data"
  },
  "body": {
    "type":"resource",
    "resourceId":"estimatorB",
    "resourceType":null,
    "resourceValue":"data.csv",
    "resourceSettings":{
    },
    "resourceContext":{ 
    },
    "p_id":"123",
    "b_id":"block_789"
  }
}

```

and I need to take only three fields and push them to Elasticsearch, which they're:

> resourceValue  
> p\_id  
> b\_id

here is the code I'm using

```auto
input {
  file {
    path => "/usr/share/input/test.json"
    start_position => beginning
    sincedb_path => "/dev/null"
   
  }
}

filter {
  json {
    source => "message"    
  }
  ruby {
    code => '
        arrayOfEvents = Array.new()
        ts = event.get("[body]")
        ts.each do |k,v|
          if k == "resourceValue"
            arrayOfEvents.push(data)
          elsif k == "pipelineId"
            arrayOfEvents.push(data)
          elsif k == "blockId"
            arrayOfEvents.push(data)
          end                        
        end
        arrayOfEvents.push(data)
        event.set("event",arrayOfEvents)        
    '
  }
  split { field => "event" }
}

output {
  stdout {}
}

```

but it throws an exception **ruby split error**!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 9, 2019, 5:05pm UTC](https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750/2 "2019-09-09T17:05:55Z")

</div>

What are you using "data" to refer to in the ruby filter, and what are you trying to do with the .push immediately before the event.set?

---

<div class="post-metadata">

**Author:** ![baselai](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@baselai](https://discuss.elastic.co/u/baselai)\
**Post date:** [September 9, 2019, 5:21pm UTC](https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750/3 "2019-09-09T17:21:01Z")

</div>

@Badger I thought about it this way:  
I get the fields, or iterate through them -\> read the targeted field value and then push it to a temp array -\> return it.  
to get this output:

```auto
{
 "resourceValue":"data.csv",
 "p_id":"123",
 "b_id":"block_789"
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 9, 2019, 5:50pm UTC](https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750/4 "2019-09-09T17:50:20Z")

</div>

data is nil, so the event array is nil, which is not splittable. If you change the first three occurrences of data to v, and change blockId to b\_id, and change pipelineId to p\_id, and delete the fourth push you will get three events

{  
"event" =\> "123",  
"@timestamp" =\> 2019-09-09T17:41:31.316Z  
}  
{  
"event" =\> "data.csv",  
"@timestamp" =\> 2019-09-09T17:41:31.316Z  
}  
{  
"event" =\> "block\_789",  
"@timestamp" =\> 2019-09-09T17:41:31.316Z  
}

which I do not think is what you want. I do not think you need to use ruby at all.

```
    mutate {
        add_field => {
            "resourceValue" => "%{[body][resourceValue]}"
            "p_id" => "%{[body][p_id]}"
            "b_id" => "%{[body][b_id]}"
        }
    }
    mutate { remove_field => ["body", "headers", "message", "type"] }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 7, 2019, 5:50pm UTC](https://discuss.elastic.co/t/extract-specific-fields-from-json-using-logstash/198750/5 "2019-10-07T17:50:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.