# Use subset of json entry as document

**URL:** <https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240>\
**Category:** Logstash\
**Created:** [July 2, 2018, 3:23pm UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240 "2018-07-02T15:23:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![pkaramol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkaramol/32/22610_2.png) [@pkaramol](https://discuss.elastic.co/u/pkaramol)\
**Post date:** [July 2, 2018, 3:23pm UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240/1 "2018-07-02T15:23:37Z")

</div>

I have several `json` entries in a huge file, in the form of:

```
{"preview":false,"offset":0,"result":{"@timestamp":"2016-11-30T21:59:43.000Z","@version":"1","_raw":"..}}

```

What I want to use this file as input but keep everything under ( **not including** ) `result`, i,e I want my input to be

`{"@timestamp":"2016-11-30T21:59:43.000Z","@version":"1","_raw":"..}`

It goes without saying that after `_raw` there are many fields following...

Any suggestions?

Do I need to work on the `.json` file level with some parsing, or is there a `logstash` filter appropriate for this use case?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 2, 2018, 5:46pm UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240/2 "2018-07-02T17:46:02Z")

</div>

So basically you want to remove all fields that are not result?

Start with a json filter

```
filter { json { source => "message" } }

```

Then you can remove fields using

```
    ruby {
        code => '
            event.to_hash().each { |k, v|
                if k != "result"
                    event.remove(k)
                end
            }
        '
    }

```

and then move the fields inside result to the top level. @timestamp needs to be parsed, which I choose not to do in ruby.

```
            event.get("result").each { |k, v|
                if k == "@timestamp"
                    event.set("timestamp", v)
                else
                    event.set(k, v)
                end
            }
            event.remove("result")

```

and

```
date { match => ["timestamp", ISO8601] }
```

---

<div class="post-metadata">

**Author:** ![pkaramol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkaramol/32/22610_2.png) [@pkaramol](https://discuss.elastic.co/u/pkaramol)\
**Post date:** [July 3, 2018, 7:10am UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240/3 "2018-07-03T07:10:42Z")

</div>

The above seems to produce no events;

are we sure about this?

`filter { json { source => "message" } }`

I do not have a `message` field anywhere.

Also my input `codec` should be `plain` or `json` ?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 3, 2018, 2:25pm UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240/4 "2018-07-03T14:25:40Z")

</div>

Oh, if you used a json codec on the input then indeed you will not have a message field and do not need a json filter.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2018, 2:25pm UTC](https://discuss.elastic.co/t/use-subset-of-json-entry-as-document/138240/5 "2018-07-31T14:25:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.