# JSON filter, query

**URL:** <https://discuss.elastic.co/t/json-filter-query/107902>\
**Category:** Logstash\
**Created:** [November 16, 2017, 10:31am UTC](https://discuss.elastic.co/t/json-filter-query/107902 "2017-11-16T10:31:24Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![runtman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/runtman/32/18136_2.png) [@runtman](https://discuss.elastic.co/u/runtman)\
**Post date:** [November 16, 2017, 10:31am UTC](https://discuss.elastic.co/t/json-filter-query/107902/1 "2017-11-16T10:31:24Z")

</div>

Hello, I am receiving alerts from SOPHOS in clear JSON, below is an example:

`{"end": "2017-11-15T21:35:18.000Z", "customer_id": "x", "endpoint_type": "computer", "dhost": "x", "type": "Event::Endpoint::WebControlViolation", "name": "'https://static.site24x7rum.com' warned due to category 'Alcohol & Tobacco'", "rt": "2017-11-15T21:35:21.942Z", "severity": "low", "duid": "5771219d2fa3360f8915b74f", "suser": "x", "group": "x", "id": "x", "whitelist_properties": {}, "endpoint_id": "x"}`

Reading up I should be able to use the JSON filter, I am using the following:

```
filter {
  if [type] == "sophos.logs" {
    json {
      source => "message"
    }
  }
}

```

However when this goes into kibana, nothing is filtered. The entire message is a single field, is this to be expected?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 16, 2017, 10:40am UTC](https://discuss.elastic.co/t/json-filter-query/107902/2 "2017-11-16T10:40:57Z")

</div>

> [@runtman](#):
>
> if [type] == "sophos.logs" {

Is this what the `type` field is set to on the events that does not get parsed correctly?

---

<div class="post-metadata">

**Author:** ![runtman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/runtman/32/18136_2.png) [@runtman](https://discuss.elastic.co/u/runtman)\
**Post date:** [November 16, 2017, 10:43am UTC](https://discuss.elastic.co/t/json-filter-query/107902/3 "2017-11-16T10:43:22Z")

</div>

Yes,

t fields.log\_type sophos.logs

Filebeat config for more clarity:

```
- input_type: log
  paths:
    - /root/repos/Sophos-Central-SIEM-Integration/log/result.txt
  fields:
    source: syslog-server
    log_type: sophos.logs
    environment: x
  tags: ['sophos-logs']
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 16, 2017, 10:45am UTC](https://discuss.elastic.co/t/json-filter-query/107902/4 "2017-11-16T10:45:17Z")

</div>

Can you show what an event looks like once it is in Kibana (please cut and paste)?

---

<div class="post-metadata">

**Author:** ![runtman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/runtman/32/18136_2.png) [@runtman](https://discuss.elastic.co/u/runtman)\
**Post date:** [November 16, 2017, 10:46am UTC](https://discuss.elastic.co/t/json-filter-query/107902/5 "2017-11-16T10:46:10Z")

</div>

![Screenshot from 2017-11-16 10-44-01](https://us1.discourse-cdn.com/elastic/original/3X/f/e/fec7aa1d649e2043e0a442090f45828f2111abeb.png)

The white parts are just text that I don't want to share 🙂

Edit:  
Is it because for raw data is being escaped (Not sure why)

```
"message": "{\"suser\": \"x", \"endpoint_type\": \"computer\", \"id\": \"x\", \"severity\": \"low\", \"dhost\": \"x\", \"customer_id\": \"x\", \"rt\": \"2017-11-16T10:30:49.666Z\", \"whitelist_properties\": {}, \"duid\": \"5x\", \"endpoint_id\": \"x\", \"name\": \"User trusted low reputation download from x\", \"group\": \"DOWNLOAD_REPUTATION\", \"end\": \"2017-11-16T10:30:49.666Z\", \"type\": \"Event::Endpoint::DownloadReputationUserAuthorised\"}",
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 16, 2017, 12:02pm UTC](https://discuss.elastic.co/t/json-filter-query/107902/6 "2017-11-16T12:02:26Z")

</div>

`type` is set to "log" in your case so your conditional is wrong. It's `[fields][log_type]` that you're setting to "sophos.logs" so you either have to change your conditional or rename the field. Perhaps you should consider setting Filebeat's `fields_under_root` option to true.

---

<div class="post-metadata">

**Author:** ![runtman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/runtman/32/18136_2.png) [@runtman](https://discuss.elastic.co/u/runtman)\
**Post date:** [November 16, 2017, 12:16pm UTC](https://discuss.elastic.co/t/json-filter-query/107902/7 "2017-11-16T12:16:28Z")

</div>

Oh my, no idea how I didn't spot that. I will re-test, thanks.

---

<div class="post-metadata">

**Author:** ![runtman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/runtman/32/18136_2.png) [@runtman](https://discuss.elastic.co/u/runtman)\
**Post date:** [November 16, 2017, 12:29pm UTC](https://discuss.elastic.co/t/json-filter-query/107902/8 "2017-11-16T12:29:46Z")

</div>

What a moron, thank you so much.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 14, 2017, 12:29pm UTC](https://discuss.elastic.co/t/json-filter-query/107902/9 "2017-12-14T12:29:59Z")

</div>

Closed on December 14, 2017, 12:29pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.