# Another Filter is not work after json filter

**URL:** <https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706>\
**Category:** Logstash\
**Created:** [March 27, 2018, 7:27am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706 "2018-03-27T07:27:36Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [March 27, 2018, 7:27am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/1 "2018-03-27T07:27:37Z")

</div>

I have some log with json type. Im using filter json for my log.  
My config look like

```
filter {
  if [type] == "wifishop" {
    json {
      source => "message"
      target => "frt.wifi"
      remove_field => ["message"]
    }

    if [frt][wifi][IPWan] {

      geoip {
        id => "frt-wifi-geoip_IPWan-city"
        source => "[frt][wifi][IPWan]"
        default_database_type => "City"
        target => "[frt][wifi][geoip]"
        database => "/etc/logstash/GeoLite2-City.mmdb"
      }
      geoip {
        id => "frt-wifi-geoip_IPWan-asn"
        source => "[frt][wifi][IPWan]"
        default_database_type => "ASN"
        target => "[frt][wifi][geoip]"
        database => "/etc/logstash/GeoLite2-ASN.mmdb"
      }

      translate {
        regex => true
        dictionary_path => "/etc/logstash/translates/frt-ipshop-bo.yaml"
        field => "[frt][wifi][IPWan]"
        add_field => { "[frt][wifi][value]" => "translated" }
      }
      json {
        source => "translation"
        #remove_field => ["translation"]
      }

      mutate {
        add_field => { "[frt][wifi][checkip]" => "true" }
      }

    } else {
      mutate {
        add_field => { "[frt][wifi][checkip]" => "false" }
      }
    }
    
    mutate {
      convert => { "[frt][wifi][DownloadResult][AverageSpeed]" => "float" }
      convert => { "frt.wifi.DownloadResult.MaxSpeed" => "float" }
      convert => { "frt.wifi.DownloadResult.MinSpeed" => "float" }
      convert => { "frt.wifi.UploadResult.AverageSpeed" => "float" }
      convert => { "frt.wifi.UploadResult.MaxSpeed" => "float" }
      convert => { "frt.wifi.UploadResult.MinSpeed" => "float" }
      convert => { "frt.wifi.WifiResult.Frequency" => "float" }
      convert => { "frt.wifi.WifiResult.LinkSpeed" => "float" }
      add_field => { "frt.wifi.value" => "mutated" }
      #remove_tag => ["beats_input_codec_plain_applied","_grokparsefailure","_geoip_lookup_failure"]
    }
  }
}

```

And this is log in kibana after filtered

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/f/cf585bd7a089efe5e9d3f620fa5d6583129c2800.png)

After first json filter

```
json {
    source => "message"
    target => "frt.wifi"
    remove_field => ["message"]
}

```

i got all field with prefix "frt.wifi.\*" but when i check field **[frt][wifi][IPWan]**, why it jump to **else condition**

```
if [frt][wifi][IPWan] {
    ........
} else {
      mutate {
           add_field => { "[frt][wifi][checkip]" => "false" }
       }
 }

```

What happen? something wrong in my config ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 27, 2018, 7:33am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/2 "2018-03-27T07:33:57Z")

</div>

Have you tried

```
target => "[frt][wifi]"

```

in your json filter?

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [March 27, 2018, 7:59am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/3 "2018-03-27T07:59:25Z")

</div>

Thank for reply,

GeoIP filter worked, but translate filter not work ☹

And i have question. What is diffirent between "frt.wifi" and "[frt].[wifi]" in json filter ?.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 27, 2018, 8:10am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/4 "2018-03-27T08:10:52Z")

</div>

`frt.wifi` creates a field with that exact name:

```nohighlight
{
  ...
  "frt.wifi": "some value",
  ...
}

```

`[frt][wifi]` creates a nested field:

```nohighlight
{
  ...
  "frt": {
    "wifi": "some value"
  },
  ...
}

```

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [March 28, 2018, 1:01am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/5 "2018-03-28T01:01:02Z")

</div>

Thank for your information!

Most filters work except translate filter

```
translate {
    regex => true
    dictionary_path => "/etc/logstash/translates/frt-ipshop-bo.yaml"
    field => "[frt][wifi][IPWan]"
    fallback => "UNKNOW"
}
json {
    source => "translation"
     #remove_field => ["translation"]
}

```

My translate file look like below

```
'118.69.201.43': '{"frt": {"wifi": {"shop_id": "0", "shop_name": " Shop1"}}}'
'118.69.183.170': '{"frt": {"wifi": {"shop_id": "1", "shop_name": "Shop2"}}}'
'118.69.60.85': '{"frt": {"wifi": {"shop_id": "2", "shop_name": "Shop3"}}}'
'118.69.108.232': '{"frt": {"wifi": {"shop_id": "3", "shop_name": "Shop4"}}}'
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 28, 2018, 6:16am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/6 "2018-03-28T06:16:21Z")

</div>

Please show

- all of your pipeline configuration and
- an example document produced by Logstash (copy/paste from Kibana's JSON tab).

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [March 28, 2018, 7:00am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/7 "2018-03-28T07:00:00Z")

</div>

My pipeline configuration

```
input {
  ..........
}

filter {
  if [type] == "wifishop" {
    json {
      source => "message"
      target => "[frt][wifi]"
      remove_field => ["message"]
    }

    if [frt][wifi][IPWan] {
      geoip {
        id => "frt-wifi-geoip_IPWan-city"
        source => "[frt][wifi][IPWan]"
        default_database_type => "City"
        target => "[frt][wifi][geoip]"
        database => "/etc/logstash/GeoLite2-City.mmdb"
      }
      geoip {
        id => "frt-wifi-geoip_IPWan-asn"
        source => "[frt][wifi][IPWan]"
        default_database_type => "ASN"
        target => "[frt][wifi][geoip]"
        database => "/etc/logstash/GeoLite2-ASN.mmdb"
      }

      translate {
        regex => true
        dictionary_path => "/etc/logstash/translates/frt-ipshop-bo.yaml"
        field => "frt.wifi.IPWan"
        fallback => "UNKNOW"
      }
      json {
        source => "translation"
        #remove_field => ["translation"]
      }
    }

    mutate {
      convert => { "frt.wifi.DownloadResult.AverageSpeed" => "float" }
      convert => { "frt.wifi.DownloadResult.MaxSpeed" => "float" }
      convert => { "frt.wifi.DownloadResult.MinSpeed" => "float" }
      convert => { "frt.wifi.UploadResult.AverageSpeed" => "float" }
      convert => { "frt.wifi.UploadResult.MaxSpeed" => "float" }
      convert => { "frt.wifi.UploadResult.MinSpeed" => "float" }
      convert => { "frt.wifi.WifiResult.Frequency" => "float" }
      convert => { "frt.wifi.WifiResult.LinkSpeed" => "float" }
      #remove_tag => ["beats_input_codec_plain_applied","_grokparsefailure","_geoip_lookup_failure"]
    }
  }
}

output {
  if [type] == "wifishop" {
    elasticsearch {
      .........
    }
  }
}

```

And example document

```
{
  "_index": "fb-data-wifishop-2018.03",
  "_type": "doc",
  "_id": "LXzdamIBtNkW3GneMn8l",
  "_version": 1,
  "_score": null,
  "_source": {
    "type": "wifishop",
    "beattype": "doc",
    "offset": 4855,
    "tags": [
      "beats_input_codec_plain_applied"
    ],
    "host": "data-log",
    "@version": "1",
    "beat": {
      "version": "6.2.2",
      "hostname": "data-log",
      "name": "data-log"
    },
    "prospector": {
      "type": "log"
    },
    "frt": {
      "wifi": {
        "PingResult": {
          "AvgTime": "65.324 ms",
          "ResultString": "PING vnexpress.net (111.65.248.132) 56(84) bytes of data.\n",
          "LostPercent": "0 %",
          "Time": "28/03/2018 11:27:21",
          "Url": "vnexpress.net"
        },
        "DownloadResult": {
          "Time": "28/03/2018 11:28:13",
          "MaxSpeed": "2.595304",
          "AverageSpeed": "1.577272",
          "MinSpeed": "0.5592400000000001"
        },
        "StartDate": "28/03/2018 11:27:18",
        "IPAddress": "vnexpress.net/111.65.248.132",
        "TracerResult": {
          "ResultString": "0 Success 10.96.88.1 64.13985 ms\n1 Success localhost 54.35453 ms\n2 Success 111.65.248.132 58.3 ms\n",
          "IPUrl": "vnexpress.net/111.65.248.132",
          "NumHop": "3",
          "Time": "28/03/2018 11:27:32",
          "Result": "Fail",
          "Url": "vnexpress.net"
        },
        "IPWan": "113.161.29.217",
        "WifiResult": {
          "score": "80",
          "Time": "28/03/2018 11:27:32",
          "SSID": "\"FPTSHOP-NV\"",
          "RSSI": -38,
          "Frequency": "2462",
          "LinkSpeed": "72"
        },
        "UploadResult": {
          "Time": "28/03/2018 11:27:45",
          "MaxSpeed": "41.94304",
          "AverageSpeed": "31.141784",
          "MinSpeed": "20.340528"
        },
        "geoip": {
          "continent_code": "AS",
          "location": {
            "lat": 14.35,
            "lon": 108
          },
          "latitude": 14.35,
          "region_code": "28",
          "ip": "113.161.29.217",
          "country_code2": "VN",
          "country_name": "Vietnam",
          "timezone": "Asia/Ho_Chi_Minh",
          "longitude": 108,
          "country_code3": "VN",
          "region_name": "Kon Tum",
          "asn": 45899,
          "as_org": "VNPT Corp",
          "city_name": "Kon Tum"
        },
        "ConnectIP": {
          "Time": "28/03/2018 11:27:32",
          "Result": "Connection to https://vnexpress.netsuccess \ntime connect: 547",
          "Link": "https://vnexpress.net"
        }
      }
    },
    "@timestamp": "2018-03-28T04:28:39.477Z",
    "source": "/home/fptlogs/uploads/866045038580732_log_28_03_2018_11_27_18.txt",
    "beatname": "fb-data-wifishop"
  },
  "fields": {
    "@timestamp": [
      "2018-03-28T04:28:39.477Z"
    ]
  },
  "sort": [
    1522211319477
  ]
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 28, 2018, 7:03am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/8 "2018-03-28T07:03:30Z")

</div>

> ```
> translate {
> regex => true
> dictionary_path => "/etc/logstash/translates/frt-ipshop-bo.yaml"
> field => "frt.wifi.IPWan"
> fallback => "UNKNOW"
> }
> 
> ```

Always use the `[frt][wifi][IPWan]` notation to reference subfields. Same thing with your mutate filter near the end.

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [March 28, 2018, 7:29am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/9 "2018-03-28T07:29:42Z")

</div>

When i change config, transslate filter worked but all another filter didn't work

I got example log

```
{
  "_index": "fb-data-wifishop-2018.03",
  "_type": "doc",
  "_id": "iwaAa2IBnIT8tAud5Ksx",
  "_version": 1,
  "_score": null,
  "_source": {
    "frt": {
      "wifi": {
        "shop_name": "HNI 20 Đông Các",
        "shop_id": "30227"
      }
    },
    "beattype": "doc",
    "type": "wifishop",
    "beatname": "fb-data-wifishop",
    "prospector": {
      "type": "log"
    },
    "beat": {
      "version": "6.2.2",
      "name": "data-log",
      "hostname": "data-log"
    },
    "@version": "1",
    "host": "data-log",
    "@timestamp": "2018-03-28T07:27:19.536Z",
    "offset": 23040,
    "tags": [],
    "source": "/home/fptlogs/uploads/864212034198313_log_28_03_2018.txt",
    "message": "{\"StartDate\":\"26\\/03\\/2018 10:15:25\",\"IPWan\":\"118.70.179.236\",\"IPAddress\":\"vnexpress.net\\/111.65.248.132\",\"PingResult\":{\"Time\":\"26\\/03\\/2018 10:15:28\",\"Url\":\"vnexpress.net\",\"IPUrl\":\"vnexpress.net\\/111.65.248.132\",\"ResultString\":\"PING vnexpress.net (111.65.248.132) 56(84) bytes of data.\\n64 bytes from 111.65.248.132: icmp_seq=1 ttl=56 time=13.3 ms\\n64 bytes from 111.65.248.132: icmp_seq=2 ttl=56 time=8.73 ms\\n64 bytes from 111.65.248.132: icmp_seq=3 ttl=56 time=4.42 ms\\n64 bytes from 111.65.248.132: icmp_seq=4 ttl=56 time=25.5 ms\\n\\n--- vnexpress.net ping statistics ---\\n4 packets transmitted, 4 received, 0% packet loss, time 3016ms\\nrtt min\\/avg\\/max\\/mdev = 4.422\\/12.996\\/25.523\\/7.884 ms\\n\",\"LostPercent\":\"0 %\",\"AvgTime\":\"12.996 ms\"},\"TracerResult\":{\"Time\":\"26\\/03\\/2018 10:15:30\",\"Url\":\"vnexpress.net\",\"IPUrl\":\"vnexpress.net\\/111.65.248.132\",\"ResultString\":\"0 Success 192.168.0.1 57.007153 ms\\n1 Success 118.69.185.149 154.47354 ms\\n2 Success 118.69.189.22 169.78107 ms\\n3 Success 118.69.132.135 237.80777 ms\\n4 Success 183.80.132.62 205.93584 ms\\n5 Success 118.69.248.150 268.607 ms\\n6 Success 10.250.4.6 42.02131 ms\\n7 Success 111.65.248.132 2.21 ms\\n\",\"NumHop\":\"8\",\"Result\":\"Success\"},\"WifiResult\":{\"Time\":\"26\\/03\\/2018 10:15:30\",\"SSID\":\"\\\"TEST-WIFI\\\"\",\"RSSI\":-53,\"LinkSpeed\":\"65\",\"Frequency\":\"2412\",\"score\":\"0\"},\"ConnectIP\":{\"Time\":\"26\\/03\\/2018 10:15:31\",\"Link\":\"https:\\/\\/vnexpress.net\",\"Result\":\"Connection to https:\\/\\/vnexpress.netsuccess \\ntime connect: 91\"},\"UploadResult\":{\"Time\":\"26\\/03\\/2018 10:15:45\",\"MinSpeed\":\"11.474144\",\"MaxSpeed\":\"27.38148\",\"AverageSpeed\":\"19.427812\"},\"DownloadResult\":{\"Time\":\"26\\/03\\/2018 10:16:19\",\"MinSpeed\":\"0.5994160000000001\",\"MaxSpeed\":\"3.77496\",\"AverageSpeed\":\"2.187188\"}}"
  },
  "fields": {
    "@timestamp": [
      "2018-03-28T07:27:19.536Z"
    ]
  },
  "sort": [
    1522222039536
  ]
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 28, 2018, 7:34am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/10 "2018-03-28T07:34:29Z")

</div>

So you're saying that changing

```
field => "frt.wifi.IPWan"

```

to

```
field => "[frt][wifi][IPWan]"

```

in your translate filter suddenly caused your initial json filter to not run? Sorry, I don't buy it.

---

<div class="post-metadata">

**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\
**Post date:** [March 28, 2018, 7:57am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/11 "2018-03-28T07:57:15Z")

</div>

Yep, first json filter not work, but i dont understand that why first json didn'd work but translate filter can be work ...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 25, 2018, 7:57am UTC](https://discuss.elastic.co/t/another-filter-is-not-work-after-json-filter/125706/12 "2018-04-25T07:57:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.