# Parse JSON input into fields

**URL:** https://discuss.elastic.co/t/parse-json-input-into-fields/120236
**Category:** Logstash
**Created:** [February 16, 2018, 7:07pm UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236 "2018-02-16T19:07:52Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![exceltior](https://avatars.discourse-cdn.com/v4/letter/e/d07c76/32.png) [@exceltior](https://discuss.elastic.co/u/exceltior)
#### Post date: [February 16, 2018, 7:07pm UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236/1 "2018-02-16T19:07:52Z")

</div>

Hi,

I have been trying to turn a JSON blob keys, which I receive from input into data fields but I have been unsuccessful for some hours.

**JSON Blob fed to input**

```
{
  "timestamp": "[16/Feb/2018:19:19:03 +0000]",
  "@version": "1",
  "clientip": "127.0.0.1",
  "url": "/test",
  "code": "200",
  "method": "GET",
  "size": "12",
  "bytes_rcvd": "135",
  "bytes_sent": "395",
  "country": "test",
  "domain": "test.com",
  "customer": "john_beast",
  "user": "beast",
  "cache": "HIT",
  "ttfb": "0.002457",
  "referer": "-",
  "user-agent": "curl/7.47.0",
  "x-forwarded-for": "x.x.x.x",
  "error": "-"
}

```

**Logstash 1**

Filter

```
    filter {
        json {
            source => "message"
        }
        mutate {
            remove_field => ["message", "path", "timestamp"]
        }

        date {
            match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
        }
    }

```

Output plugin

```
  tcp {
        id => "es-output"
        host => "x.x.x.x"
        port => 80
        mode => "client"
        codec=> "json_lines"
 }

```

**Logstash 2**

```
input {
  tcp {
    mode => "server"
    port => 31311
  }
   
}
output {
    elasticsearch {
        hosts => ["http:/x.x.x.x1:9200"]
    }
}

```

**ES Output**

```
 {
   "_index": "logstash-2018.02.16",
   "_type": "doc",
   "_id": "wQcAoGEBkMC_ERzMJgWb",
   "_version": 1,
   "_score": null,
   "_source": {
     "message": "{\"country\":\"test\",\"referer\":\"NULL\",\"code\":\"200\",\"user\":\"beast\",\"ttfb\":\"0.002468\",\"error\":\"NULL\",\"clientip\":\"127.0.0.1\",\"@version\":\"1\",\"host\":\"test-server\",\"cache\":\"hit\",\"method\":\"GET\",\"x-forwarded-for\":\"x.x.x.x\",\"bytes_sent\":\"397\",\"url\":\"/test\",\"@timestamp\":\"2018-02-16T19:03:39.820Z\",\"bytes_rcvd\":\"135\",\"size\":\"12\",\"domain\":\"test.com\",\"customer\":\"john_beast\",\"user-agent\":\"curl/7.47.0\"}",
     "@version": "1",
     "host": "gcloud-123df2tr43g34g3",
     "port": 1264,
     "@timestamp": "2018-02-16T19:03:39.829Z"
   },
   "fields": {
     "@timestamp": [
       "2018-02-16T19:03:39.829Z"
     ]
   },
   "sort": [
    1518807819829
   ]
 }

```

Could you help me turn each message key into field data? I have tried many solutions from forum but no success yet.

Thanks very much.

Kind regards,  
John

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 16, 2018, 8:27pm UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236/2 "2018-02-16T20:27:25Z")

</div>

What's on the other end of the tcp output? In other words, what is Logstash sending to?

---

<div class="post-metadata">

### Author: ![exceltior](https://avatars.discourse-cdn.com/v4/letter/e/d07c76/32.png) [@exceltior](https://discuss.elastic.co/u/exceltior)
#### Post date: [February 17, 2018, 11:34am UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236/3 "2018-02-17T11:34:11Z")

</div>

I have two logstash(i edited post with other logstash config), the first one parses that configuration and uses output tcp module to send it to other logstash which is in server mode, and only then, it's sent to Elasticsearch and watched in Kibana.

**Architecture:**

`Logstash 1(parses JSON, filters and tcps output) -> Logstash 2(tcp input and outputs ES) -> Elasticsearch -> Kibana`

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 19, 2018, 7:02am UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236/4 "2018-02-19T07:02:21Z")

</div>

Make sure the codec of your tcp _input_ matches the one of your tcp _output_.

---

<div class="post-metadata">

### Author: ![exceltior](https://avatars.discourse-cdn.com/v4/letter/e/d07c76/32.png) [@exceltior](https://discuss.elastic.co/u/exceltior)
#### Post date: [February 19, 2018, 11:14am UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236/5 "2018-02-19T11:14:12Z")

</div>

Thanks very much, that worked 🤦‍♂️

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 19, 2018, 11:14am UTC](https://discuss.elastic.co/t/parse-json-input-into-fields/120236/6 "2018-03-19T11:14:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
