# Nginx json logs + filebeat + elasticsearch but unwanted key: value order
**URL:** https://discuss.elastic.co/t/nginx-json-logs-filebeat-elasticsearch-but-unwanted-key-value-order/70471
**Category:** Beats
**Tags:** filebeat
**Created:** [January 3, 2017, 7:06pm UTC](https://discuss.elastic.co/t/nginx-json-logs-filebeat-elasticsearch-but-unwanted-key-value-order/70471 "2017-01-03T19:06:08Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@cmartear](https://discuss.elastic.co/u/cmartear)
#### Post date: [January 3, 2017, 7:06pm UTC](https://discuss.elastic.co/t/nginx-json-logs-filebeat-elasticsearch-but-unwanted-key-value-order/70471/1 "2017-01-03T19:06:08Z")
Hello everyone...
I'm using json config on my nginx logs, that's parsed ans shipped by filebeat to elasticshearch and works very fine.
```
log_format json '{ "time": "$time_local", '
'"remote_ip": "$remote_addr", '
'"remote_user": "$remote_user", '
'"request": "$request", '
'"response": "$status", '
'"bytes": "$body_bytes_sent", '
'"referrer": "$http_referer", '
'"agent": "$http_user_agent" }';
```
but, on elasticsearch the resultant json isn't exactly I want, for example:
```
{
"took" : 63,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 8842259,
"max_score" : 1.0,
"hits" : [ {
"_index" : "logs",
"_type" : "log",
"_id" : "AVktD_xOzZme0fqPZkS3",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2016-12-23T19:02:14.450Z",
"beat" : {
"hostname" : "CMD-KALTURA-NGI03",
"name" : "CMD-KALTURA-NGI03",
"version" : "5.1.1"
},
"input_type" : "log",
"message" : "{ \"time\": \"23/Dec/2016:16:02:13 -0300\", \"remote_ip\": \"200.43.80.131\", \"remote_user\": \"-\", \"request\": \"GET /hls/p/116/sp/11600/serveFlavor/entryId/0_qo9vt9ie/v/2/flavorId/0_rs2urn52/name/a.mp4/seg-2-v1-a1.ts HTTP/1.1\", \"response\": \"200\", \"bytes\": \"183864\", \"referrer\": \"https://domain.com/html5/html5lib/v2.49/mwEmbedFrame.php/p/116/uiconf_id/23448615/entry_id/0_qo9vt9ie?wid=_116&iframeembed=true&playerId=kaltura_player_1482494144&entry_id=0_qo9vt9ie\", \"agent\": \"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36\" }",
"offset" : 1768277605,
"source" : "/opt/kaltura/log/json_access.log",
"type" : "log"
}
},
```
As you see, the entire line from nginx log, are included in one key "message" instead separated and own keys like:
```
"time": "23/Dec/2016:16:02:13 -0300\",
"agent":""Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36"
```
How I can get the config I want...?
I'm looking filebeat.template.json but i'm not sure if is the place I need to edit.
Regards
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [January 3, 2017, 8:14pm UTC](https://discuss.elastic.co/t/nginx-json-logs-filebeat-elasticsearch-but-unwanted-key-value-order/70471/2 "2017-01-03T20:14:38Z")
You need to configure Filebeat to decode the JSON payloads: [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-json)
---
### Author:  [@cmartear](https://discuss.elastic.co/u/cmartear)
#### Post date: [January 4, 2017, 5:35pm UTC](https://discuss.elastic.co/t/nginx-json-logs-filebeat-elasticsearch-but-unwanted-key-value-order/70471/3 "2017-01-04T17:35:52Z")
Well...
Finally I get this:
```
"_index" : "logs",
"_type" : "log",
"_id" : "AVlqg8Cy00hSCEIh5Vig",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2017-01-04T17:25:31.296Z",
"beat" : {
"hostname" : "CMD-KALTURA-NGI03",
"name" : "CMD-KALTURA-NGI03",
"version" : "5.1.1"
},
"input_type" : "log",
"json" : {
"agent" : "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36",
"bytes" : "334823",
"referrer" : "http://www.clarin.com/sociedad/ven-espacio-incendios-pampa-rio-negro_0_SyFgZ2FHg.html",
"remote_ip" : "201.216.242.69",
"request" : "GET /hds/p/102/sp/10200/serveFlavor/entryId/0_lc72fcd5/v/2/flavorId/0_,cl6h1161,f92y1jm1,35tgklp2,x05gtxp0,/forceproxy/true/name/a.mp4.urlset/frag-f1-v1-a1-Seg1-Frag39 HTTP/1.1",
"response" : "200",
"time" : "04/Jan/2017:14:25:30 -0300"
},
"offset" : 73804472,
"source" : "/opt/kaltura/log/json_access.log",
"type" : "log"
}
```
That's more usefull for me, and the only thing I need to do was this:
```
### JSON configuration
#json.message_key: log
# By default, the decoded JSON is placed under a "json" key in the output document.
# If you enable this setting, the keys are copied top level in the output document.
json.keys_under_root: false
# If keys_under_root and this setting are enabled, then the values from the decoded
# JSON object overwrite the fields that Filebeat normally adds (type, source, offset, etc.)
# in case of conflicts.
json.overwrite_keys: false
# If this setting is enabled, Filebeat adds a "json_error" key in case of JSON
# unmarshaling errors or when a text key is defined in the configuration but cannot
# be used.
json.add_error_key: false
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 1, 2017, 5:36pm UTC](https://discuss.elastic.co/t/nginx-json-logs-filebeat-elasticsearch-but-unwanted-key-value-order/70471/4 "2017-02-01T17:36:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.