# Grok Expression for Nginx logs through logstash to Kibana

**URL:** https://discuss.elastic.co/t/grok-expression-for-nginx-logs-through-logstash-to-kibana/191095
**Category:** Logstash
**Created:** [July 17, 2019, 9:52pm UTC](https://discuss.elastic.co/t/grok-expression-for-nginx-logs-through-logstash-to-kibana/191095 "2019-07-17T21:52:41Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![vijayakrishna.rg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayakrishna.rg/32/36798_2.png) [@vijayakrishna.rg](https://discuss.elastic.co/u/vijayakrishna.rg)
#### Post date: [July 17, 2019, 9:52pm UTC](https://discuss.elastic.co/t/grok-expression-for-nginx-logs-through-logstash-to-kibana/191095/1 "2019-07-17T21:52:41Z")

</div>

Can someone please help me with Grok Expression to forward Nginx access logs to elasticsearch using filebeat is 1st preference, if filebeat doesn't support filter then through logstash to elasticsearch using below Grok Expression.

Nginx Log:

[2019-07-17T17:51:20+00:00] vip="10.10.10.10" http\_method="GET" uri="/errors/error.json" status\_code="400" payload\_size="0" http\_ua="Apache-HttpClient/4.5.2 (Java/1.8.0\_112)" request\_time="0.060" response\_time="-" env="tt" rid="59ce8cb4-9950" event\_action="Created" request\_uri="/event/consumer" affectedMarkets="us" version="v1" proxy\_url=""

Grok Expression:

^[%{TIMESTAMP\_ISO8601:timestamp}] %{DATA:vip} %{DATA:http\_method} %{DATA:uri} %{DATA:status\_code} %{DATA:payload\_size} %{DATA:http\_ua} %{DATA:data} %{DATA:request\_time} %{DATA:response\_time} %{DATA:env} %{DATA:rid} %{DATA:event\_action} %{DATA:request\_uri} %{DATA:affectedMarkets} %{DATA:version} %{DATA:proxy\_url}

Result:

{  
"timestamp": [  
"2019-07-17T17:51:20+00:00"  
],  
"vip": [  
"vip="10.10.10.10""  
],  
"http\_method": [  
"http\_method="GET""  
],  
"uri": [  
"uri="/errors/error.json""  
],  
"status\_code": [  
"status\_code="400""  
],  
"payload\_size": [  
"payload\_size="0""  
],  
"http\_ua": [  
"http\_ua="Apache-HttpClient/4.5.2"  
],  
"data": [  
"(Java/1.8.0\_112)""  
],  
"request\_time": [  
"request\_time="0.060""  
],  
"response\_time": [  
"response\_time="-""  
],  
"env": [  
"env="tt""  
],  
"rid": [  
"rid="59ce8cb4-9950""  
],  
"event\_action": [  
"event\_action="Created""  
],  
"request\_uri": [  
"request\_uri="/event/consumer""  
],  
"affectedMarkets": [  
"affectedMarkets="us""  
],  
"version": [  
"version="v1""  
],  
"proxy\_url": [  
""  
]  
}

log file already has key:value pairs, when i do Grok expression result has key name twice, how can i make not to having key name twice into result?, is there better way of Grok expression?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2019, 10:02pm UTC](https://discuss.elastic.co/t/grok-expression-for-nginx-logs-through-logstash-to-kibana/191095/2 "2019-07-17T22:02:19Z")

</div>

I would not use grok for that. Instead dissect out the key-value fields and then use a kv filter.

```
    dissect { mapping => { "message" => "[%{[@metadata][timestamp]}] %{[@metadata][kvData]}" } }
    kv { source => "[@metadata][kvData]" }
    date { match => ["[@metadata][timestamp]", "YYYY-MM-dd'T'HH:mm:ssZZ" ] }
```

---

<div class="post-metadata">

### Author: ![vijayakrishna.rg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayakrishna.rg/32/36798_2.png) [@vijayakrishna.rg](https://discuss.elastic.co/u/vijayakrishna.rg)
#### Post date: [July 17, 2019, 10:06pm UTC](https://discuss.elastic.co/t/grok-expression-for-nginx-logs-through-logstash-to-kibana/191095/3 "2019-07-17T22:06:59Z")

</div>

Thanks.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 14, 2019, 10:06pm UTC](https://discuss.elastic.co/t/grok-expression-for-nginx-logs-through-logstash-to-kibana/191095/4 "2019-08-14T22:06:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
