Access custom fields on nginx from logstash

Elastic Stack Logstash
1

I've wrote a nginx log_format as below.

log_format main '$http_host $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for" "$http_user_browser_language" $sent_http_sv_il $sent_http_sv_cid $sent_http_sv_uid "User-Time-Zone=$http_User_Time_Zone"';

And I want to filter message for $sent_http_sv_il and $sent_http_sv_cid, $sent_http_sv_uid from Logstash for getting informations more details.

I tried to grok filter as below.

add_field => { "sv_cid" => "%{sent_http_sv_cid}" "sv_uid" => "%{sent_http_sv_uid}"}

the result of them as below

"sv_uid": "%{sent_http_sv_uid}"

what's the missing?
anyone who answer me the way?

2

You need to adjust the grok expression you use to also extract the new fields. If you show us your current grok expression we can give more specific advice.

3

I'm very appreciate for your reply.

it is our current grok expression shown below.

grok {
match => { "message" => ["%{IPORHOST:[nginx][access][remote_ip]} - %{DATA:[nginx][access][user_name]} [%{HTTPDATE:[nginx][access][time]}] "%{WORD:[nginx][access][method]} %{DATA:[nginx][access][url]} HTTP/%{NUMBER:[nginx][access][http_version]}" %{NUMBER:[nginx][access][response_code]} %{NUMBER:[nginx][access][body_sent][bytes]} "%{DATA:[nginx][access][agent]}" "%{IPORHOST:[nginx][access][remote_ip_list]}""] }
remove_field => "message"
add_field => { "sv_cid" => "%{NUMBER:[nginx][access][sent_http_sv_uid]}" "sv_uid" => "%{sent_http_sv_uid}" }
}

4

Okay, so your current grok expression only covers up to and including "$http_x_forwarded_for". After that you have "$http_user_browser_language" $sent_http_sv_il $sent_http_sv_cid $sent_http_sv_uid so you need to append something like

%{QS} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE:sv_cid} %{NOTSPACE:sv_uid}

to capture the additional fields you're interested in.

5

awesome!

it works as intended.
And I'm very appreciate for your help!!

best regards.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.