# Logstash cannot parse user\_agent field of nginx **URL:** \ **Category:** Logstash\ **Created:** [July 18, 2022, 5:46pm UTC](https://discuss.elastic.co/t/logstash-cannot-parse-user-agent-field-of-nginx/309934 "2022-07-18T17:46:45Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![kosmylo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kosmylo/32/99660_2.png) [@kosmylo](https://discuss.elastic.co/u/kosmylo)\ **Post date:** [July 18, 2022, 5:46pm UTC](https://discuss.elastic.co/t/logstash-cannot-parse-user-agent-field-of-nginx/309934/1 "2022-07-18T17:46:45Z")
I have nginx logs with the following format: ```auto 192.168.0.1 - - [18/Jul/2022:11:20:28 +0000] "GET / HTTP/1.1" 200 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" "-" 192.168.128.1 - - [18/Jul/2022:13:22:15 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" "-" ``` I am using the following pipeline to parse them and store them into elasticsearch: ```auto input { beats { port => 5044 } } filter { grok { match => ["message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"] } mutate { convert => ["response", "integer"] convert => ["bytes", "integer"] convert => ["responsetime", "float"] } geoip { source => "clientip" target => "geoip" add_tag => ["nginx-geoip"] } date { match => ["timestamp" , "dd/MMM/YYYY:HH:mm:ss Z"] } useragent { source => "agent" } } output { elasticsearch { hosts => ["http://elasticsearch:9200"] index => "weblogs-%{+YYYY.MM.dd}" document_type => "nginx_logs" user => "elastic" password => "changeme" } stdout { codec => rubydebug } } ``` However, it seems that the part of useragent does not work, since I cannot see it: ```auto { "httpversion" => "1.1", "clientip" => "192.168.0.1", "ident" => "-", "timestamp" => "18/Jul/2022:11:20:28 +0000", "verb" => "GET", "@timestamp" => 2022-07-18T11:20:28.000Z, "@version" => "1", "tags" => [ [0] "beats_input_codec_plain_applied", [1] "_geoip_lookup_failure" ], "host" => { "name" => "9a852bd136fd" }, "auth" => "-", "bytes" => 15, "referrer" => "\"-\"", "geoip" => {}, "message" => "192.168.0.1 - - [18/Jul/2022:11:20:28 +0000] \"GET / HTTP/1.1\" 200 15 \"-\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36\" \"-\"", "response" => 200, "agent" => { "version" => "7.3.2", "ephemeral_id" => "0c38336d-1e30-4aaa-9ba8-20bd7bd8fb48", "type" => "filebeat", "hostname" => "9a852bd136fd", "id" => "8991142a-95df-4aed-a190-bda4649c04cd" }, "input" => { "type" => "log" }, "request" => "/", "extra_fields" => " \"-\"", "log" => { "file" => { "path" => "/var/log/nginx/access.log" }, "offset" => 11021 }, "ecs" => { "version" => "1.0.1" } } ``` What I need is to have a field including the whole `http_user_agent` content. Any idea of what is causing the error? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 18, 2022, 6:03pm UTC](https://discuss.elastic.co/t/logstash-cannot-parse-user-agent-field-of-nginx/309934/2 "2022-07-18T18:03:16Z")
> [@kosmylo](#): > > Any idea of what is causing the error? I think this is an ECS compatibility issue. When I run your pipeline I get a field called [user\_agent], not [agent]. If I add `ecs_compatibility => "disabled"` to the grok filter then I do not get the user agent in the [agent] field because filebeat has created an [agent] object with details of the beat. I also get errors ```auto :exception=>TypeError, :message=>"cannot convert instance of class org.jruby.RubyHash to class java.lang.String" ``` You can change ecs\_compatibility at the filter, pipeline, or instance level. ---
**Author:** ![kosmylo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kosmylo/32/99660_2.png) [@kosmylo](https://discuss.elastic.co/u/kosmylo)\ **Post date:** [July 18, 2022, 6:33pm UTC](https://discuss.elastic.co/t/logstash-cannot-parse-user-agent-field-of-nginx/309934/3 "2022-07-18T18:33:36Z")
First of all, I did not get if I should use` ecs_compatibility => "disabled"` and at which part? But even if I will be able to parse it, I do not want to have different fields, like the following: ```auto { "name"=>"Firefox", "version"=>"45.0", # since plugin version 3.3.0 "major"=>"45", "minor"=>"0", "os_name"=>"Mac OS X", "os_version"=>"10.11", # since plugin version 3.3.0 "os_full"=>"Mac OS X 10.11", "os_major"=>"10", "os_minor"=>"11", "device"=>"Mac" } ``` but one field to include the whole http\_user\_agent, like this one: ```auto user_agent => "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 18, 2022, 6:51pm UTC](https://discuss.elastic.co/t/logstash-cannot-parse-user-agent-field-of-nginx/309934/4 "2022-07-18T18:51:43Z")
That is what I get by default. You need to look at the setting of ecs\_compatibility at the instance, pipeline, and filter levels, and read your logstash logs which will have messages related to it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 15, 2022, 6:52pm UTC](https://discuss.elastic.co/t/logstash-cannot-parse-user-agent-field-of-nginx/309934/5 "2022-08-15T18:52:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.