# Log flooded with error messages **URL:** \ **Category:** Logstash\ **Created:** [February 25, 2020, 4:45pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897 "2020-02-25T16:45:11Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![stefan\_schumacher](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@stefan\_schumacher](https://discuss.elastic.co/u/stefan_schumacher)\ **Post date:** [February 25, 2020, 4:45pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/1 "2020-02-25T16:45:11Z")
Hi, I am trying to setup the ELK-Stack using this - [https://pawelurbanek.com/elk-nginx-logs-setup](https://pawelurbanek.com/elk-nginx-logs-setup) - Blog article here. Filebeat is running and sending data to logstash. My logstash configuration looks like this (I removed the SSL part for testing) input { beats { port =\> 5400 } } filter { grok { match =\> ["message" , "%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra\_fields}"] overwrite =\> ["message"] } mutate { convert =\> ["response", "integer"] convert =\> ["bytes", "integer"] convert =\> ["responsetime", "float"] } geoip { source =\> "clientip" target =\> "geoip" add\_tag =\> ["nginx-geoip"] } date { match =\> ["timestamp" , "dd/MMM/YYYY:HH:mm:ss Z"] remove\_field =\> ["timestamp"] } useragent { source =\> "agent" } } output { elasticsearch { hosts =\> ["localhost:9200"] index =\> "weblogs-%{+YYYY.MM.dd}" document\_type =\> "nginx\_logs" } stdout { codec =\> rubydebug } Elasticsearch is running and accepting input on port 9200. But when I try to discover the data in Kibana I dont find the configured pattern. /var/log/logstash/logstash-plain.log is full of these error which I can't interprete. [2020-02-25T17:43:05,842][ERROR][logstash.filters.useragent][main] Uknown error while parsing user agent data {:exception=\>#\, :field=\>"agent", :event=\>#LogStash::Event:0x4a5caa09} It seems no Data arrives at Kibana at all. How do i fix this? Yours faithfully Stefan Malte Schumacher ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 25, 2020, 5:21pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/2 "2020-02-25T17:21:35Z")
> [@stefan\_schumacher](#): > > cannot convert instance of class org.jruby.RubyHash to class java.lang.String\>, :field=\>"agent" The useragent filter expect to parse a string field called [agent], which I would expect to have been created by the COMBINEDAPACHELOG pattern. However, it is finding that the [agent] field is a hash, not a string. What does an event look like in the rubydebug output? ---
**Author:** ![stefan\_schumacher](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@stefan\_schumacher](https://discuss.elastic.co/u/stefan_schumacher)\ **Post date:** [February 26, 2020, 12:11pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/3 "2020-02-26T12:11:12Z")
Hello, I have started logstash manually and monitored the output. There are sections like the following: (Hostnames and IPs replaced by XXX) "log" =\> { "offset" =\> 36099, "file" =\> { "path" =\> "/var/log/nginx/access\_corporate-lounge.de.log" } }, "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "nginx-geoip" ], "clientip" =\> "89.XXX", "request" =\> "/", "message" =\> "89.XXX - - [26/Feb/2020:13:03:46 +0100] "GET / HTTP/1.1" 301 178 "-" "check\_http/v2.2 (monitoring-plugins 2.2)"", "httpversion" =\> "1.1", "ident" =\> "-", "referrer" =\> ""-"", "auth" =\> "-", "verb" =\> "GET", "@timestamp" =\> 2020-02-26T12:03:46.000Z, "response" =\> 301, "ecs" =\> { "version" =\> "1.1.0" }, "agent" =\> { "id" =\> "b79a760d-b445-430b-86eb-c27229ebea56", "ephemeral\_id" =\> "32cda409-1a33-4864-a478-9c83110f45ce", "version" =\> "7.5.2", "hostname" =\> "xxxx", "type" =\> "filebeat" }, "@version" =\> "1", "host" =\> { "containerized" =\> false, "os" =\> { "family" =\> "debian", "version" =\> "9 (stretch)", "codename" =\> "stretch", "kernel" =\> "4.9.0-8-amd64", "platform" =\> "debian", "name" =\> "Debian GNU/Linux" }, "hostname" =\> "xxxxxx", "id" =\> "522a68580a704f4b85b17ef9c7e870a7", "architecture" =\> "x86\_64", "name" =\> "xxxxx" }, "bytes" =\> 178, "geoip" =\> { "region\_code" =\> "NH", "ip" =\> "89.XXX", "timezone" =\> "Europe/Amsterdam", "country\_code2" =\> "NL", "city\_name" =\> "Schellinkhout", "latitude" =\> 52.6371, "country\_name" =\> "Netherlands", "country\_code3" =\> "NL", "postal\_code" =\> "1697", "continent\_code" =\> "EU", "region\_name" =\> "North Holland", "longitude" =\> 5.1224, "location" =\> { "lat" =\> 52.6371, "lon" =\> 5.1224 } }, "input" =\> { "type" =\> "log" interspersed with the errors seen in the logstash logs: [ERROR] 2020-02-26 13:08:45.590 [[main]\>worker1] useragent - Uknown error while parsing user agent data {:exception=\>#\, :field=\>"agent", :event=\>#LogStash::Event:0x18ece00f} [ERROR] 2020-02-26 13:07:50.580 [[main]\>worker2] useragent - Uknown error while parsing user agent data {:exception=\>#\, :field=\>"agent", :event=\>#LogStash::Event:0x596ebf58} Is this of any help in diagnosing the problem? Yours faithfully Stefan ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 26, 2020, 3:03pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/4 "2020-02-26T15:03:42Z")
> [@stefan\_schumacher](#): > > "agent" =\> { > "id" =\> "b79a760d-b445-430b-86eb-c27229ebea56", > "ephemeral\_id" =\> "32cda409-1a33-4864-a478-9c83110f45ce", > "version" =\> "7.5.2", > "hostname" =\> "xxxx", > "type" =\> "filebeat" > }, filebeat is adding this hash called agent, which conflicts with the use of COMBINEDAPACHELOG ---
**Author:** ![stefan\_schumacher](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@stefan\_schumacher](https://discuss.elastic.co/u/stefan_schumacher)\ **Post date:** [February 26, 2020, 3:24pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/5 "2020-02-26T15:24:38Z")
And how do I prevent filebeat from adding this hash? You have to remember that I am just beginning to work with ELK and that things which might be very easy to an experienced user are still difficult to me. Yours Stefan ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 26, 2020, 3:48pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/6 "2020-02-26T15:48:17Z")
I do not run filebeat so I do not know if you can prevent filebeat adding it, but you can certainly mutate+rename it to something else to eliminate the conflict. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 25, 2020, 3:48pm UTC](https://discuss.elastic.co/t/log-flooded-with-error-messages/220897/7 "2020-03-25T15:48:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.