# Filebeat is not pushing nginx logs to logstash in correct format

**URL:** <https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 3, 2024, 9:43am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659 "2024-04-03T09:43:56Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sudhir\_singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhir_singh/32/104889_2.png) [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\
**Post date:** [April 3, 2024, 9:43am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659/1 "2024-04-03T09:43:56Z")

</div>

```auto
66.249.68.6, 23.59.21.50, 23.66.234.17, 104.97.76.62, 172.24.23.111, 172.24.58.27 "03/Apr/2024:03:50:44 +0530" "66.249.68.6" GET "GET /astrology/others/dream-interpretation-beginning-with-alphabet-p/articleshow/68205693.cms HTTP/1.1" 1131 200 53200 52544 - "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.6261.94 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 10.67.7.101:80 200 0.417 0.417 0.000 0.415 - vsp1toispav2.indiatimes.com vsp1toispav2.indiatimes.com vsp1toispav2.indiatimes.com "-"

```

This is my nginx logs and I'm getting below error in logstash:

```auto
[main][88aef9b53ace32249a3f3afaed43c3e0ac24d6c1df8b56b3e4c7e7942709604e] Received an event that has a different character encoding than you configured. {:text=>"hY?&ln\\x89\\x96\\xECQL\\xAEh\\x90i\\x83n \\xFB\\u0014o\\xB0a\\xEE\\x87Ҕ\\xAA\\xA1v\\xDB\\\"t]u\\xF6\\u0002\\x9B\\xFF\\xD5h|\\xF6\\xFB\\xCB'\\xD3\\xDBA\\e\\x86R.h!\\x9E\\xB41s m\\x9A\\x9C\\u0014\\xD8&m\\xB2\\u0003\\xD8\\xED\\v\\u0017Y\\xF3\\xFA\\xF49\\x9C\\xCF\\xCFl\\bo\\v\\u05EE÷I\\xA8\\\"\\x9DZ\\x91BEYN,1\\xA1\\x8A]\\xED\\x87\\xC1$і\\u0013\\u0013\\xB7g\\x92\\x8C\\xF5\\xF6Ld\\u001Fiܾ\\tCs\\xE2\\xF6\\xA6٨e2\\x8Dϔ;\\x88\\xD5PEK\\u001A\\a%\\u001E\\u0015B,\\xF3]Dϓ#\\x82I;Q#h]\\xD8|\\u001A\\x87_\\xBD\\xB93\\e\\x96O\\x83\\xB1NL?\\x88z\\x81\\u0002\\xD91\\xFBN6Т\\x92\\xB4\\x98\\xED\\xBC\\xB4_\\xEA\\vLZ\\xBFh\\xA5\\xFA4\\xFD\\x82\\u001C\\u001C\\x98/0\\xC302i\\xF65U#\\x9D\\x9DtU:\\xCCN\\xF2\\xD7kWE\\xBD\\xEC\\xA6\\xEA\\xA7:\\xCE\\xCE\\xD2a\\xFE\\xC8\\xEC\\xB5\\xCF~H\\xF1\\xFA\\x83/\\u0013u\\xE7\\xE5\\x97ou\\xEF\\u000EV×_\\xDF;y\\xAD\\u00027z\\xCE\\xF8ag\\xFF\\xF4\\xD5A\\xA73\\xEA<\\xF0\\x8F\\u0006mH\\xBF\\xEDD\\xE6\\xDE\\xDD\\xFD\\x97߶\\xBF\\xFDr\\x9A\\f\\xEE\\xEC?}{\\xA4N\\xC8\\xD3\\xE8\\xFB\\xC3\\xEF\\xF8\\xF1\\xABgo\\x8E&\\x83\\xD7n\\xEA\\x9A\\xFB߭\\u0014\\xA1\\xFA\\xBC\\xDC\\u0015\\xAF6\\x802[\\u0012q\\x94\\tC؞_\\xD5j\\u0003\\x8B\\xE6\\xE4'.\\xE6\\x94Ĝ\\u000FS\\x8Cʤ\\xF8\\xD6\\xC5:\\xF2l\\xFE\\xBF\\e\\x8D\\x9D\\xCB\\xCC\\xE6\\x9F7\\u0019\\xF5\\xC4\\xE0:Ki\\xABApL &\\u001F`j\\xD5[\\xCAs\\xAEm2\\u{3CAE8}X\\u0012AThW\\xAC.\\xCF\\xE9m\\xE0E\\x84A7V+Q\\xCE\\xD7*\\x9A\\xAA\\xF8\\xCC\\xF5r\\x8D*1~\\xA0B7\\u0019\\u0005a\\x98\\xB8\\xC90\\x88\\x8A\\xB5l\\x96F\\xBD8'\\u001E\\x91+j\\u0013BLd\\f\\x9C1*=\\xAB6!+ܬW\\x9Bn\\xC5<kVF\\x8Ab\\u0002\\xED9\\x93W\\xE32o\\xD0TlK\\xE7\\xAF\\xEF\\bXu\\xA5\\xB8\\xDC\\xE6\\xA9\\xC0\\xE2\\xFF4\\u001A;/.\\xB1\\xA9\\x98\\x87`\\u0010\\xE1\\u0000\\u0013\\f\\xA4\\u0004\\xD6\\xE7\\x94\\u0014H\\x8B,\\u0004 \\x82\\u05CC\\xA3 o\\u0011\\x87\\xE0\\xDC$]\\x83C\\xBC슖.\\\\\\xD7\\f̗GqS\\u0015\\x9B\\xB4\\xFC\\x8Db\\xC5\\u0015\\xECڹ\\u0014\\u0013\\xE5k\\xAA\\xA4\\xB1", :expected_charset=>"UTF-8"}

```

---

<div class="post-metadata">

**Author:** ![sudhir\_singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhir_singh/32/104889_2.png) [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\
**Post date:** [April 10, 2024, 7:27am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659/2 "2024-04-10T07:27:20Z")

</div>

Can anybody help on this ?

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 10, 2024, 7:56am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659/3 "2024-04-10T07:56:55Z")

</div>

> [main][88aef9b53ace32249a3f3afaed43c3e0ac24d6c1df8b56b3e4c7e7942709604e] Received an event that has a different character encoding than you configured.
> 
> :expected\_charset=\>"UTF-8"

1. Make sure that FB read and send in correct character set. Set temporary output to a file

```auto
output.file:
  path: "/path/"
  filename: output.txt

```

Check your source to see in which the encoding format they send, might be different from UTF-8. Try with

1. If you receiving from log file, set [encoding](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#_encoding_2):  
paths:  
- /path/nginx.log  
input\_type: log  
encoding: "utf-8"  
If you are receiving from syslog, tcp instead of log/filestream, you should set on the source side, or again try with encoding

2. If still doesn't help, LS input has ability to set encoding:

```auto
input {
        beats {
        port => 5044
        codec => plain { charset => "UTF-8" }
     }

```

Also check [the documentation](https://github.com/elastic/examples/blob/master/Common%20Data%20Formats/nginx_json_logs/README.md#warning-invalid-json)

---

<div class="post-metadata">

**Author:** ![sudhir\_singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sudhir_singh/32/104889_2.png) [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\
**Post date:** [April 24, 2024, 7:23am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659/4 "2024-04-24T07:23:50Z")

</div>

I have already used this logstash configuration but doesn't help

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 24, 2024, 7:28am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659/5 "2024-04-24T07:28:08Z")

</div>

Most likely you will have to set UTF-8 at a source.  
Maybe someone will a better idea.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2024, 9:28am UTC](https://discuss.elastic.co/t/filebeat-is-not-pushing-nginx-logs-to-logstash-in-correct-format/356659/6 "2024-05-22T09:28:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.