# Logstash parsing broken after upgrading from filebeat 5.6 to 7.6. Appears to be an issue with logs not getting tagged correctly

**URL:** <https://discuss.elastic.co/t/logstash-parsing-broken-after-upgrading-from-filebeat-5-6-to-7-6-appears-to-be-an-issue-with-logs-not-getting-tagged-correctly/221119>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 26, 2020, 10:38pm UTC](https://discuss.elastic.co/t/logstash-parsing-broken-after-upgrading-from-filebeat-5-6-to-7-6-appears-to-be-an-issue-with-logs-not-getting-tagged-correctly/221119 "2020-02-26T22:38:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dfinn](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@dfinn](https://discuss.elastic.co/u/dfinn)\
**Post date:** [February 26, 2020, 10:38pm UTC](https://discuss.elastic.co/t/logstash-parsing-broken-after-upgrading-from-filebeat-5-6-to-7-6-appears-to-be-an-issue-with-logs-not-getting-tagged-correctly/221119/1 "2020-02-26T22:38:35Z")

</div>

We recently upgraded our ELK stack and I realized we were still using filebeats 5.6 on our servers for log shipping. I'm testing out upgrading to 7.6 on a dev server. The upgrade itself went fine and it is still shipping logs but I noticed none of them seem to be showing up in kibana. I enabled ruby debug and the logs look totally different as they are coming in and they don't appear to be being parsed at all by logstash. Our logstash rules use the type field to match log types. For example, here is our logstash rule for logs that are tagged as syslog:

> filter {  
> if [type] == "syslog" {  
> grok {  
> break\_on\_match =\> false  
> match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
> match =\> { "syslog\_message" =\> ["^%{SPACE}%{WORD:sudo\_requested\_user}%{SPACE}:%{SPACE}TTY", "USER=%{WORD:sudo\_user}", "COMMAND=%{GREEDYDATA:sudo\_command}"] }  
> match =\> { "syslog\_message" =\> "Failed password for (invalid user |)%{USERNAME:username} from %{IP:src\_ip} port %{BASE10NUM:port} ssh2" }  
> add\_field =\> ["received\_at", "%{@timestamp}"]  
> add\_field =\> ["received\_from", "%{host}"]  
> }  
> syslog\_pri { }  
> date {  
> match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
> }  
> }  
> if [syslog\_program] == "db\_percona\_backup" {  
> grok {  
> match =\> { "syslog\_message" =\> "%{WORD:percona\_backup\_shard} %{WORD:percona\_backup\_status}" }  
> }  
> }  
> }

And here is our previous, working filebeat config:

filebeat.prospectors:

- input\_type: log  
paths:

- input\_type: log  
paths:

- input\_type: log  
paths:

- input\_type: log  
paths:

- input\_type: log  
paths:

After upgrading to filebeat 7.6 and running into some issues, this is the new filebeat config I'm currently trying out but it's not tagging the log messages correctly with the type field:

filebeat.inputs:

- type: log  
paths:

- type: log  
paths:

- type: log  
paths:

- type: log  
paths:

- type: log  
paths:

output.logstash:  
hosts: ["[ps-dev-elk.plansourcedev.com:5044](http://ps-dev-elk.plansourcedev.com:5044)"]  
ssl.certificate\_authorities: ["/etc/pki/tls/certs/filebeat.crt"]  
timeout: 30

Here's an example ruby debug log message from the old version of filebeat that is still working:

> {  
> "offset" =\> 703466488,  
> "syslog\_program" =\> "CRON",  
> "syslog\_message" =\> "pam\_unix(cron:session): session closed for user test",  
> "received\_at" =\> "2020-02-26T22:20:44.349Z",  
> "received\_from" =\> "%{host}",  
> "syslog\_severity\_code" =\> 5,  
> "source" =\> "/var/log/auth.log",  
> "message" =\> "Feb 26 22:20:35 ps-test-util01 CRON[62395]: pam\_unix(cron:session): session closed for user test",  
> "syslog\_pid" =\> "62395",  
> "@timestamp" =\> 2020-02-26T22:20:35.000Z,  
> "input\_type" =\> "log",  
> "beat" =\> {  
> "version" =\> "5.6.16",  
> "hostname" =\> "[ps-test-util01.plansourcetest.com](http://ps-test-util01.plansourcetest.com)",  
> "name" =\> "[ps-test-util01.plansourcetest.com](http://ps-test-util01.plansourcetest.com)"  
> },  
> "type" =\> "syslog",  
> "tags" =\> [  
> [0] "beats\_input\_codec\_plain\_applied"  
> ],  
> "syslog\_timestamp" =\> "Feb 26 22:20:35",  
> "syslog\_facility\_code" =\> 1,  
> "syslog\_facility" =\> "user-level",  
> "syslog\_severity" =\> "notice",  
> "@version" =\> "1",  
> "syslog\_hostname" =\> "ps-test-util01"  
> }

and here's one from filebeat 7.6 that is not working and looks completely different:

> {  
> "fields" =\> {  
> "type" =\> "syslog"  
> },  
> "log" =\> {  
> "offset" =\> 145468709,  
> "file" =\> {  
> "path" =\> "/var/log/secure"  
> }  
> },  
> "host" =\> {  
> "name" =\> "[ps-dev-web01.plansourcedev.com](http://ps-dev-web01.plansourcedev.com)"  
> },  
> "message" =\> "Jul 28 09:04:03 ps-dev-web01 sudo: pam\_unix(sudo:session): session closed for user root",  
> "agent" =\> {  
> "version" =\> "7.6.0",  
> "hostname" =\> "[ps-dev-web01.plansourcedev.com](http://ps-dev-web01.plansourcedev.com)",  
> "type" =\> "filebeat",  
> "ephemeral\_id" =\> "b29a0eaf-71de-4256-9624-268e12af6b10",  
> "id" =\> "f3149f8f-75fc-41bc-b781-6d0b4e90d319"  
> },  
> "@timestamp" =\> 2020-02-26T22:20:11.641Z,  
> "type" =\> "logs",  
> "tags" =\> [  
> [0] "syslog",  
> [1] "beats\_input\_codec\_plain\_applied"  
> ],  
> "input" =\> {  
> "type" =\> "log"  
> },  
> "ecs" =\> {  
> "version" =\> "1.4.0"  
> },  
> "@version" =\> "1"  
> }

Any suggestions on how to get things working again with 7.6?

---

<div class="post-metadata">

**Author:** ![dfinn](https://avatars.discourse-cdn.com/v4/letter/d/8c91f0/32.png) [@dfinn](https://discuss.elastic.co/u/dfinn)\
**Post date:** [February 26, 2020, 10:51pm UTC](https://discuss.elastic.co/t/logstash-parsing-broken-after-upgrading-from-filebeat-5-6-to-7-6-appears-to-be-an-issue-with-logs-not-getting-tagged-correctly/221119/2 "2020-02-26T22:51:41Z")

</div>

Also just noticed I'm getting these errors in the logstash log:

```
[2020-02-26T22:28:08,539][WARN][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2020.02.26", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x4548a9f1>], :response=>{"index"=>{"_index"=>"logstash-2020.02.26", "_type"=>"_doc", "_id"=>"vCCdg3ABQ1tpPHB5Scma", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [log] of type [text] in document with id 'vCCdg3ABQ1tpPHB5Scma'. Preview of field's value: '{file={path=/var/log/secure}, offset=213788658}'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:35"}}}}}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 25, 2020, 10:51pm UTC](https://discuss.elastic.co/t/logstash-parsing-broken-after-upgrading-from-filebeat-5-6-to-7-6-appears-to-be-an-issue-with-logs-not-getting-tagged-correctly/221119/3 "2020-03-25T22:51:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.