Incorrect multiline flag detected

Elastic Stack Beats
1

I am adding SQL Server error log files to logstash using filebeat. Some of the log entries are getting combined into a single event, even though they are on separate lines on the log file. The only difference I can see if that the concatenated logs have \r\n in the JSON, while the end of the message only has \r. I have tried using filebeat multiline settings, but what I need is really the opposite.

"message": "2019-10-28 13:56:59.17 Backup      Database backed up. Database: master, creation date(time): 2019/08/19(22:56:18), pages dumped: 467, first LSN: 354:405:73, last LSN: 354:436:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {'\\\\cdlenc1nasv1\\sqlbackup\\Backup1\\ROSQC14\\master_backup_2019_10_28_135659_0199173.bak'}). This is an informational message only. No user action is required.\r\n2019-10-28 13:56:59.18 Backup      BACKUP DATABASE successfully processed 458 pages in 0.048 seconds (74.534 MB/sec).\r\n2019-10-28 13:56:59.28 Backup      Database backed up. Database: model, creation date(time): 2003/04/08(09:13:36), pages dumped: 291, first LSN: 37:154:37, last LSN: 37:171:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {'\\\\cdlenc1nasv1\\sqlbackup\\Backup1\\ROSQC14\\model_backup_2019_10_28_135659_0199173.bak'}). This is an informational message only. No user action is required.\r\n2019-10-28 13:56:59.30 Backup      BACKUP DATABASE successfully processed 282 pages in 0.031 seconds (70.832 MB/sec).\r\n2019-10-28 13:56:59.59 Backup      Database backed up. Database: msdb, creation date(time): 2012/02/10(21:02:17), pages dumped: 7826, first LSN: 17961:16:129, last LSN: 17961:71:1, number of dump devices: 1, device information: (FILE=1, TYPE=DISK: {'\\\\cdlenc1nasv1\\sqlbackup\\Backup1\\ROSQC14\\msdb_backup_2019_10_28_135659_0199173.bak'}). This is an informational message only. No user action is required.\r\n2019-10-28 13:56:59.61 Backup      BACKUP DATABASE successfully processed 7756 pages in 0.185 seconds (327.510 MB/sec).\r",
2

I have also tried using the line_terminator setting in filebeat.yml, but it does not seem to change the issue.

3

I believe this is due to the log entries having timestamps that are either the same or just a few seconds apart. I tested this with several events, and it look like anything less than 10 seconds apart on the log gets treated as the same event. Is there a way to change a config setting to address this issue?

4

How do you want to concatenate/split your events? How did you change line_terminator settings exactly?

Filebeat is only able to split lines along one terminator. If you want to split messages both at \r\n and \n, I suggest you do the second splitting in Logstash.

5

I just want to have each log entry be a separate event. I tried all of the options for line_terminator in filebeat.yml (carriage_return, carriage_return_line_feed, etc.)

I did attempt to manipulate the message in logstash, but I don't know how to separate parts of a single message into multiple events. If there is an example on how to do that I will give it a try.

Thanks!

6

Solved the issue by replacing the newline characters with a tilde and then with a split in logstash.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.