# Filebeat Issue - all log entries are merged in a single message instead of multiple messages

**URL:** https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055
**Category:** Beats
**Tags:** filebeat
**Created:** [June 3, 2019, 8:41pm UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055 "2019-06-03T20:41:36Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![sukanta007](https://avatars.discourse-cdn.com/v4/letter/s/96bed5/32.png) [@sukanta007](https://discuss.elastic.co/u/sukanta007)
#### Post date: [June 3, 2019, 8:41pm UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055/1 "2019-06-03T20:41:36Z")

</div>

Hello team,

Hope I will get a resolution for this issue ASAP.

I am using filebeat v5.6.2 to send log file to ELK where log entries will be displayed in Kibana for visualization & analysis purpose. I have noticed that log entries are showing in Kibana as a single message string instead of separate message for each log entry.  
I could not find anything suspicious either in Filebeat or logstash which can raise this issue. I might be missing something. It was working before and this issue suddenly started showing in filebeat. No clue what happened.

I have enabled **DEBUG** in **filebeat.yml** file to check if events are published to logstash. Here is the entry -

logging.level: debug  
logging.selectors: ["\*"]  
logging.files.name: test.log

executed the below command  
./filebeat.sh -e -c /etc/filebeat/filebeat.yml -d "publish"

then, verified the log file & can see all log entries are published in single message field.  
Please help to resolve this issue. PFA for more details.

 ![filebeat_log.jpg](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb8f4b293d9e044f2c1259aed8a8c1d25bbdb4f9.jpeg)

 ![kibana_log.jpg](https://us1.discourse-cdn.com/elastic/original/3X/5/3/53eb0ab9d2a9cd207be5f6732e2408a236318801.jpeg)

**NOTE :** I have verified from logstash end to check if log entries are properly displayed in Kibana **without using Filebeat**. I specified a log file in **input** section of logstash conf file and it's working.

---

<div class="post-metadata">

### Author: ![MarianaD](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marianad/32/42687_2.png) [@MarianaD](https://discuss.elastic.co/u/MarianaD)
#### Post date: [June 4, 2019, 8:21am UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055/2 "2019-06-04T08:21:56Z")

</div>

hi @sukanta007, have you tried configuring a multiline pattern in filebeat in order to separate these multiline events ([https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html))?  
If so, and you are still unsuccessful, can you provide us with a small sample of a file processed by filebeat and the filebeat.yml file ?

---

<div class="post-metadata">

### Author: ![sukanta007](https://avatars.discourse-cdn.com/v4/letter/s/96bed5/32.png) [@sukanta007](https://discuss.elastic.co/u/sukanta007)
#### Post date: [June 4, 2019, 9:50am UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055/3 "2019-06-04T09:50:59Z")

</div>

Thanks **MarianaD** for your response.

In my case, I doubt if I need to apply multiline as each log entry is recorded in separate line instead of spreading one log entry in multiple lines. However, I applied the same as suggested, but still can see same issue. Please find the below samples & advise how to fix this issue. Appreciate your quick help here.

**Filebeat.yml**  
##################### Filebeat Configuration Example #########################

# This file is an example configuration file highlighting only the most common

# options. The filebeat.full.yml file from the same directory contains all the

# supported options with more comments. You can use it as a reference.

# 

# You can find the full configuration reference here:

# [https://www.elastic.co/guide/en/beats/filebeat/index.html](https://www.elastic.co/guide/en/beats/filebeat/index.html)

#=========================== Filebeat prospectors =============================

filebeat.prospectors:

# Each - is a prospector. Most options can be set at the prospector level, so

# you can use different prospectors for various configurations.

# Below are the prospector specific configurations.

- input\_type: log  
enabled: true

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group

# all the transactions sent by a single shipper in the web interface.

#name:

# The tags of the shipper are included in their own field with each

# transaction published.

#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the

# output.

#fields:

# env: staging

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.

# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------

# output.elasticsearch:

# Array of hosts to connect to.

# hosts: ["[sd-3fbe-d57e.nam.nsroot.net:9200](http://sd-3fbe-d57e.nam.nsroot.net:9200)"]

# Optional protocol and basic auth credentials.

#protocol: "https"  
#username: "elastic"  
#password: "changeme"

#----------------------------- Logstash output --------------------------------  
output.logstash:

# The Logstash hosts

hosts: ["[sd-3fbe-d57e.nam.nsroot.net:5044](http://sd-3fbe-d57e.nam.nsroot.net:5044)"]

# loadbalance: true

# Optional SSL. By default is off.

# List of root certificates for HTTPS server verifications

#ssl.certificate\_authorities: ["/etc/pki/root/ca.pem"]

# Certificate for SSL client authentication

#ssl.certificate: "/etc/pki/client/cert.pem"

# Client Certificate Key

#ssl.key: "/etc/pki/client/cert.key"

#================================ Logging =====================================

# Sets log level. The default log level is info.

# Available log levels are: critical, error, warning, info, debug

logging.level: debug

# At debug level, you can selectively enable logging only for some components.

# To enable all selectors use ["\*"]. Examples of other selectors are "beat",

# "publish", "service".

logging.selectors: ["\*"]

logging.to\_files: true  
logging.to\_syslog: false  
logging.files.path: /var/log/filebeat  
logging.files.name: bbdemo.log  
logging.files.keepfiles: 2  
registry\_file: /etc/filebeat/.filebeat.registry

**Sample Log File**  
127.0.0.1 | ssh | 8727584@bdhfjd | us12345 | 2019-06-04 05:00:31,211 | SSH - git-upload-pack '/sample/test2.git' | "SSH-2.0-JSCH-0.1.51" | - | - | - | - | - | 01bqdss |  
127.0.0.1 | ssh | bvy090@bdhfjd | ma12345 | 2019-06-04 05:00:31,243 | SSH - git-upload-pack '/sample/test1.git' | "SSH-2.0-JSCH-0.1.51" | 1 | 0 | 0 | ssh:user:id:007 | 32 | 02bqdss |  
127.0.0.1 | https | x@hjdf895ko | - | 2019-06-04 05:00:40,879 | "GET /status HTTP/1.0" | "" "" | - | - | - | - | - | - |  
127.0.0.1 | https | b@hjdfg89845k | - | 2019-06-04 05:00:40,880 | "GET /status HTTP/1.0" | "" "" | 200 | 0 | 19 | - | 1 | - |  
127.0.0.1 | https | myu@abh387fhg | - | 2019-05-27 19:07:31,787 | "GET /rest/api/sample/test1/pull-requests HTTP/1.1" | "[https://demo.com:8446/sample/test](https://demo.com:8446/sample/test)" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36" | - | - | - | - | - | 23hghe4 |  
127.0.0.1 | ssh | yu@1WZertk | xy94583 | 2019-05-28 19:07:31,268 | SSH - git-upload-pack '/sample/test3.git' | "SSH-2.0-JSCH-0.1.51" | - | - | - | - | - | 89p8kh |  
127.0.0.1 | https | 12k@x0 | - | 2019-05-29 19:07:16,293 | "GET /status HTTP/1.0" | "" "" | 200 | 0 | 19 | - | 1 | - |  
127.0.0.1 | ssh | k@1gue23 | ts97159 | 2019-05-30 14:09:44,885 | SSH - git-upload-pack '/sample/test4.git' | "SSH-2.0-JSCH-0.1.51" | 1 | 0 | 0 | ssh:user:id:008 | 009 | 1234mw56 |

**Kibana Output**

 ![Kibana_result](https://us1.discourse-cdn.com/elastic/original/3X/0/7/078a2daa5ec12399a89d4afee951597cb7643242.png)

---

<div class="post-metadata">

### Author: ![sukanta007](https://avatars.discourse-cdn.com/v4/letter/s/96bed5/32.png) [@sukanta007](https://discuss.elastic.co/u/sukanta007)
#### Post date: [June 4, 2019, 9:54am UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055/4 "2019-06-04T09:54:36Z")

</div>

Here is the filebeat.yml file snapshots. Please advise.

 ![yml-1](https://us1.discourse-cdn.com/elastic/original/3X/5/b/5b57402b97c353089957d95ea219af23335be5b2.png) ![yml-2](https://us1.discourse-cdn.com/elastic/original/3X/3/9/394171884bed9a25d93c0b41c0c39a52aa2e5258.png)

---

<div class="post-metadata">

### Author: ![sukanta007](https://avatars.discourse-cdn.com/v4/letter/s/96bed5/32.png) [@sukanta007](https://discuss.elastic.co/u/sukanta007)
#### Post date: [June 4, 2019, 11:39am UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055/5 "2019-06-04T11:39:35Z")

</div>

Hi team,

I have fixed this issue. This might be causing due to the space issue in **filebeat.yml** file.  
We must be very careful for the indentation of filebeat entry.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 2, 2019, 11:39am UTC](https://discuss.elastic.co/t/filebeat-issue-all-log-entries-are-merged-in-a-single-message-instead-of-multiple-messages/184055/6 "2019-07-02T11:39:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
