# Multiline read 2 times
**URL:** https://discuss.elastic.co/t/multiline-read-2-times/122684
**Category:** Beats
**Tags:** filebeat
**Created:** [March 6, 2018, 9:40am UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684 "2018-03-06T09:40:58Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@flochon](https://discuss.elastic.co/u/flochon)
#### Post date: [March 6, 2018, 9:40am UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/1 "2018-03-06T09:40:58Z")
I follow this example: [https://www.elastic.co/guide/en/beats/filebeat/master/\_examples\_of\_multiline\_configuration.html](https://www.elastic.co/guide/en/beats/filebeat/master/_examples_of_multiline_configuration.html)
I read the multiline properly but I read also the same log line by line.
My config:
```auto
- type: log
paths:
- /test/ll.log
multiline.pattern: '^[[:space:]]|^Caused by:'
multiline.negate: false
multiline.match: after
```
It's filebeat 6.2.2.
How to fix this problem ?
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [March 11, 2018, 7:44pm UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/2 "2018-03-11T19:44:14Z")
If I understand it right, you end up having the correct multline events in Elasticsearch but also 1 event for each line?
Can you share your full config and also the filebeat log?
---
### Author:  [@flochon](https://discuss.elastic.co/u/flochon)
#### Post date: [March 12, 2018, 8:09am UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/3 "2018-03-12T08:09:29Z")
Yes
filebeat.yml
```auto
#=========================== Filebeat prospectors =============================
filebeat.prospectors:
- type: log
#enabled: false
paths:
- /var/log/*.log
- /test/*.log
- type: log
paths:
- /test/ll.log
multiline.pattern: '^[[:space:]]|^Caused by:'
multiline.negate: false
multiline.match: after
# Exclude lines. A list of regular expressions to match. It drops the lines that are
# matching any regular expression from the list.
#exclude_lines: ['^DBG']
# Include lines. A list of regular expressions to match. It exports the lines that are
# matching any regular expression from the list.
#include_lines: ['^ERR', '^WARN']
# Exclude files. A list of regular expressions to match. Filebeat drops the files that
# are matching any regular expression from the list. By default, no files are dropped.
#exclude_files: ['.gz$']
# Optional additional fields. These fields can be freely picked
# to add additional information to the crawled log files for filtering
#fields:
# level: debug
# review: 1
#============================= Filebeat modules ===============================
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: false
# Period on which files under path should be checked for changes
#reload.period: 10s
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 3
#index.codec: best_compression
#_source.enabled: false
#================================ General =====================================
# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:
# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]
# Optional fields that you can specify to add additional information to the
# output.
fields:
env: bae-gold
#============================== Kibana =====================================
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
#host: "localhost:5601"
#================================ Outputs =====================================
# Configure what output to use when sending the data collected by the beat.
#----------------------------- Logstash output --------------------------------
output.logstash:
hosts: ["10.22.130.136:5443"]
# index: "filebeat"
loadbalance: true
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
#================================ Logging =====================================
# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug
# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]
```
Logstash-input.conf
```auto
input {
beats {
port => 5443
type => syslog
}
}
```
Logstash-filter.conf
```auto
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA::syslog_message}" }
add_field => ["received_at", "%{@timestamp}"]
add_field => ["received_from", "%{host}"]
}
date {
match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
}
}
}
```
Logstash-output.conf
```auto
output {
elasticsearch {
hosts => ["10.22.130.134:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
```
Elasticsearch.yml
```auto
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: exxoss-logs
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: logs-1
node.master: true
node.data: true
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 10.22.130.134
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["10.22.130.134", "10.22.130.131", "10.22.130.130"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 2
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
```
I think all my config is here.
---
### Author:  [@flochon](https://discuss.elastic.co/u/flochon)
#### Post date: [March 12, 2018, 10:05am UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/4 "2018-03-12T10:05:10Z")
Here the filbert log
```auto
Exception in thread "main" java.lang.NullPointerException
at com.example.myproject.Book.getTitle(Book.java:16)
at com.example.myproject.Author.getBookTitles(Author.java:25)
at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
```
---
### Author:  [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [March 12, 2018, 12:30pm UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/5 "2018-03-12T12:30:20Z")
The file `/test/ll.log` matches two of your prospectors as it matches also `/test/*.log`. This means the file is read twice and can lead to rather quite strange behaviour. Make sure the first prospector does not read the file.
```auto
- type: log
#enabled: false
paths:
- /var/log/*.log
- /test/*.log
- type: log
paths:
- /test/ll.log
```
---
### Author:  [@flochon](https://discuss.elastic.co/u/flochon)
#### Post date: [March 12, 2018, 12:42pm UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/6 "2018-03-12T12:42:41Z")
Oh yes I forget completely this line.
Thank you
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 9, 2018, 12:43pm UTC](https://discuss.elastic.co/t/multiline-read-2-times/122684/7 "2018-04-09T12:43:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.