# Multiline filter - single threaded?

**URL:** https://discuss.elastic.co/t/multiline-filter-single-threaded/77783
**Category:** Logstash
**Created:** [March 8, 2017, 8:18am UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783 "2017-03-08T08:18:06Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)
#### Post date: [March 8, 2017, 8:18am UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783/1 "2017-03-08T08:18:06Z")

</div>

Hi All,

it looks like the multi-line filter cannot be used with more than one Logstash worker: I get the following error, when the Logstash gets started with `-w N` (N \> 1):

```
09:01:17.819 [[main]>worker0] ERROR logstash.pipeline - Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {
	"exception"=>#<NoMethodError: undefinedmethod`to_hash_with_metadata'fornil: NilClass>,
	"backtrace"=>["C:/programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-multiline-3.0.2/lib/logstash/filters/multiline.rb:273:in `merge'",
	"org/jruby/RubyArray.java:1613:in `each'",
	"org/jruby/RubyEnumerable.java:852:in `inject'",
	"C:/programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-multiline-3.0.2/lib/logstash/filters/multiline.rb:272:in `merge'",
	"C:/programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-multiline-3.0.2/lib/logstash/filters/multiline.rb:230:in `previous_filter!'",
	"C:/programs/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-multiline-3.0.2/lib/logstash/filters/multiline.rb:173:in `filter'",
	"C:/programs/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'",
	"C:/programs/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `multi_filter'",
	"org/jruby/RubyArray.java:1613:in `each'",
	"C:/programs/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'",
	"C:/programs/logstash/logstash-core/lib/logstash/filter_delegator.rb:41:in `multi_filter'",
	"(eval):303:in `filter_func'",
	"C:/programs/logstash/logstash-core/lib/logstash/pipeline.rb:378:in `filter_batch'",
	"org/jruby/RubyProc.java:281:in `call'",
	"C:/programs/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:201:in `each'",
	"org/jruby/RubyHash.java:1342:in `each'",
	"C:/programs/logstash/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:200:in `each'",
	"C:/programs/logstash/logstash-core/lib/logstash/pipeline.rb:377:in `filter_batch'",
	"C:/programs/logstash/logstash-core/lib/logstash/pipeline.rb:365:in `worker_loop'",
	"C:/programs/logstash/logstash-core/lib/logstash/pipeline.rb:331:in `start_workers'"]
}

```

the multi-line config:

```
multiline {
	pattern => "^%{TIMESTAMP_ISO8601}\s+"
	negate => true
	what => "previous"
}

```

Env: Logstash v5.2.2 on windows 10

Has anyone else experienced similar problems?  
Thank you!

---

<div class="post-metadata">

### Author: ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)
#### Post date: [March 8, 2017, 8:32am UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783/2 "2017-03-08T08:32:06Z")

</div>

just adding some more details:  
the logstash crashes not immediately, but after processing a couple of thousand log entries (~2000 - 12000...)

---

<div class="post-metadata">

### Author: ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)
#### Post date: [March 8, 2017, 10:55am UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783/3 "2017-03-08T10:55:45Z")

</div>

**Workaround** (or maybe the preferred way?):

move the multi-line processing to `codecs`, e.g.:

```
input {
  file {
	path => "C:/log/**/*.log"

	start_position => "beginning"
	sincedb_path => "NUL"
	ignore_older => 0

	codec => multiline {
	  pattern => "^%{TIMESTAMP_ISO8601}\s+"
	  negate => true
	  what => previous
	}		
}
```

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [March 8, 2017, 1:18pm UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783/4 "2017-03-08T13:18:40Z")

</div>

There is a multiline filter (deprecated, and the one you initially specified) as well as a multiline codec (the recommended component to use), which you seem to be correctly using in your last example.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 5, 2017, 1:18pm UTC](https://discuss.elastic.co/t/multiline-filter-single-threaded/77783/5 "2017-04-05T13:18:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
