# Logstash filter multiline not working
**URL:** https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885
**Category:** Logstash
**Created:** [July 5, 2017, 11:22am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885 "2017-07-05T11:22:28Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [July 5, 2017, 11:22am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/1 "2017-07-05T11:22:28Z")
I have a java log which has multiline in the log entries. So I have used multiline in filter function in logstash as below.
```
input {
file {
path => "/etc/logstash-5.2.2/sample7.log"
start_position => beginning
}
}
filter {
multiline {
negate => true
pattern => "^%{TIMESTAMP_ISO8601}"
what => "previous"
}
grok {
match => ["message", "\[%{TIMESTAMP_ISO8601:java_timestamp}\] %{LOGLEVEL:severity} +%{JAVACLASS:java_class} : %{GREEDYDATA:message}",
"message", "%{JAVASTACKTRACEPART}",
"message", "%{JAVACLASS:java_class}Exception(: %{GREEDYDATA:logmessage})?",
"message", "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log_level} +\[%{DATA:logger}\] %{GREEDYDATA:massage_data}"
]
}
}
output {
stdout { codec => rubydebug }
}
```
But above code gives following error
Sending Logstash's logs to /etc/logstash-5.2.2/logs which is now configured via log4j2.properties
[2017-07-05T16:05:43,638][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=\>"filter", :name=\>"multiline", :path=\>"logstash/filters/multiline", :error\_message=\>"NameError", :error\_class=\>NameError, :error\_backtrace=\>["/etc/logstash-5.2.2/logstash-core/lib/logstash/plugins/registry.rb:221:in `namespace_lookup'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/plugins/registry.rb:157:in`legacy\_lookup'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/plugins/registry.rb:133:in `lookup'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/plugins/registry.rb:175:in`lookup\_pipeline\_plugin'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/plugin.rb:129:in `lookup'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/pipeline.rb:95:in`plugin'", "(eval):12:in `initialize'", "org/jruby/RubyKernel.java:1079:in`eval'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/pipeline.rb:65:in `initialize'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/pipeline.rb:144:in`initialize'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/agent.rb:275:in `create_pipeline'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/agent.rb:95:in`register\_pipeline'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/runner.rb:264:in `execute'", "/etc/logstash-5.2.2/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in`run'", "/etc/logstash-5.2.2/logstash-core/lib/logstash/runner.rb:183:in `run'", "/etc/logstash-5.2.2/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in`run'", "/etc/logstash-5.2.2/lib/bootstrap/environment.rb:71:in `(root)'"]}
[2017-07-05T16:05:43,662][ERROR][logstash.agent] Cannot load an invalid configuration {:reason=\>"Couldn't find any filter plugin named 'multiline'. Are you sure this is correct? Trying to load the multiline filter plugin resulted in this error: Problems loading the requested plugin named multiline of type filter. Error: NameError NameError"}
How do I place multiline inside filter function properly, but not in input function ?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 5, 2017, 11:30am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/2 "2017-07-05T11:30:01Z")
You don't have the multiline filter plugin installed. That plugin is deprecated. Use the multiline _codec_ instead.
[https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html)
---
### Author:  [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [July 5, 2017, 11:58am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/3 "2017-07-05T11:58:18Z")
@magnusbaeck,
Thanks, after installed the plugin, filter worked fine .
And I need some advice on removing following beats code from my log
```
"@timestamp" => 2017-07-05T11:46:14.319Z,
"@version" => "1",
"beat" => {
"hostname" => "graylog-server-one",
"name" => "graylog-server-one",
"version" => "5.4.0"
},
"host" => "graylog-server-one",
"timestamp" => "2017-05-12T14:17:23.479+05:30"
```
I need to remove following block from the log. I tried with if condition, But it does't remove that part
```
"beat" => {
"hostname" => "graylog-server-one",
"name" => "graylog-server-one",
"version" => "5.4.0"
}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 5, 2017, 12:20pm UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/4 "2017-07-05T12:20:02Z")
Have you tried using a mutate filter and its `remove_field` option?
> I tried with if condition, But it does't remove that part
Please _always_ show exactly what you attempted.
---
### Author:  [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [July 6, 2017, 4:02am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/5 "2017-07-06T04:02:20Z")
@magnusbaeck
```
remove_field
```
does not removed
```
"beat" => {
"hostname" => "graylog-server-one",
"name" => "graylog-server-one",
"version" => "5.4.0"
}
```
block.
I have tried following syntax but it don't remove any fields
```
mutate
{
remove_field => { "[beat][version]" }
}
mutate
{
remove_field => ["beat", "version"]
}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 6, 2017, 8:52am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/6 "2017-07-06T08:52:58Z")
Do you try this?
```
remove_field => ["[beat][version]"]
```
---
### Author:  [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [July 6, 2017, 9:39am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/7 "2017-07-06T09:39:43Z")
I tried given code. But it doesn't remove the mentioned fields
```
mutate {
remove_field => ["[beat][version]","[beat][hostname]" ]
}
```
Below is the beat log part that I need to remove from my parse log
```
"beat" => {
"hostname" => "graylog-server-one",
"name" => "graylog-server-one",
"version" => "5.4.0"
},
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 6, 2017, 8:04pm UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/8 "2017-07-06T20:04:14Z")
Works fine for me:
```nohighlight
$ cat data
{"beat": {"hostname": "graylog-server-one", "name": "graylog-server-one", "version": "5.4.0"}}
$ cat test.config
input { stdin { codec => json } }
output { stdout { codec => rubydebug } }
filter {
mutate {
remove_field => ["[beat][version]"]
}
}
$ logstash -f test.config < data
Settings: Default pipeline workers: 8
Pipeline main started
{
"beat" => {
"hostname" => "graylog-server-one",
"name" => "graylog-server-one"
},
"@version" => "1",
"@timestamp" => "2017-07-06T20:03:48.188Z",
"host" => "bertie"
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}
```
---
### Author:  [@tharu85](https://discuss.elastic.co/u/tharu85)
#### Post date: [July 7, 2017, 3:28am UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/9 "2017-07-07T03:28:53Z")
Here is my code. But below code doesn't remove the version filed under beat
```
input {
beats {
port => 5052
}
}
filter {
multiline {
negate => true
pattern => "^%{TIMESTAMP_ISO8601}"
what => "previous"
}
if "beats_input_codec_plain_applied" in [tags] {
mutate {
remove_tag => ["beats_input_codec_plain_applied"]
}
}
grok {
match => ["message", "\[%{TIMESTAMP_ISO8601:java_timestamp}\] %{LOGLEVEL:severity} +%{JAVACLASS:java_class} : %{GREEDYDATA:message}",
"message", "%{JAVASTACKTRACEPART}",
"message", "%{JAVACLASS:java_class}Exception(: %{GREEDYDATA:logmessage})?",
"message", "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:log_level} +\[%{DATA:logger}\] %{GREEDYDATA:massage_data}"
]
}
mutate {
remove_field => ["[beat][version]" ]
}
}
output {
stdout { codec => rubydebug }
}
```
Below is the sample output, see -\> "version" =\> "5.4.0" is there in the output
```
],
"@timestamp" => 2017-07-07T03:23:04.738Z,
"file" => "NioClientBoss.java",
"@version" => "1",
"beat" => {
"hostname" => "graylog-server-one",
"name" => "graylog-server-one",
"version" => "5.4.0"
},
"host" => "graylog-server-one",
"class" => "org.jboss.netty.channel.socket.nio.NioClientBoss"
}
```
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 11, 2017, 6:46pm UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/10 "2017-07-11T18:46:58Z")
Well, my configuration works and yours doesn't. I suggest you transform your configuration to my configuration step by step until the `[beat][version]` field disappears. Start with the multiline filter.
And, I repeat, do not use the multiline filter. At all. Use Filebeat's multiline feature instead.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 8, 2017, 6:47pm UTC](https://discuss.elastic.co/t/logstash-filter-multiline-not-working/91885/11 "2017-08-08T18:47:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.