Docker.elastic.co/logstash/logstash:5.4.2 failed to start

Elastic Stack Logstash
1

Hello. When I try to run next service:

docker service create --name logstash \
  --mount "type=bind,source=$PWD/docker/logstash/logstash.conf,target=/usr/share/logstash/pipeline/logstash.conf" \
  --mount "type=bind,source=$PWD/docker/logstash/log4j2.properties,target=/usr/share/logstash/config/log4j2.properties" \
  --network elk \
  docker.elastic.co/logstash/logstash:5.4.2

I gets exception in a container log:

2017-06-23 05:02:00,426 main DEBUG LoggerContext[name=1517365b, org.apache.logging.log4j.core.LoggerContext@6df12b91] started OK with configuration org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@424b990d.
org/logstash/log/LogstashMessageFactory.java:28:in newMessage': java.lang.NoClassDefFoundError: org/logstash/log/StructuredMessage from org/apache/logging/log4j/spi/MessageFactory2Adapter.java:49:in newMessage'
from org/apache/logging/log4j/spi/AbstractLogger.java:2004:in logMessage' from org/apache/logging/log4j/spi/AbstractLogger.java:1876:in logIfEnabled'
from org/apache/logging/log4j/spi/AbstractLogger.java:1421:in info' from java/lang/reflect/Method.java:498:in invoke'
from /usr/share/logstash/logstash-core/lib/logstash/logging/logger.rb:52:in info' from /usr/share/logstash/logstash-core/lib/logstash/settings.rb:457:in value'
from org/jruby/RubyKernel.java:1858:in tap' from /usr/share/logstash/logstash-core/lib/logstash/settings.rb:453:in value'
from /usr/share/logstash/logstash-core/lib/logstash/settings.rb:216:in validate_value' from /usr/share/logstash/logstash-core/lib/logstash/settings.rb:132:in validate_all'
from org/jruby/RubyHash.java:1342:in each' from /usr/share/logstash/logstash-core/lib/logstash/settings.rb:131:in validate_all'
from /usr/share/logstash/logstash-core/lib/logstash/runner.rb:217:in execute' from /usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'
from /usr/share/logstash/logstash-core/lib/logstash/runner.rb:185:in run' from /usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'
from /usr/share/logstash/lib/bootstrap/environment.rb:71:in (root)' from usr/share/logstash/lib/bootstrap//usr/share/logstash/lib/bootstrap/environment.rb:71:in (root)'
Caused by:
URLClassLoader.java:381:in `findClass': java.lang.ClassNotFoundException: org.logstash.log.StructuredMessage

And of course docker tries to restart it again and again.
log4j2.properties file:

status = debug
name = LogstashPropertiesConfig

appender.console.type = Console
appender.console.name = plain_console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

appender.json_console.type = Console
appender.json_console.name = json_console
appender.json_console.layout.type = JSONLayout
appender.json_console.layout.compact = true
appender.json_console.layout.eventEol = true

rootLogger.level = ${sys:ls.log.level}
rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console

logstash.conf file:

input {
beats {
port => 5044
}
}

filter {
grok {
match => [ "message", "%{TIMESTAMP_ISO8601:timestamp}|%{DATA:EventID:int}|%{DATA:Logger}|%{LOGLEVEL:LogLevel}|%{GREEDYDATA:Message}" ]
tag_on_failure => [ "failedPattern1" ]
}
date {
timezone => "UTC"
match => [ "timestamp", "ISO8601" ]
remove_field => [ "timestamp" ]
}
if "beats_input_codec_plain_applied" in [tags] {
mutate {
remove_tag => ["beats_input_codec_plain_applied"]
}
mutate {
remove_field => [ "message", "host", "type", "input_type" ]
}
}
}
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
}

Remove in production

stdout {
codec => rubydebug
}
}

Update:
It only happens when I mounts log4j2.properties file to the service.

docker service create --name logstash \
  --mount "type=bind,source=$PWD/docker/logstash/logstash.conf,target=/usr/share/logstash/pipeline/logstash.conf" \
  --network elk \
  docker.elastic.co/logstash/logstash:5.4.2

log4j2.properties file I got from container using "docker cp" command and I changed in it only debug level to understand why I gets

"tags" => [
[0] "_dateparsefailure"

if incoming messages look like this:
2017-06-16 08:10:35.6889|1|Microsoft.AspNetCore.Mvc.Internal.ObjectResultExecutor|INFO|Executing ObjectResult, writing value Microsoft.AspNetCore.Mvc.ControllerContext. \n

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.