Hi. I am trying to store spring boot tomcat access log to logstash.
So I implement org.springframework.boot.context.embedded.EmbeddedServletContainerCustomizer like below. It works fine, I can see the access-* logs in Kibana. However It does not looks different with catalina logs.
As far as I know, if I use LogstashAccessTcpSocketAppender, I would able to get access log.
I would like to know how to set logstash to store embedded tomcat access logs.
Here is the code that I have implemented.
- At Spring boot project ...
@Configuration
public class EmbeddedTomcatConfig implements EmbeddedServletContainerCustomizer {
private Logger logger = Logger.getLogger(EmbeddedTomcatConfig.class);
private boolean useLogbackValve = true;
public void ableLogbackValve(boolean use){
this.useLogbackValve = use;
}
@Override
public void customize(ConfigurableEmbeddedServletContainer container) {
try{
if (container instanceof TomcatEmbeddedServletContainerFactory) {
TomcatEmbeddedServletContainerFactory factory = (TomcatEmbeddedServletContainerFactory) container;
if(useLogbackValve) {
LogbackValve logbackValve = new LogbackValve();
LogstashAccessTcpSocketAppender logstashAccessAppender = new LogstashAccessTcpSocketAppender();
logstashAccessAppender.addDestination("ABC.DEF.GH.IJK:5001");
LogstashAccessEncoder lae = new LogstashAccessEncoder();
lae.setWriteVersionAsString(true);
logstashAccessAppender.setEncoder(lae);
logbackValve.addAppender(logstashAccessAppender);
logbackValve.setAsyncSupported(true);
factory.addContextValves(logbackValve);
}
AccessLogValve accessLogValve = new AccessLogValve();
accessLogValve.setDirectory("/resource/ice2/tomcat/access-logs");
accessLogValve.setPattern("common");
accessLogValve.setSuffix(".log");
factory.addContextValves(accessLogValve);
// debugging
logger.info("============================================================");
logger.info("==================== Context Valves ========================");
logger.info("============================================================");
factory.getContextValves().stream().forEach(v -> {
logger.info("context valve :: " + v.getClass().getName());
});
logger.info("============================================================");
logger.info("============================================================");
logger.info("============================================================");
} else {
logger.error("WARNING! this customizer does not support your configured container");
}
} catch (Exception e) {
logger.error("Access Log Activation failed :: ", e);
}
}
}
And Here is my logstash io configurations...
input {
tcp {
port => 5001
codec => multiline {
pattern => "^{"
negate => "true"
what => "previous"
}
}
}
filter {
json {
source => "message"
}
}
output {
elasticsearch {
hosts => ['localhost:9200']
index => 'access-%{+YYYY.MM.dd}'
}
stdout { codec => rubydebug }
}
I can see the index access-2017.06.23 in elastic search, so I guess storing log works fine, but that does not looks different with catalina.(I want access logs like 0:0:0:0:0:0:0:1 - - [23/Jun/2017:11:14:53 +0900] "GET / HTTP/1.1" 200 56135)
Thanks for helping me.
FYI, below is logstash setting for catalina
In Spring, I used xml setting file like ...
<appender name="STASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>ABC.DEF.GH.IJK:5000</destination>
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
<reconnectionDelay>1 second</reconnectionDelay>
</appender>
<root level="INFO">
<appender-ref ref="STASH" />
</root>
AND logstash io like almost same with other one
input {
tcp {
port => 5000
codec => multiline {
pattern => "^{"
negate => "true"
what => "previous"
}
}
}
filter {
json {
source => "message"
}
}
output {
elasticsearch {
hosts => ['localhost:9200']
index => 'logstash-%{+YYYY.MM.dd}'
}
stdout { codec => rubydebug }
}