How to config properly to send logs from Spring Boot to Logstash and Logstash send to Elasticsearch

[franzisk (Francisco)]

Hi there,

I have this scenario where I have a Spring Boot application using Logback appenders to send logs to Logstash:

<?xml version="1.0" encoding="UTF-8"?>

<!-- CONSOLE OUTPUT -->
<appender
	name="STDOUT"
	class="ch.qos.logback.core.ConsoleAppender">
	<layout class="ch.qos.logback.classic.PatternLayout">
		<Pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</Pattern>
	</layout>
</appender>

<!-- LOG FILES CONFIG -->
<appender
	name="FILEROLLING"
	class="ch.qos.logback.core.rolling.RollingFileAppender">
	<append>true</append>
	<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
		<fileNamePattern>/tmp/rolling-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
		<maxFileSize>1024KB</maxFileSize>
		<maxHistory>7</maxHistory>
		<totalSizeCap>10GB</totalSizeCap>
	</rollingPolicy>
	<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
		<Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</Pattern>
	</encoder>
</appender>

<!-- SUPPOSED TO INSERT LOGS ON LOGSTASH VIA TCP PORT -->
<appender
	name="STASH"
	class="net.logstash.logback.appender.LogstashTcpSocketAppender">
	<destination>${LOGSTASH_HOST}</destination>
	<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
	<keepAliveDuration>5 minutes</keepAliveDuration>
</appender>

<logger
	name="${LOGGER_NAME}"
	level="${LOGGER_LEVEL:-DEBUG}"
	additivity="false">
	<appender-ref ref="STDOUT" />
	<appender-ref ref="FILEROLLING" />
	<appender-ref ref="STASH" />
</logger>

<root level="${ROOT_LEVEL:-WARN}">
	<appender-ref ref="STDOUT" />
</root>

And I have the entire ELK stack running with docker-compose:
version: '3.7'

services:
     
  elasticsearch:
   container_name: elasticsearch
   hostname: elasticsearch
   image: docker.elastic.co/elasticsearch/elasticsearch:${VERSAO_ELK}
   ports:
     - "9200:9200"
     - "9300:9300"     
   environment:
     ES_JAVA_OPTS: "-Xmx512m -Xms512m"
     ELASTIC_PASSWORD: changeme
     discovery.type: single-node
     bootstrap.memory_lock: "true"
   ulimits:
      memlock:
        soft: -1
        hard: -1
   networks:
     - m2g_network
   volumes:
     - esdata1:/usr/share/elasticsearch/data

  kibana:
    container_name: kibana
    hostname: kibana
    image: docker.elastic.co/kibana/kibana:${VERSAO_ELK}
    ports:
      - "5601:5601"
    networks:
      - m2g_network
  
  logstash:
    hostname: logstash
    container_name: logstash
    domainname: elk-local       
    command: logstash -e 'input { tcp { port => 25827 codec => json type => json_lines } } output { elasticsearch { hosts => [ 'elasticsearch:9200' ] index => "elk-data-%{+dd.MM.YYYY}" } stdout { } }'
    image: docker.elastic.co/logstash/logstash:${VERSAO_ELK}
    restart: unless-stopped
    ports:
      - "8479:8479" 
    environment:
      LS_JAVA_OPTS: "-Xmx512m -Xms512m"
    networks:
      - m2g_network
  
volumes:
  esdata1:
    driver: local
    
networks:
  m2g_network:
    driver: overlay

First problem:
That command line seems to be ignored, if I try to connect via this 25827 port on the Spring application it says "Connection refused", but if I try via 8479 port there is no complaining about anything, so I assumed that the is OK on that port.

What's wrong here? (Maybe a bunch of things, since this is the first time I am trying to get this up and running).

Thanks in advance