Receive SIGTERM on logstash startup version 7.1.1

I am trying to use logstash to send data from kafka to s3 via logstash, and I am getting an SIGTERM in the logstash process with no apparent error messages.

I am using the following helm template override.yaml file.

# overrides stable/logstash helm templates
inputs:
  main: |-
    input {
      kafka{
        bootstrap_servers =>  "kafka.system.svc.cluster.local:9092"
        group_id => "kafka-s3"
        topics => ["device","message"]
        consumer_threads => 3
        codec => json { charset => "UTF-8" }
        decorate_events => true
      }
    }

# time_file default = 15 minutes
# size_file default = 5242880 bytes
outputs:
  main: |-
    output {
      s3 {
        codec => "json"
        prefix => "kafka/%{+YYYY}/%{+MM}/%{+dd}/%{+HH}-%{+mm}.json"
        time_file => 5
        size_file => 5242880
        region => "ap-northeast-1"
        bucket => "logging"
        canned_acl => "private"
      }
    }

podAnnotations: {
  iam.amazonaws.com/role: kafka-s3-rules
  }

image:
  tag: 7.1.1

my AWS IAM role should be attached to the container via iam2kube. The role itself allows all actions on S3.

My S3 bucket has a policy as follows:

{
    "Version": "2012-10-17",
    "Id": "LoggingBucketPolicy",
    "Statement": [
        {
            "Sid": "Stmt1554291237763",
            "Effect": "Allow",
            "Principal": {
                "AWS": "xxxxxxxxxxxxxxx"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::logging/*"
        }
    ]
}

The logs for the container are as follows.

2019/06/13 10:31:15 Setting 'path.config' from environment.
2019/06/13 10:31:15 Setting 'queue.max_bytes' from environment.
2019/06/13 10:31:15 Setting 'queue.drain' from environment.
2019/06/13 10:31:15 Setting 'http.port' from environment.
2019/06/13 10:31:15 Setting 'http.host' from environment.
2019/06/13 10:31:15 Setting 'path.data' from environment.
2019/06/13 10:31:15 Setting 'queue.checkpoint.writes' from environment.
2019/06/13 10:31:15 Setting 'queue.type' from environment.
2019/06/13 10:31:15 Setting 'config.reload.automatic' from environment.
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2019-06-13T10:31:38,061][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-06-13T10:31:38,078][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.1.1"}
[2019-06-13T10:32:02,882][WARN ][logstash.runner          ] SIGTERM received. Shutting down.

Is there anyway to get more detailed logs, or does anyone know what I am dealing with?
I greatly appriciate any help or advice! :no_mouth:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.