Filebeat Not Pushing Python Logging Messages

[ryh]

Hi All,

Thanks in advance.

I'm having issues with filebeat not pushing my Python logging messages to Logstash (not visible in Kibana).

Here is my logging config.

def get_module_logger(mod_name):
    """
    To use this, do logger = get_module_logger(__name__)
    """
    logger = logging.getLogger(mod_name)
    handler = logging.StreamHandler()
    formatter = logging.Formatter(
        '%(asctime)s [%(name)-12s] %(levelname)-8s %(message)s')
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    logger.setLevel(logging.DEBUG)
    return logger

logger = get_module_logger(__name__)

Here's my filebeat.yml
> filebeat.inputs:

  - type: docker
    combine_partial: true
    containers:
      path: "/usr/share/dockerlogs/data"
      stream: "stdout"
      ids:
        - "*"
    exclude_files: ['\.gz$']
    ignore_older: 10m

filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

processors:
  # decode the log field (sub JSON document) if JSON encoded, then maps it's fields to elasticsearch fields
- decode_json_fields:
    fields: ["log", "message"]
    target: ""
    # overwrite existing target elasticsearch fields while decoding json fields
    overwrite_keys: true
- add_docker_metadata:
    host: "unix:///var/run/docker.sock"

output:
  logstash:
    hosts: 'logstash'

logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644
ssl.verification_mode: none

If I run a simple print statement, it does get forwarded to logstash and it's visible in Kibana. Ex. print("{\"app\": \"mimidae\", \"info\": \"Still no SpitzTlmHk\", \"test\": \"Is it working??\"}")

Thanks,
Ry