Filebeat does not send container logs to elastic cloud

[Sharadha_Krishna]

Hi,

I have set up log collection from docker containers running on EC2 machines to elastic cloud using file beat. But I don't find the logs on kibana.

Note: I have setup my own indexes and ILM and templates. I also find the logs of filebeat don't specify any issue as such. Pasting the configuration of filebeat and the indexes related snippets.

filebeat.inputs:
#------------------------------ Container input --------------------------------
- type: container
  enabled: true

  # Paths for container logs that should be crawled and fetched.
  paths:
    -/var/lib/docker/containers/*/*.log

  # Configure stream to filter to a specific stream: stdout, stderr or all (default)
  stream: all


# ================================= Processors =================================

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_host_metadata: ~
#
# =============================== Elastic Cloud ================================
cloud.id: "tazapay-dev:xxx"

cloud.auth: "elastic:xxx"

# ====================== Index Lifecycle Management (ILM) ======================

setup.ilm.enabled: false
setup.template.enabled: false
setup.template.name: "core-template"
setup.template.pattern: "core-*"

output.elasticsearch:
  indices:
    - index: "core"

TEMPLATE :

PUT _template/core-template
{
  "index_patterns": ["core-*"], 
  "settings": {
    "number_of_shards": 1,
    "index.lifecycle.name": "core", 
    "index.lifecycle.rollover_alias": "core" 
  }
}

INDEX :

PUT core-000001 
{
  "aliases": {
    "core":{
      "is_write_index": true
    }
  }
}

FILEBEAT LOGS:

pipeline/module.go:113 Beat name: ip-20-10-1-15
[monitoring] log/log.go:142 Starting metrics logging every 30s
instance/beat.go:473 filebeat start running.
memlog/store.go:119 Loading data file of '/var/lib/filebeat/registry/filebeat' succeeded. Active transaction id=0
memlog/store.go:124 Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=0
[registrar] registrar/registrar.go:109 States Loaded from registrar: 0
[crawler] beater/crawler.go:71 Loading Inputs: 1
[input] log/input.go:164 Configured paths: [/-/var/lib/docker/containers//.log] {"input_id": "08243e87-463a-4814-bed0-823e8438cfe6"}
[crawler] beater/crawler.go:141 Starting input (ID: 3876510880850742928)
[crawler] beater/crawler.go:108 Loading and starting Inputs completed. Enabled inputs: 1