Kibana spaces and index customization

[Marta_Zagrajek]

Hi
I created few spaces in one elasticsearch.
I had one big log file and one index which differed only by field calles app_name.

So I decided to stick with one, but also shard them for different spaces. My filebeat looks that way:
filebeat.prospectors:

• type: log
  enabled: true
  paths:

  • /1/logs/1.log
    exclude_files: ['.gz$']
    fields:
    app_name: 1
    multiline.pattern: '^[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9].[0-2][0-9]:[0-5][0-9]:[0-5][0-9].[0-9][0-9][0-9]'
    multiline.negate: true
    multiline.match: after

• type: log
  enabled: true
  paths:

  • /2/logs/2.log
    exclude_files: ['.gz$']
    fields:
    app_name:2
    multiline.pattern: '^[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9].[0-2][0-9]:[0-5][0-9]:[0-5][0-9].[0-9][0-9][0-9]'
    multiline.negate: true
    multiline.match: after

• type: log
  enabled: true
  paths:

  • /3/logs/3.log
    exclude_files: ['.gz$']
    fields:
    app_name: 3
    multiline.pattern: '^[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9].[0-2][0-9]:[0-5][0-9]:[0-5][0-9].[0-9][0-9][0-9]'
    multiline.negate: true
    multiline.match: after

filebeat.config.modules:
reload.enabled: false
reload.period: 10s

setup.template.name: "microservice"
setup.template.pattern: "microservice-*"
setup.template.settings:
index.number_of_shards: 1
setup.dashboards.enabled: false

#setup.kibana:
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
hosts: ["elk-microservice:9200"]
indices:
- index: "microservice-1-%{+yyyy.MM.dd}"
when.contains:
app_name: "1"
- index: "microservice-2-%{+yyyy.MM.dd}"
when.contains:
app_name: "2"
- index: "microservice-3-%{+yyyy.MM.dd}"
when.contains:
app_name: "3"
template.name: filebeat

logging:
level: error

level: warning

level: info

to_syslog: false
to_files: true
files:
    path: "*/*/*/filebeat/logs"
    name: microservice.log
    keepfiles: 2
    permissions: 0644

It is my first try with spaces and sharding the existing index.
What did I do wrong? Because there is new index called filebeat-beat-name-date.
I did not want that to happen

[Marta_Zagrajek]

Problem was in field app_name. I did not write specifically fields.app_name and filebeat did not understand this.