Index for a single server

[garcia (Luis)]

I have several servers already configured on the elk server via filebeat, but for a separate server I need to configure my index in logstash

2018-11-27 10:49:12,148 - app.routing.router - INFO - name_app - [DEVELOPMENT] - ROUTING: backpage.example.com URL: / IP: IP REFERRER: http://backpage.example.com/
2018-11-27 10:49:12,148 - app.routing.router - DEBUG - name_app - [DEVELOPMENT] - TEST ROUTER: default
2018-11-27 10:49:12,157 - app.services.settings - DEBUG - name_app - [DEVELOPMENT] - domain_id 0, country_id 2, active_country {u'category': u'active_country', u'country_id': 2, u'domain_name': u'mofug.net', u'value': 2, u'country_name': u'United States', u'_id': ObjectId('5af51c27a78b94767ff6ae06'), u'domain_id': 0}
2018/11/27 10:49:12 [error] 136#136: *1405 open() "/var/www/name_app/serv/css/desktop/escortalligator/newstyle.css" failed (2: No such file or directory), client: IP, server: _default_, request: "GET /static/css/desktop/escortalligator/newstyle.css?v=1543306373.21 HTTP/1.0", host: "backpage.example.com", referrer: "http://backpage.example.com/"
MemCached: MemCache: inet:dev.apnot.com:11211: connect: No route to host.  Marking dead.
2018-11-27 10:49:28,022 - app.routing.router - INFO - name_app - [DEVELOPMENT] - ROUTING: backpage.example.com URL: / IP: IP REFERRER: None
2018-11-27 10:49:28,022 - app.routing.router - DEBUG - name_app - [DEVELOPMENT] - TEST ROUTER: default
2018-11-27 10:49:28,030 - app.services.settings - DEBUG - name_app - [DEVELOPMENT] - domain_id 0, country_id 2, active_country {u'category': u'active_country', u'country_id': 2, u'domain_name': u'mofug.net', u'value': 2, u'country_name': u'United States', u'_id': ObjectId('5af51c27a78b94767ff6ae06'), u'domain_id': 0}

for example, server logs, I need to create an index from this server in the following form: name_app.development.YYYY-MM-DD and add labels in kibana

module: app.services.settings
loglevel: DEBUG
env: DEVELOPMENT

filebeat.yml

filebeat.inputs:
- type: docker
  containers.ids:
    - "*"

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
  hosts: ["logstash_server:5044"]

logstash.conf

input {
    beats {
        port => 5044
    }
    http {
        host => "0.0.0.0"
        port => 8010
    }
}

filter {
    grok {
        match => {
            "message" => "%{LOGLEVEL:loglevel}"
        }
    }
}

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
    }
}

}

Tell me how to set it up better? do I need to put some tags in the filebeat or is it only necessary to configure logstash?

[garcia (Luis)]

tried to do like this
filebeat.yml

filebeat.inputs:
- type: docker
  containers.ids:
    - "*"
  fields:
    tags: ["dev"]

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 3

output.logstash:
  hosts: ["logstash_server:5044"]

logstash.conf

input {
    beats {
        port => 5044
    }
    http {
        host => "0.0.0.0"
        port => 8010
    }
}

filter {
    if [fields.tags] == "dev" {
      grok {
        match => {
            "message" => "%{TIMESTAMP_ISO8601:timestamp}\s(-%{DATA:module}-)\s%{LOGLEVEL:loglevel}\s(-%{DATA:dev_index}-)\s\[%{DATA:env}\]\s(-%{GREEDYDATA:message})"
        }
      }
    }
    else {
      grok {
        match => {
           "message" => "%{LOGLEVEL:loglevel}"
        }
      }
    }
}

output {
   if [fields.tags] == "dev" {
      elasticsearch {
        hosts => "elasticsearch:9200"
        index => "%{dev_index}-%{+YYYY.MM.dd}"
       }
   }
   else {
      elasticsearch {
        hosts => "elasticsearch:9200"
      }
   }
}

I did not create a new index, I see in the kiban fields.tags, but the messages are broken down into pieces

what am I doing wrong?