Два хоста filebeat +ELK

Добрый день.
У меня настроен ELK 7.8.0 и filebeat 7.8.0 который читает лог1 и отдает с индексом лог1 в logstash.
Собственно возникла потребность подключить еще один хост filbeat с лог2 и индексом лог2 для logstash.
На первом хосте такие настройки filebeat.yml

ilebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  paths:
    - /opt/software/logs/xxx.log
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.0.246:5044"]
  protocol: "https"
  index: "index1-%{[beat.version]}-%{+yyyy.MM.dd}"
  setup.template.name: "index1"
  setup.template.pattern: "*-index1-%{[beat.version]}-*"
  setup.template.enabled: true
  compression_level: 9

Делаю такие же настройки на втором хосте filebeat соответственно поменяв index1 на index2, но он не появляется в кибане т.е. его как бы нет.
В чем может быть проблема?

UPD.
на хосте2 стоит centos 6 а на остальных хостах centos 7.
на втором хосте файлбита, пробовал установить filebeat 6.8, результат тот же.

Лог при старте:

2020-07-02T16:25:36.432+0300    INFO    input/input.go:114      Starting input of type: log; ID: 5479216256157809241 
2020-07-02T16:25:36.432+0300    INFO    input/input.go:114      Starting input of type: log; ID: 7629700818984689685 
2020-07-02T16:25:36.432+0300    INFO    input/input.go:114      Starting input of type: log; ID: 12526209409590384984 
2020-07-02T16:25:36.432+0300    INFO    input/input.go:114      Starting input of type: log; ID: 8446139989763442651 
2020-07-02T16:25:36.434+0300    INFO    log/input.go:138        Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-07-02T16:25:36.434+0300    INFO    log/input.go:138        Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-07-02T16:25:36.434+0300    INFO    input/input.go:114      Starting input of type: log; ID: 12421169689888292265 
2020-07-02T16:25:36.435+0300    INFO    input/input.go:114      Starting input of type: log; ID: 10618728313673870856 
2020-07-02T16:25:36.435+0300    INFO    cfgfile/reload.go:205   Loading of config files completed.
2020-07-02T16:25:39.390+0300    INFO    add_cloud_metadata/add_cloud_metadata.go:346    add_cloud_metadata: hosting provider type not detected.
2020-07-02T16:26:06.398+0300    INFO    [monitoring]    log/log.go:144  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":23}},"total":{"ticks":80,"time":{"ms":85},"value":80},"user":{"ticks":60,"time":{"ms":62}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":4},"info":{"ephemeral_id":"22cc81f0-8be7-4669-a132-4161ff862795","uptime":{"ms":30025}},"memstats":{"gc_next":4194304,"memory_alloc":2013560,"memory_total":8634408,"rss":23093248}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":15,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":4},"load":{"1":0.04,"15":0,"5":0.05,"norm":{"1":0.01,"15":0,"5":0.0125}}}}}}

В начале f не хватает и пробелов перед - type: log

Странно, надругих хостах у меня все работает именно в формате:

#=========================== Filebeat inputs =============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
   - /varl/log/messages

После того как привожу конфиг к вашему формату:

#=========================== Filebeat inputs =============================

  filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

   - type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
     - /varl/log/messages

Получаю ошибку

[root@app filebeat]# /etc/init.d/filebeat start
Starting filebeat: Exiting: error loading config file: yaml: line 68: did not find expected <document start>

Видимо съехали поля при редактировании, привожу конфиг к нормальному виду и все запускается

root@app filebeat]# /etc/init.d/filebeat start
Starting filebeat: 2020-07-03T10:00:17.627+0300 INFO    instance/beat.go:571    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2020-07-03T10:00:17.627+0300    INFO    instance/beat.go:579    Beat ID: ee7adef5-45a4-4baa-ab06-966a2db6c22c
2020-07-03T10:00:17.627+0300    INFO    [index-management.ilm]  ilm/ilm.go:129  Policy name: filebeat-7.1.0
2020-07-03T10:00:17.630+0300    INFO    [beat]  instance/beat.go:827    Beat info       {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "ee7adef5-45a4-4baa-ab06-966a2db6c22c"}}}
2020-07-03T10:00:17.630+0300    INFO    [beat]  instance/beat.go:836    Build info      {"system_info": {"build": {"commit": "03b3db2a1d9d76fdf10475e829fce436c61901e4", "libbeat": "7.1.0", "time": "2019-05-15T23:59:19.000Z", "version": "7.1.0"}}}
2020-07-03T10:00:17.630+0300    INFO    [beat]  instance/beat.go:839    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.11.5"}}}
2020-07-03T10:00:17.632+0300    INFO    [beat]  instance/beat.go:843    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-05-21T14:40:54+03:00","containerized":true,"name":"app.dev","ip":["127.0.0.1/8","::1/128","192.168.0.206/24","fe80::20c:29ff:fe95:eb75/64"],"kernel_version":"2.6.32-754.15.3.el6.x86_64","mac":["00:0c:29:95:eb:75"],"os":{"family":"redhat","platform":"centos","name":"CentOS","version":"6.10 (Final)","major":6,"minor":10,"patch":0,"codename":"Final"},"timezone":"MSK","timezone_offset_sec":10800,"id":"628a515858a7dc0c40847abf00000008"}}}
2020-07-03T10:00:17.632+0300    INFO    [beat]  instance/beat.go:872    Process info    {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"ambient":null}, "cwd": "/etc/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 53464, "ppid": 53463, "seccomp": {"mode":""}, "start_time": "2020-07-03T10:00:16.630+0300"}}}
2020-07-03T10:00:17.632+0300    INFO    instance/beat.go:280    Setup Beat: filebeat; Version: 7.1.0
2020-07-03T10:00:17.633+0300    INFO    [publisher]     pipeline/module.go:97   Beat name: app.dev
Config OK
                                                           [  OK  ]

Вот секция output:

#----------------------------- Logstash output --------------------------------
  output.logstash:
  # The Logstash hosts
   hosts: ["192.168.0.246:5044"]
   protocol: "https"
   index: "messages3-%{[beat.version]}-%{+yyyy.MM.dd}"
   setup.template.name: "messages3"
   setup.template.pattern: "*-messages3-%{[beat.version]}-*"
   setup.template.enabled: true
   compression_level: 9

Но в кибане я его так и не вижу(((
Единственное отличие этого хоста от других, в центоси 6.10, на других 7-ка и версии filebeat 7.1.0 после того как не взлетело на 7.8.0, хотя и та и другая версия совместимы по матрице со стеком ELK версии 7.8.0 который я развернул.
В чем проблема непонятно.

Далее видим вот что в логе:

2020-07-03T10:16:22.740+0300    INFO    [monitoring]    log/log.go:153  Uptime: 38.085991ms
2020-07-03T10:16:22.740+0300    INFO    [monitoring]    log/log.go:130  Stopping metrics logging.
2020-07-03T10:16:22.740+0300    INFO    instance/beat.go:401    filebeat stopped.
2020-07-03T10:16:22.741+0300    ERROR   instance/beat.go:802    Exiting: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')
Exiting: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')

А там то уже все ок поидее:

#=========================== Filebeat inputs =============================

  filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

   - type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /varl/log/messages

Покопался в гуглах:
Привел к виду:

#=========================== Filebeat inputs =============================

  filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

   - type: log

  # Change to true to enable this input configuration.
     enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
     paths:
       - /varl/log/messages

В логах

2020-07-03T10:50:38.797+0300    INFO    instance/beat.go:280    Setup Beat: filebeat; Version: 7.1.0
2020-07-03T10:50:38.798+0300    INFO    [publisher]     pipeline/module.go:97   Beat name: app.dev
Config OK
                                                           [  OK  ]
[root@app filebeat]# tail -f /var/log/filebeat/filebeat
2020-07-03T10:50:38.867+0300    INFO    input/input.go:114      Starting input of type: log; ID: 5479216256157809241 
2020-07-03T10:50:38.867+0300    INFO    input/input.go:114      Starting input of type: log; ID: 7629700818984689685 
2020-07-03T10:50:38.867+0300    INFO    input/input.go:114      Starting input of type: log; ID: 12526209409590384984 
2020-07-03T10:50:38.867+0300    INFO    input/input.go:114      Starting input of type: log; ID: 8446139989763442651 
2020-07-03T10:50:38.869+0300    INFO    log/input.go:138        Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-07-03T10:50:38.869+0300    INFO    log/input.go:138        Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-07-03T10:50:38.869+0300    INFO    input/input.go:114      Starting input of type: log; ID: 12421169689888292265 
2020-07-03T10:50:38.869+0300    INFO    input/input.go:114      Starting input of type: log; ID: 10618728313673870856 
2020-07-03T10:50:38.869+0300    INFO    cfgfile/reload.go:205   Loading of config files completed.
2020-07-03T10:50:41.844+0300    INFO    add_cloud_metadata/add_cloud_metadata.go:346    add_cloud_metadata: hosting provider type not detected.
2020-07-03T10:51:08.851+0300    INFO    [monitoring]    log/log.go:144  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":53}},"total":{"ticks":110,"time":{"ms":119},"value":0},"user":{"ticks":60,"time":{"ms":66}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":4},"info":{"ephemeral_id":"ddee7a9d-76c1-4918-8458-1f0522b4de07","uptime":{"ms":30036}},"memstats":{"gc_next":4204096,"memory_alloc":2156176,"memory_total":8808296,"rss":29483008}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":15,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":4},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
2020-07-03T10:51:38.849+0300    INFO    [monitoring]    log/log.go:144  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":1}},"total":{"ticks":120,"time":{"ms":10},"value":120},"user":{"ticks":70,"time":{"ms":9}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":4},"info":{"ephemeral_id":"ddee7a9d-76c1-4918-8458-1f0522b4de07","uptime":{"ms":60034}},"memstats":{"gc_next":4204096,"memory_alloc":2285976,"memory_total":8954032}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":15,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}

В кибане по прежнему тишина.

В логах logstash

Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: -12

Но при этом остальные хосты нормально пишут логи в елку.

Поправил output закаментил protocol b и ошибка ушла.

#----------------------------- Logstash output --------------------------------
  output.logstash:
  # The Logstash hosts
   hosts: ["192.168.0.246:5044"]
   #protocol: "https"
   index: "pfc.qa2-%{[beat.version]}-%{+yyyy.MM.dd}"
   setup.template.name: "pfc.qa2"
   setup.template.pattern: "*-pfc.qa2-%{[beat.version]}-*"
   setup.template.enabled: true
   compression_level: 9

Лог логстеша:

root@localhost logstash]# tail -f /var/log/logstash/logstash-plain.log 
[2020-07-03T04:09:47,333][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2020-07-03T04:09:47,509][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//192.168.0.246:9200"]}
[2020-07-03T04:09:47,658][INFO ][logstash.outputs.elasticsearch][main] Using default mapping template
[2020-07-03T04:09:48,012][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-07-03T04:09:48,217][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/filter.conf", "/etc/logstash/conf.d/input.conf", "/etc/logstash/conf.d/output.conf"], :thread=>"#<Thread:0x6708f832 run>"}
[2020-07-03T04:09:49,715][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-07-03T04:09:49,745][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-07-03T04:09:49,897][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-07-03T04:09:49,919][INFO ][org.logstash.beats.Server][main][d752886843c30471c83f7a7dca39ebf6f3c3501e8d768c280e6fa83dc24d7773] Starting server on port: 5044
[2020-07-03T04:09:50,279][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

В кибане ноыого индекса не появилось(
Может дело в том что я это уже третий сервер с filebeat?

Снес все к фигам и установил снова filebeat 7.8.0.

020-07-03T12:19:05.505+0300    INFO    cfgfile/reload.go:164   Config reloader started
2020-07-03T12:19:05.510+0300    INFO    log/input.go:152        Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-07-03T12:19:05.511+0300    INFO    log/input.go:152        Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-07-03T12:19:05.511+0300    INFO    cfgfile/reload.go:224   Loading of config files completed.
2020-07-03T12:19:08.483+0300    INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:89     add_cloud_metadata: hosting provider type not detected.
2020-07-03T12:19:09.484+0300    INFO    [publisher_pipeline_output]     pipeline/output.go:144  Connecting to backoff(async(tcp://192.168.0.246:5044))
2020-07-03T12:19:09.484+0300    INFO    [publisher]     pipeline/retry.go:221   retryer: send unwait signal to consumer
2020-07-03T12:19:09.484+0300    INFO    [publisher]     pipeline/retry.go:225     done
2020-07-03T12:19:09.484+0300    INFO    [publisher_pipeline_output]     pipeline/output.go:152  Connection to backoff(async(tcp://192.168.0.246:5044)) established
2020-07-03T12:19:35.495+0300    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":60,"time":{"ms":68}},"total":{"ticks":170,"time":{"ms":179},"value":0},"user":{"ticks":110,"time":{"ms":111}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral_id":"97fc95fc-7877-4a77-ac6c-5c67954fccc2","uptime":{"ms":30063}},"memstats":{"gc_next":14513616,"memory_alloc":7480472,"memory_total":20947432,"rss":47783936},"runtime":{"goroutines":49}},"filebeat":{"events":{"added":66,"done":66},"harvester":{"files":{"2a0280e4-4ba9-46ce-81ae-b16beab1c4d0":{"last_event_published_time":"2020-07-03T12:19:08.484Z","last_event_timestamp":"2020-07-03T12:19:08.484Z","name":"/var/log/messages","read_offset":6112,"size":6112,"start_time":"2020-07-03T12:19:05.493Z"}},"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"events":{"acked":65,"batches":1,"total":65},"read":{"bytes":6},"type":"logstash","write":{"bytes":3385}},"pipeline":{"clients":3,"events":{"active":0,"filtered":1,"published":65,"retry":65,"total":66},"queue":{"acked":65}}},"registrar":{"states":{"current":1,"update":66},"writes":{"success":2,"total":2}},"system":{"cpu":{"cores":4},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}

КОнфиг

output.logstash:
  # The Logstash hosts
    hosts: ["192.168.0.246:5044"]
    protocol: "https"
    index: "messages3-%{[beat.version]}-%{+yyyy.MM.dd}"
    setup.template.name: "messages3"
    setup.template.pattern: "*-messages3-%{[beat.version]}-*"
    setup.template.enabled: true
    compression_level: 9

Но индекс messages3 в кибане не появился.

Логи логстеша

[2020-07-03T05:26:31,580][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.8.0", "jruby.version"=>"jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1a40 OpenJDK 64-Bit Server VM 25.252-b09 on 1.8.0_252-b09 +indy +jit [linux-x86_64]"}
[2020-07-03T05:26:35,731][INFO ][org.reflections.Reflections] Reflections took 117 ms to scan 1 urls, producing 21 keys and 41 values 
[2020-07-03T05:26:37,704][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://192.168.0.246:9200/]}}
[2020-07-03T05:26:37,985][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://192.168.0.246:9200/"}
[2020-07-03T05:26:38,069][INFO ][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2020-07-03T05:26:38,077][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2020-07-03T05:26:38,200][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//192.168.0.246:9200"]}
[2020-07-03T05:26:38,286][INFO ][logstash.outputs.elasticsearch][main] Using default mapping template
[2020-07-03T05:26:38,417][INFO ][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-07-03T05:26:38,624][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/filter.conf", "/etc/logstash/conf.d/input.conf", "/etc/logstash/conf.d/output.conf"], :thread=>"#<Thread:0x4d08a51f run>"}
[2020-07-03T05:26:40,198][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-07-03T05:26:40,254][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-07-03T05:26:40,462][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-07-03T05:26:40,571][INFO ][org.logstash.beats.Server][main][d752886843c30471c83f7a7dca39ebf6f3c3501e8d768c280e6fa83dc24d7773] Starting server on port: 5044
[2020-07-03T05:26:41,073][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Перед filebeat.output пробелов быть не должно, перед - type пробелы должны быть. Ознакомьтесь, пожалуйста, с форматом YAML, пробелы там играют большую роль, а у вас они как только не разбросаны.

Все так, именно структура конфига.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.