# Два хоста filebeat +ELK

**URL:** <https://discuss.elastic.co/t/filebeat-elk/239596>\
**Category:** Вопросы на русском языке\
**Created:** [July 2, 2020, 8:52am UTC](https://discuss.elastic.co/t/filebeat-elk/239596 "2020-07-02T08:52:34Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 2, 2020, 8:52am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/1 "2020-07-02T08:52:34Z")

</div>

Добрый день.  
У меня настроен ELK 7.8.0 и filebeat 7.8.0 который читает лог1 и отдает с индексом лог1 в logstash.  
Собственно возникла потребность подключить еще один хост filbeat с лог2 и индексом лог2 для logstash.  
На первом хосте такие настройки filebeat.yml

```auto
ilebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  paths:
    - /opt/software/logs/xxx.log
output.logstash:
  # The Logstash hosts
  hosts: ["192.168.0.246:5044"]
  protocol: "https"
  index: "index1-%{[beat.version]}-%{+yyyy.MM.dd}"
  setup.template.name: "index1"
  setup.template.pattern: "*-index1-%{[beat.version]}-*"
  setup.template.enabled: true
  compression_level: 9

```

Делаю такие же настройки на втором хосте filebeat соответственно поменяв index1 на index2, но он не появляется в кибане т.е. его как бы нет.  
В чем может быть проблема?

UPD.  
на хосте2 стоит centos 6 а на остальных хостах centos 7.  
на втором хосте файлбита, пробовал установить filebeat 6.8, результат тот же.

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 2, 2020, 1:27pm UTC](https://discuss.elastic.co/t/filebeat-elk/239596/2 "2020-07-02T13:27:05Z")

</div>

Лог при старте:

```auto
2020-07-02T16:25:36.432+0300 INFO input/input.go:114 Starting input of type: log; ID: 5479216256157809241 
2020-07-02T16:25:36.432+0300 INFO input/input.go:114 Starting input of type: log; ID: 7629700818984689685 
2020-07-02T16:25:36.432+0300 INFO input/input.go:114 Starting input of type: log; ID: 12526209409590384984 
2020-07-02T16:25:36.432+0300 INFO input/input.go:114 Starting input of type: log; ID: 8446139989763442651 
2020-07-02T16:25:36.434+0300 INFO log/input.go:138 Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-07-02T16:25:36.434+0300 INFO log/input.go:138 Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-07-02T16:25:36.434+0300 INFO input/input.go:114 Starting input of type: log; ID: 12421169689888292265 
2020-07-02T16:25:36.435+0300 INFO input/input.go:114 Starting input of type: log; ID: 10618728313673870856 
2020-07-02T16:25:36.435+0300 INFO cfgfile/reload.go:205 Loading of config files completed.
2020-07-02T16:25:39.390+0300 INFO add_cloud_metadata/add_cloud_metadata.go:346 add_cloud_metadata: hosting provider type not detected.
2020-07-02T16:26:06.398+0300 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":20,"time":{"ms":23}},"total":{"ticks":80,"time":{"ms":85},"value":80},"user":{"ticks":60,"time":{"ms":62}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":4},"info":{"ephemeral_id":"22cc81f0-8be7-4669-a132-4161ff862795","uptime":{"ms":30025}},"memstats":{"gc_next":4194304,"memory_alloc":2013560,"memory_total":8634408,"rss":23093248}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":15,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":4},"load":{"1":0.04,"15":0,"5":0.05,"norm":{"1":0.01,"15":0,"5":0.0125}}}}}}

```

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [July 2, 2020, 3:25pm UTC](https://discuss.elastic.co/t/filebeat-elk/239596/3 "2020-07-02T15:25:37Z")

</div>

В начале `f` не хватает и пробелов перед `- type: log`

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 3, 2020, 7:08am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/4 "2020-07-03T07:08:12Z")

</div>

Странно, надругих хостах у меня все работает именно в формате:

```auto
#=========================== Filebeat inputs =============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
   - /varl/log/messages

```

После того как привожу конфиг к вашему формату:

```auto
#=========================== Filebeat inputs =============================

  filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

   - type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
     - /varl/log/messages

```

Получаю ошибку

```auto
[root@app filebeat]# /etc/init.d/filebeat start
Starting filebeat: Exiting: error loading config file: yaml: line 68: did not find expected <document start>

```

Видимо съехали поля при редактировании, привожу конфиг к нормальному виду и все запускается

```auto
root@app filebeat]# /etc/init.d/filebeat start
Starting filebeat: 2020-07-03T10:00:17.627+0300 INFO instance/beat.go:571 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2020-07-03T10:00:17.627+0300 INFO instance/beat.go:579 Beat ID: ee7adef5-45a4-4baa-ab06-966a2db6c22c
2020-07-03T10:00:17.627+0300 INFO [index-management.ilm] ilm/ilm.go:129 Policy name: filebeat-7.1.0
2020-07-03T10:00:17.630+0300 INFO [beat] instance/beat.go:827 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "ee7adef5-45a4-4baa-ab06-966a2db6c22c"}}}
2020-07-03T10:00:17.630+0300 INFO [beat] instance/beat.go:836 Build info {"system_info": {"build": {"commit": "03b3db2a1d9d76fdf10475e829fce436c61901e4", "libbeat": "7.1.0", "time": "2019-05-15T23:59:19.000Z", "version": "7.1.0"}}}
2020-07-03T10:00:17.630+0300 INFO [beat] instance/beat.go:839 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.11.5"}}}
2020-07-03T10:00:17.632+0300 INFO [beat] instance/beat.go:843 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-05-21T14:40:54+03:00","containerized":true,"name":"app.dev","ip":["127.0.0.1/8","::1/128","192.168.0.206/24","fe80::20c:29ff:fe95:eb75/64"],"kernel_version":"2.6.32-754.15.3.el6.x86_64","mac":["00:0c:29:95:eb:75"],"os":{"family":"redhat","platform":"centos","name":"CentOS","version":"6.10 (Final)","major":6,"minor":10,"patch":0,"codename":"Final"},"timezone":"MSK","timezone_offset_sec":10800,"id":"628a515858a7dc0c40847abf00000008"}}}
2020-07-03T10:00:17.632+0300 INFO [beat] instance/beat.go:872 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63"],"ambient":null}, "cwd": "/etc/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 53464, "ppid": 53463, "seccomp": {"mode":""}, "start_time": "2020-07-03T10:00:16.630+0300"}}}
2020-07-03T10:00:17.632+0300 INFO instance/beat.go:280 Setup Beat: filebeat; Version: 7.1.0
2020-07-03T10:00:17.633+0300 INFO [publisher] pipeline/module.go:97 Beat name: app.dev
Config OK
                                                           [OK]

```

Вот секция output:

```auto
#----------------------------- Logstash output --------------------------------
  output.logstash:
  # The Logstash hosts
   hosts: ["192.168.0.246:5044"]
   protocol: "https"
   index: "messages3-%{[beat.version]}-%{+yyyy.MM.dd}"
   setup.template.name: "messages3"
   setup.template.pattern: "*-messages3-%{[beat.version]}-*"
   setup.template.enabled: true
   compression_level: 9

```

Но в кибане я его так и не вижу(((  
Единственное отличие этого хоста от других, в центоси 6.10, на других 7-ка и версии filebeat 7.1.0 после того как не взлетело на 7.8.0, хотя и та и другая версия совместимы по матрице со стеком ELK версии 7.8.0 который я развернул.  
В чем проблема непонятно.

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 3, 2020, 7:23am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/5 "2020-07-03T07:23:23Z")

</div>

Далее видим вот что в логе:

```auto
2020-07-03T10:16:22.740+0300 INFO [monitoring] log/log.go:153 Uptime: 38.085991ms
2020-07-03T10:16:22.740+0300 INFO [monitoring] log/log.go:130 Stopping metrics logging.
2020-07-03T10:16:22.740+0300 INFO instance/beat.go:401 filebeat stopped.
2020-07-03T10:16:22.741+0300 ERROR instance/beat.go:802 Exiting: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')
Exiting: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')

```

А там то уже все ок поидее:

```auto
#=========================== Filebeat inputs =============================

  filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

   - type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /varl/log/messages

```

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 3, 2020, 8:11am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/6 "2020-07-03T08:11:49Z")

</div>

Покопался в гуглах:  
Привел к виду:

```auto
#=========================== Filebeat inputs =============================

  filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

   - type: log

  # Change to true to enable this input configuration.
     enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
     paths:
       - /varl/log/messages

```

В логах

```auto
2020-07-03T10:50:38.797+0300 INFO instance/beat.go:280 Setup Beat: filebeat; Version: 7.1.0
2020-07-03T10:50:38.798+0300 INFO [publisher] pipeline/module.go:97 Beat name: app.dev
Config OK
                                                           [OK]
[root@app filebeat]# tail -f /var/log/filebeat/filebeat
2020-07-03T10:50:38.867+0300 INFO input/input.go:114 Starting input of type: log; ID: 5479216256157809241 
2020-07-03T10:50:38.867+0300 INFO input/input.go:114 Starting input of type: log; ID: 7629700818984689685 
2020-07-03T10:50:38.867+0300 INFO input/input.go:114 Starting input of type: log; ID: 12526209409590384984 
2020-07-03T10:50:38.867+0300 INFO input/input.go:114 Starting input of type: log; ID: 8446139989763442651 
2020-07-03T10:50:38.869+0300 INFO log/input.go:138 Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-07-03T10:50:38.869+0300 INFO log/input.go:138 Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-07-03T10:50:38.869+0300 INFO input/input.go:114 Starting input of type: log; ID: 12421169689888292265 
2020-07-03T10:50:38.869+0300 INFO input/input.go:114 Starting input of type: log; ID: 10618728313673870856 
2020-07-03T10:50:38.869+0300 INFO cfgfile/reload.go:205 Loading of config files completed.
2020-07-03T10:50:41.844+0300 INFO add_cloud_metadata/add_cloud_metadata.go:346 add_cloud_metadata: hosting provider type not detected.
2020-07-03T10:51:08.851+0300 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":53}},"total":{"ticks":110,"time":{"ms":119},"value":0},"user":{"ticks":60,"time":{"ms":66}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":4},"info":{"ephemeral_id":"ddee7a9d-76c1-4918-8458-1f0522b4de07","uptime":{"ms":30036}},"memstats":{"gc_next":4204096,"memory_alloc":2156176,"memory_total":8808296,"rss":29483008}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"type":"logstash"},"pipeline":{"clients":15,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":4},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}
2020-07-03T10:51:38.849+0300 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":50,"time":{"ms":1}},"total":{"ticks":120,"time":{"ms":10},"value":120},"user":{"ticks":70,"time":{"ms":9}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":4},"info":{"ephemeral_id":"ddee7a9d-76c1-4918-8458-1f0522b4de07","uptime":{"ms":60034}},"memstats":{"gc_next":4204096,"memory_alloc":2285976,"memory_total":8954032}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":15,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}

```

В кибане по прежнему тишина.

В логах logstash

```auto
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: -12

```

Но при этом остальные хосты нормально пишут логи в елку.

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 3, 2020, 8:30am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/7 "2020-07-03T08:30:16Z")

</div>

Поправил output закаментил protocol b и ошибка ушла.

```auto
#----------------------------- Logstash output --------------------------------
  output.logstash:
  # The Logstash hosts
   hosts: ["192.168.0.246:5044"]
   #protocol: "https"
   index: "pfc.qa2-%{[beat.version]}-%{+yyyy.MM.dd}"
   setup.template.name: "pfc.qa2"
   setup.template.pattern: "*-pfc.qa2-%{[beat.version]}-*"
   setup.template.enabled: true
   compression_level: 9

```

Лог логстеша:

```auto
root@localhost logstash]# tail -f /var/log/logstash/logstash-plain.log 
[2020-07-03T04:09:47,333][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2020-07-03T04:09:47,509][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//192.168.0.246:9200"]}
[2020-07-03T04:09:47,658][INFO][logstash.outputs.elasticsearch][main] Using default mapping template
[2020-07-03T04:09:48,012][INFO][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-07-03T04:09:48,217][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/filter.conf", "/etc/logstash/conf.d/input.conf", "/etc/logstash/conf.d/output.conf"], :thread=>"#<Thread:0x6708f832 run>"}
[2020-07-03T04:09:49,715][INFO][logstash.inputs.beats][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-07-03T04:09:49,745][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2020-07-03T04:09:49,897][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-07-03T04:09:49,919][INFO][org.logstash.beats.Server][main][d752886843c30471c83f7a7dca39ebf6f3c3501e8d768c280e6fa83dc24d7773] Starting server on port: 5044
[2020-07-03T04:09:50,279][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}

```

В кибане ноыого индекса не появилось(  
Может дело в том что я это уже третий сервер с filebeat?

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 3, 2020, 9:28am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/8 "2020-07-03T09:28:46Z")

</div>

Снес все к фигам и установил снова filebeat 7.8.0.

```auto
020-07-03T12:19:05.505+0300 INFO cfgfile/reload.go:164 Config reloader started
2020-07-03T12:19:05.510+0300 INFO log/input.go:152 Configured paths: [/var/log/logstash/logstash-slowlog-plain*.log]
2020-07-03T12:19:05.511+0300 INFO log/input.go:152 Configured paths: [/var/log/logstash/logstash-plain*.log]
2020-07-03T12:19:05.511+0300 INFO cfgfile/reload.go:224 Loading of config files completed.
2020-07-03T12:19:08.483+0300 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:89 add_cloud_metadata: hosting provider type not detected.
2020-07-03T12:19:09.484+0300 INFO [publisher_pipeline_output] pipeline/output.go:144 Connecting to backoff(async(tcp://192.168.0.246:5044))
2020-07-03T12:19:09.484+0300 INFO [publisher] pipeline/retry.go:221 retryer: send unwait signal to consumer
2020-07-03T12:19:09.484+0300 INFO [publisher] pipeline/retry.go:225 done
2020-07-03T12:19:09.484+0300 INFO [publisher_pipeline_output] pipeline/output.go:152 Connection to backoff(async(tcp://192.168.0.246:5044)) established
2020-07-03T12:19:35.495+0300 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":60,"time":{"ms":68}},"total":{"ticks":170,"time":{"ms":179},"value":0},"user":{"ticks":110,"time":{"ms":111}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral_id":"97fc95fc-7877-4a77-ac6c-5c67954fccc2","uptime":{"ms":30063}},"memstats":{"gc_next":14513616,"memory_alloc":7480472,"memory_total":20947432,"rss":47783936},"runtime":{"goroutines":49}},"filebeat":{"events":{"added":66,"done":66},"harvester":{"files":{"2a0280e4-4ba9-46ce-81ae-b16beab1c4d0":{"last_event_published_time":"2020-07-03T12:19:08.484Z","last_event_timestamp":"2020-07-03T12:19:08.484Z","name":"/var/log/messages","read_offset":6112,"size":6112,"start_time":"2020-07-03T12:19:05.493Z"}},"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"events":{"acked":65,"batches":1,"total":65},"read":{"bytes":6},"type":"logstash","write":{"bytes":3385}},"pipeline":{"clients":3,"events":{"active":0,"filtered":1,"published":65,"retry":65,"total":66},"queue":{"acked":65}}},"registrar":{"states":{"current":1,"update":66},"writes":{"success":2,"total":2}},"system":{"cpu":{"cores":4},"load":{"1":0,"15":0,"5":0,"norm":{"1":0,"15":0,"5":0}}}}}}

```

КОнфиг

```auto
output.logstash:
  # The Logstash hosts
    hosts: ["192.168.0.246:5044"]
    protocol: "https"
    index: "messages3-%{[beat.version]}-%{+yyyy.MM.dd}"
    setup.template.name: "messages3"
    setup.template.pattern: "*-messages3-%{[beat.version]}-*"
    setup.template.enabled: true
    compression_level: 9

```

Но индекс messages3 в кибане не появился.

Логи логстеша

```auto
[2020-07-03T05:26:31,580][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.8.0", "jruby.version"=>"jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1a40 OpenJDK 64-Bit Server VM 25.252-b09 on 1.8.0_252-b09 +indy +jit [linux-x86_64]"}
[2020-07-03T05:26:35,731][INFO][org.reflections.Reflections] Reflections took 117 ms to scan 1 urls, producing 21 keys and 41 values 
[2020-07-03T05:26:37,704][INFO][logstash.outputs.elasticsearch][main] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://192.168.0.246:9200/]}}
[2020-07-03T05:26:37,985][WARN][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>"http://192.168.0.246:9200/"}
[2020-07-03T05:26:38,069][INFO][logstash.outputs.elasticsearch][main] ES Output version determined {:es_version=>7}
[2020-07-03T05:26:38,077][WARN][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2020-07-03T05:26:38,200][INFO][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//192.168.0.246:9200"]}
[2020-07-03T05:26:38,286][INFO][logstash.outputs.elasticsearch][main] Using default mapping template
[2020-07-03T05:26:38,417][INFO][logstash.outputs.elasticsearch][main] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
[2020-07-03T05:26:38,624][INFO][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/etc/logstash/conf.d/filter.conf", "/etc/logstash/conf.d/input.conf", "/etc/logstash/conf.d/output.conf"], :thread=>"#<Thread:0x4d08a51f run>"}
[2020-07-03T05:26:40,198][INFO][logstash.inputs.beats][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-07-03T05:26:40,254][INFO][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[2020-07-03T05:26:40,462][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-07-03T05:26:40,571][INFO][org.logstash.beats.Server][main][d752886843c30471c83f7a7dca39ebf6f3c3501e8d768c280e6fa83dc24d7773] Starting server on port: 5044
[2020-07-03T05:26:41,073][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}

```

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [July 6, 2020, 3:20pm UTC](https://discuss.elastic.co/t/filebeat-elk/239596/9 "2020-07-06T15:20:38Z")

</div>

Перед `filebeat.output` пробелов быть не должно, перед `- type` пробелы должны быть. Ознакомьтесь, пожалуйста, с форматом [YAML](https://ru.wikipedia.org/wiki/YAML), пробелы там играют большую роль, а у вас они как только не разбросаны.

---

<div class="post-metadata">

**Author:** ![Ivan\_Bykov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ivan_bykov/32/71087_2.png) [@Ivan\_Bykov](https://discuss.elastic.co/u/Ivan_Bykov)\
**Post date:** [July 7, 2020, 6:27am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/10 "2020-07-07T06:27:54Z")

</div>

Все так, именно структура конфига.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 4, 2020, 6:27am UTC](https://discuss.elastic.co/t/filebeat-elk/239596/11 "2020-08-04T06:27:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.