Сбор syslog с помощью ELK

Savai · July 29, 2021, 1:38pm

Здравствуйте. Не подскажете, в чем может быть проблема?
Запускаю ELK стек через docker-compose и пытаюсь получить syslog через logstash, однако индекс в elastic не создается и при запуске logstash выдается предупреждение.

"lgs01    | [2021-07-29T13:37:30,507][ERROR][logstash.outputs.elasticsearch][main] Failed to
 install template {:message=>"Got response code '403' contacting Elasticsearch at URL 
'https://es01:9200/_template/logstash'", 
:exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, 
:backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-
elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client
/manticore_adapter.rb:80:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0
/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client
/pool.rb:306:in `perform_request_to_url'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0
/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client
/pool.rb:293:in `block in perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0
/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client
/pool.rb:382:in `with_connection'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0
/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client
/pool.rb:292:in `perform_request'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0
/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client
/pool.rb:300:in `block in Pool'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-
output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:379:in `exists?'", 
"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-
java/lib/logstash/outputs/elasticsearch/http_client.rb:384:in `template_exists?'", "/usr/share
/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash
/outputs/elasticsearch/http_client.rb:80:in `template_install'", "/usr/share/logstash/vendor
/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs
/elasticsearch/template_manager.rb:29:in `install'", "/usr/share/logstash/vendor/bundle/jruby
/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch
/template_manager.rb:17:in `install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0
/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch.rb:496:in 
`install_template'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-
elasticsearch-11.0.2-java/lib/logstash/outputs/elasticsearch.rb:309:in `finish_register'", 
"/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-
java/lib/logstash/outputs/elasticsearch.rb:279:in `block in register'", "/usr/share/logstash/vendor
/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-11.0.2-java/lib/logstash/plugin_mixins
/elasticsearch/common.rb:145:in `block in after_successful_connection'"]}
"

Igor_Motov · July 29, 2021, 11:20pm

Здравствуйте! Хотелось бы услышать о Вашей проблеме поподробнее. Пока могу только указать на Syslog input plugin | Logstash Reference [7.13] | Elastic если интересует именно ELK, если пойдет BELK, то можно еще взглянуть на Syslog input | Filebeat Reference [7.13] | Elastic

Savai · July 30, 2021, 4:54am

Конфигурация pipeline

input {
  udp {
    port => 514
    type => "syslog"
  }
}

output {
  elasticsearch
          {
           hosts => ["https://es01:9200"]
           user => "logstash_system"
           password => "logstashpasswd"
           ssl_certificate_verification => false
           index => "syslog-%{+YYYY.MM.dd}"
          }
}

Savai · July 30, 2021, 5:11am

А еще, не подскажете- в интерфейсе Elastic-a есть пункт создания pipeline для logstash- можно ли создавать pipelines только там, без использования файлов, и если да- то как заставить это работать?

Savai · July 30, 2021, 11:54am

Зарегистрировал пользователя logstash_internal, как написанно в документации, теперь ошибок доступа при запуске logstash нет, но индекс все равно не создается. Что я делаю не так?

input {
  udp {
    port => 514
    type => "syslog"
  }
}

output {
  elasticsearch
          {
           hosts => ["https://es01:9200"]
           user => logstash_internal
           password => logstas_internal_password
           ssl_certificate_verification => false
           index => "syslog-%{+YYYY.MM.dd}"
          }
}

Savai · July 30, 2021, 2:00pm

docker-compose для запуска ELK

version: '2.2'

services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
      - xpack.license.self_generated.type=trial
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
      - certs:$CERTS_DIR
    ports:
      - 9200:9200
    networks:
      - elastic

    healthcheck:
      test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
      interval: 30s
      timeout: 10s
      retries: 5

  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
    container_name: es02
    environment:
      - node.name=es02
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
      - xpack.license.self_generated.type=trial
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=$CERTS_DIR/es02/es02.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es02/es02.crt
      - xpack.security.transport.ssl.enabled=true
 - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es02/es02.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es02/es02.key
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data02:/usr/share/elasticsearch/data
      - certs:$CERTS_DIR
    networks:
      - elastic

  es03:
    image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}
    container_name: es03
    environment:
      - node.name=es03
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es01,es02
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
      - xpack.license.self_generated.type=trial
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=$CERTS_DIR/es03/es03.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es03/es03.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es03/es03.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es03/es03.key
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data03:/usr/share/elasticsearch/data
      - certs:$CERTS_DIR
    networks:
      - elastic
  kib01:
    image: docker.elastic.co/kibana/kibana:${VERSION}
    container_name: kib01
    depends_on: {"es01": {"condition": "service_healthy"}}
    ports:
      - 5601:5601
    environment:
      SERVERNAME: localhost
      ELASTICSEARCH_URL: https://es01:9200
      ELASTICSEARCH_HOSTS: https://es01:9200
      ELASTICSEARCH_USERNAME: kibana_system
      ELASTICSEARCH_PASSWORD: kibana_system_password
      ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES: $CERTS_DIR/ca/ca.crt
      SERVER_SSL_ENABLED: "true"
      SERVER_SSL_KEY: $CERTS_DIR/kib01/kib01.key
 SERVER_SSL_CERTIFICATE: $CERTS_DIR/kib01/kib01.crt
    volumes:
      - certs:$CERTS_DIR
    networks:
      - elastic
  lgs01:
     image: docker.elastic.co/logstash/logstash:${VERSION}
     container_name: lgs01
     depends_on: {"es01": {"condition": "service_healthy"}}
     ports:
       - 5000:5000
       - 5044:5044
     environment:
       - xpack.security.enabled=true
       - xpack.monitoring.enabled=true
       - xpack.monitoring.elasticsearch.hosts=["https://es01:9200"]
       - xpack.monitoring.elasticsearch.username=logstash_system
       - xpack.monitoring.elasticsearch.PASSWORD=logstash_system_password
       - xpack.monitoring.elasticsearch.ssl.certificate_authority=$CERTS_DIR/ca/ca.crt
       - xpack.monitoring.elasticsearch.ssl.verification_mode=certificate
       - xpack.management.enabled:=true
       - xpack.management.elasticsearch.hosts:="https://es01:9200/"
       - xpack.management.elasticsearch.username:=logstash_admin
       - xpack.management.elasticsearch.password:=logstash_admin_password
       - xpack.management.logstash.poll_interval:=5s
       - "LS_OPTS= --config.reload.automatic"
       - "LS_JAVA_OPTS= -Xms1024m -Xmx1024m"
     volumes:
       - certs:$CERTS_DIR
       #- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
       #- ./logstash/pipeline:/usr/share/logstash/pipeline:ro
     networks:
       - elastic
volumes:
  data01:
    driver: local
  data02:
    driver: local
  data03:
    driver: local
  certs:
    driver: local

networks:
  elastic:
    driver: bridge

Igor_Motov · July 30, 2021, 4:07pm

Не понятно, кто у вас сообщения в syslog посылает и как он до logstash добирается, если у вас порт 514 в докере не открыт.

Savai · August 2, 2021, 6:46am

Syslog шлются на 514 порт сервера где установлен ELK
добавил в docker-compose строчку

 - 192.168.1.15:514:5044

ошибок при запуске нет, но индекс все так же не создется.
пробовал создавать pipelines через интерфейс kibana- тоже не работает

Результат вывода команды docker container port lgs01

5044/tcp -> 0.0.0.0:5044
5044/tcp -> :::5044
5044/tcp -> 192.168.1.15:514
5000/tcp -> 0.0.0.0:5000
5000/tcp -> :::5000

Igor_Motov · August 2, 2021, 3:36pm

input {
  udp {
    port => 514
    type => "syslog"
  }
}

Судя по этой конфигурации, ваш Logstash внутри контейнера ожидает syslog трафик на порту 514, а не 5044 и по UDP, а не по TCP.

Savai · August 3, 2021, 5:58am

Syslog на сервер отправляется по udp, поэтому для udp и настраивал.
Указал порт 5044- все равно индек не создается и данные не собираются.

Igor_Motov · August 3, 2021, 3:44pm

У вас маппинг порта в докере настроен по TCP а посылаете и слушаете вы по UDP. Посмотрите документацию докера.

Savai · August 5, 2021, 11:31am

Спасибо за подсказки! С настройкой pipeline через файл все заботало.
А не подскажете- в интерфейсе Kibana есть создания logstash pipeline - копию туда рабочий pipeline из файла- а он не работает, в какую сторону смотреть?

Igor_Motov · August 5, 2021, 10:50pm

       - xpack.management.enabled:=true
       - xpack.management.elasticsearch.hosts:="https://es01:9200/"
       - xpack.management.elasticsearch.username:=logstash_admin
       - xpack.management.elasticsearch.password:=logstash_admin_password
       - xpack.management.logstash.poll_interval:=5s

Двоеточие перед = попробуйте убрать.

system · September 2, 2021, 10:50pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Simple setup question Elasticsearch	3	551	July 6, 2017
Help in Data collection and indexing Elasticsearch	26	1872	June 14, 2018
Issues when running Logstash in Docker Logstash	8	5517	May 10, 2017
Is my Data being received by Elasticsearch? Elasticsearch	18	26077	June 2, 2017
Logstash won't create index in ES Logstash	26	24533	July 6, 2017

Сбор syslog с помощью ELK

Related topics