# Setup output to Elasticsearch through Centralized Management **URL:** \ **Category:** Beats\ **Created:** [November 22, 2018, 1:05pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881 "2018-11-22T13:05:14Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 1:05pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/1 "2018-11-22T13:05:14Z")
Hello, could you please help me to setup Elasticsearch output. I managed to enroll my metricbeat agent but I don't get any information from agent. As you can see I have created tag razvoj\_metricbeat: ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3cf385ad5c3b8bbfbfbce06c9683a617814e328d.png) And this is my output of tag razvoj\_metricbeat: ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/7/f735ab4b6145f09c1aea3fe66749fa0e5bf9efcb.png) Am i doing something wrong? Version of elasticsearch and metricbeat agent 5.6.0. BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 1:19pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/2 "2018-11-22T13:19:33Z")
Hello @mladen Did you also add metricbeat module to the agent? ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 1:54pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/3 "2018-11-22T13:54:55Z")
Hello @pierhugues, thank you for your quick answer. If you mean to enable system module, yes I have it. BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 1:59pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/4 "2018-11-22T13:59:53Z")
Is there any error in the Metricbeat log? ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 2:10pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/5 "2018-11-22T14:10:48Z")
No, there is. This is message in log after I restarted agent: ``` 2018-11-22T14:49:42.857+0100 INFO instance/beat.go:302 Setup Beat: metricbeat; Version: 6.5.0 2018-11-22T14:49:42.857+0100 INFO instance/beat.go:328 Output is configured through Central Management 2018-11-22T14:49:42.858+0100 INFO [publisher] pipeline/module.go:110 Beat name: kaplaytest.it.telekom.yu 2018-11-22T14:49:42.858+0100 INFO instance/beat.go:424 metricbeat start running. 2018-11-22T14:49:42.858+0100 WARN [cfgwarn] management/manager.go:100 BETA: Central management is enabled 2018-11-22T14:49:42.858+0100 INFO [centralmgmt] management/manager.go:101 Starting central management service 2018-11-22T14:49:42.858+0100 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s 2018-11-22T14:49:43.710+0100 INFO [centralmgmt] management/manager.go:198 Applying settings for output 2018-11-22T14:49:43.711+0100 INFO elasticsearch/client.go:163 Elasticsearch url: http://kaeskibanademo:9200 ``` I will try to upgrade to 6.5.1 and add one or more metricbeat agent to Centralized management. I will provide you with an update upon completion this tasks. BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 2:18pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/6 "2018-11-22T14:18:29Z")
Thanks I do not see anything weird in the current trace, Can you start one of the metricbeat with the following flags: `metricbeat -v -e -d "*"`, this should give us the complete trace in debug mode. ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 2:29pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/7 "2018-11-22T14:29:51Z")
This is the output: ``` 2018-11-22T15:22:50.574+0100 INFO instance/beat.go:616 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat] 2018-11-22T15:22:50.575+0100 DEBUG [beat] instance/beat.go:653 Beat metadata path: /var/lib/metricbeat/meta.json 2018-11-22T15:22:50.575+0100 INFO instance/beat.go:623 Beat UUID: 774ee895-7adf-4d70-b556-09333e23ab80 2018-11-22T15:22:50.575+0100 DEBUG [keystore] keystore/keystore.go:119 accessing key 'management.accesstoken' from the keystore 2018-11-22T15:22:50.575+0100 INFO kibana/client.go:118 Kibana url: http://kaeskibanademo:5601 2018-11-22T15:22:50.575+0100 INFO [seccomp] seccomp/seccomp.go:93 Syscall filter could not be installed because the kernel does not support secc omp 2018-11-22T15:22:50.575+0100 INFO [beat] instance/beat.go:849 Beat info {"system_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "774ee895-7adf-4d70-b556-09333e23ab80"}}} 2018-11-22T15:22:50.575+0100 INFO [beat] instance/beat.go:858 Build info {"system_info": {"build": {"commit": "ff5b9b3db49856a25b5eda133b6997f2157a4910", "libbeat": "6.5.0", "time": "2018-11-09T18:03:04.000Z", "version": "6.5.0"}}} 2018-11-22T15:22:50.575+0100 INFO [beat] instance/beat.go:861 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.10.3"}}} 2018-11-22T15:22:50.576+0100 INFO [beat] instance/beat.go:865 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2018-07 -03T16:16:01+02:00","containerized":true,"name":"kaplaytest","ip":["127.0.0.1/8","::1/128","10.123.33.166/27","fe80::250:56ff:fe8a:965/64"],"ker nel_version":"3.10.0-693.el7.x86_64","mac":["00:50:56:8a:09:65"],"os":{"family":"","platform":"rhel","name":"Red Hat Enterprise Linux Server","version":"7.4 ( Maipo)","major":7,"minor":4,"patch":0,"codename":"Maipo"},"timezone":"CET","timezone_offset_sec":3600,"id":"ae4fe779130e4623b7de590ceb7e3a5e"}}} 2018-11-22T15:22:50.576+0100 INFO [beat] instance/beat.go:894 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"perm itted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","ne t_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys _time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective": ["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin ","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time", "sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown" ,"dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_r aw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty _config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/var/log/metricbeat", "exe": "/usr/share/metricbeat/bin/metricbeat", "name": "metricbeat", "pid": 20046, "ppid": 19988, "seccomp": {"mode":"disabled"}, "start_time": "2018-11-22T15:22:49.930+0100"}}} ``` ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 2:30pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/8 "2018-11-22T14:30:06Z")
``` 2018-11-22T15:22:50.576+0100 INFO instance/beat.go:302 Setup Beat: metricbeat; Version: 6.5.0 2018-11-22T15:22:50.576+0100 DEBUG [beat] instance/beat.go:323 Initializing output plugins 2018-11-22T15:22:50.576+0100 INFO instance/beat.go:328 Output is configured through Central Management 2018-11-22T15:22:50.576+0100 DEBUG [processors] processors/processor.go:66 Processors: 2018-11-22T15:22:50.577+0100 DEBUG [publish] pipeline/consumer.go:137 start pipeline event consumer 2018-11-22T15:22:50.577+0100 INFO [publisher] pipeline/module.go:110 Beat name: kaplaytest 2018-11-22T15:22:50.577+0100 DEBUG [modules] beater/metricbeat.go:103 Register [ModuleFactory:[docker, mongodb, mysql, postgresql, system, u wsgi], MetricSetFactory:[aerospike/namespace, apache/status, ceph/cluster_disk, ceph/cluster_health, ceph/cluster_status, ceph/monitor_health, ceph/osd_df, ce ph/osd_tree, ceph/pool_disk, couchbase/bucket, couchbase/cluster, couchbase/node, docker/container, docker/cpu, docker/diskio, docker/healthcheck, docker/imag e, docker/info, docker/memory, docker/network, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster_stats, elasticsearch/index, elasticsearch/index_ recovery, elasticsearch/index_summary, elasticsearch/ml_job, elasticsearch/node, elasticsearch/node_stats, elasticsearch/pending_tasks, elasticsearch/shard, e nvoyproxy/server, etcd/leader, etcd/self, etcd/store, golang/expvar, golang/heap, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, jolokia /jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/event, kubernetes/node, kubern etes/pod, kubernetes/state_container, kubernetes/state_deployment, kubernetes/state_node, kubernetes/state_pod, kubernetes/state_replicaset, kubernetes/state_ statefulset, kubernetes/system, kubernetes/volume, kvm/dommemstat, logstash/node, logstash/node_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mo ngodb/metrics, mongodb/replstatus, mongodb/status, munin/node, mysql/galera_status, mysql/status, nginx/stubstatus, php_fpm/pool, php_fpm/process, postgresql/ activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, prometheus/stats, rabbitmq/connection, rabbitmq/exchange, rabb itmq/node, rabbitmq/queue, redis/info, redis/keyspace, system/core, system/cpu, system/diskio, system/filesystem, system/fsstat, system/load, system/memory, s ystem/network, system/process, system/process_summary, system/raid, system/socket, system/socket_summary, system/uptime, traefik/health, uwsgi/status, vsphere /datastore, vsphere/host, vsphere/virtualmachine, zookeeper/mntr]] 2018-11-22T15:22:50.577+0100 INFO instance/beat.go:424 metricbeat start running. 2018-11-22T15:22:50.577+0100 WARN [cfgwarn] management/manager.go:100 BETA: Central management is enabled 2018-11-22T15:22:50.577+0100 INFO [centralmgmt] management/manager.go:101 Starting central management service 2018-11-22T15:22:50.577+0100 INFO [monitoring] log/log.go:117 Starting metrics logging every 30s 2018-11-22T15:22:50.577+0100 DEBUG [centralmgmt] management/manager.go:164 Retrieving new configurations from Kibana 2018-11-22T15:22:51.567+0100 DEBUG [centralmgmt] management/manager.go:172 configuration didn't change, sleeping 2018-11-22T15:22:51.567+0100 INFO [centralmgmt] management/manager.go:198 Applying settings for output 2018-11-22T15:22:51.567+0100 INFO elasticsearch/client.go:163 Elasticsearch url: http://kaeskibanademo:9200 ``` I try also network connectivity and everything is OK. BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 2:39pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/9 "2018-11-22T14:39:59Z")
Can you add the content of the `data/management.yml` to this post? ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 2:43pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/10 "2018-11-22T14:43:01Z")
From the log I only see the output to be applied to metricbeat, with the content of the `data/management.yml` I should be able see what is the actual config of the beat. Also can you go to the management ui and show me all the config block for the specific tag? From your screenshot it should be `razvoj-metricbeat` ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 2:47pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/11 "2018-11-22T14:47:21Z")
This is the config of management.yml file: ``` configok: true configs: - type: output blocks: - raw: elasticsearch: hosts: - kaeskibanademo:9200 password: ******** username: elastic output: elasticsearch ``` And I have just one config block for this tag: ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/2/c25b5b79d94a2fb775550bbdd0e5bbe2b67aba45.png) BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 3:09pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/12 "2018-11-22T15:09:05Z")
When you setup metricbeat to use config management, The management API becomes the source of truth and will ignore any local configs. So from what I see in the screenshots, you only have an **elasticsearch output** configured in the management UI and looking at the content of the _management.yml_ metricbeat only have the output configured. So both of theses view are sync. If you want to have metricbeat send data to Elasticsearch you have to click `Add configuration block` and add a metricbeat module configuration block. Doing so will make Metricbeat send event to Elasticsearch. ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 22, 2018, 3:26pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/13 "2018-11-22T15:26:21Z")
Sorry, I didn't understand part with system module in centralized management. I add this block: ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/f/5f804f6563c2e9f61f5db25cb53f5cfe657e6253.png) Now I have two blocks: ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/f/9f426d534c7025bb4c1b3c187951098e82e7287f.png) Should I add some more blocks? BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 22, 2018, 4:09pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/14 "2018-11-22T16:09:07Z")
@mladen With the two config block, you see events send to elasticsearch now from a metricbeat instance, and we should also see the change pick up in the metricbeat log and data/management.yml. ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 23, 2018, 8:03am UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/15 "2018-11-23T08:03:12Z")
@pierhugues thank you very much for your help and patience. Please note that beside configuration that was applied I still had to load the index template in Elasticsearch manually. It seams to me that if you use centralized management, agent don't load index template in Elasticsearch automatically. Just one more question, is it possible to use metricbeat key-store and put variable in management.yml file instead of plain text? BR, Mladen ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [November 26, 2018, 1:14pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/16 "2018-11-26T13:14:39Z")
@mladen Yes the template and the dashboard still requires you to run the setup command to make them available. Its a known issue and we are working on it. > Just one more question, is it possible to use metricbeat key-store and put variable in management.yml file instead of plain text? Yes you can do that, you can add secret to the local keystore and reference them in the config management. We are considering allowing config management to send a keystore to the running instances. FYI: The token that config management is using on the beats to connect to the API is already stored in the keystore. ---
**Author:** ![mladen](https://avatars.discourse-cdn.com/v4/letter/m/e36b37/32.png) [@mladen](https://discuss.elastic.co/u/mladen)\ **Post date:** [November 26, 2018, 2:24pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/17 "2018-11-26T14:24:29Z")
@pierhugues thanks a lot for your explanations. You help me to understand how bets management works. BR, Mladen ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 24, 2018, 4:24pm UTC](https://discuss.elastic.co/t/setup-output-to-elasticsearch-through-centralized-management/157881/18 "2018-12-24T16:24:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.