# ECE deployment cannot connect filebeats to kibana

**URL:** <https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575>\
**Category:** Elastic Cloud Enterprise (ECE)\
**Created:** [October 16, 2018, 1:35am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575 "2018-10-16T01:35:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![JeanD-SYD](https://avatars.discourse-cdn.com/v4/letter/j/85e7bf/32.png) [@JeanD-SYD](https://discuss.elastic.co/u/JeanD-SYD)\
**Post date:** [October 16, 2018, 1:35am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/1 "2018-10-16T01:35:43Z")

</div>

Hi,

I have deployment the on-premise ECE, following the guide. Just a simple POC, but it seems the the licence will expire before I can get some real data in the system.  
Best to then stick with Splunk 🙂

3 server  
ece-3a-01.ece-elastic.xyz  
ece-3b-01.ece-elastic.xyz  
ece-3c-01.ece-elastic.xyz  
I have a wildcard DNS and Wildcard SSL certificate \*.ece-elastic.xyz

I have deploy a single deployment kibana, no customization.  
I have not modified or changed the default kibana deployment as deploy from ECE.

I installed 2 ubuntu 16.04 beats servers and installed filebeat-6.4.2-amd64.deb

I modified the filebeat.yml, but the setting seem a bit of a mystery and not well documented for this configuration.

I was told not to use the cloud.id: or cloud.auth as this is for the cloud version

Here are my setting  
I have tried various combinations, but does any know what is should be pointed to.  
eg  
Elasticsearch  
[https://261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243/](https://261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243/)  
kibana  
[https://2f2122e03fe0499db11412a9e0b69b4a.ece-elastic.xyz:9243/](https://2f2122e03fe0499db11412a9e0b69b4a.ece-elastic.xyz:9243/)

#-------------------------- Elasticsearch output ------------------------------  
output.elasticsearch:

# Array of hosts to connect to.

#hosts: ["localhost:9200"]

hosts: ["ece-3a-01.ece-elastic.xyz:9343"]

protocol: "https"  
username: "elastic"  
password: "3VC933BDccAl8Q90iZo0yJg9"

---

<div class="post-metadata">

**Author:** ![JeanD-SYD](https://avatars.discourse-cdn.com/v4/letter/j/85e7bf/32.png) [@JeanD-SYD](https://discuss.elastic.co/u/JeanD-SYD)\
**Post date:** [October 16, 2018, 1:36am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/2 "2018-10-16T01:36:28Z")

</div>

this was with port :9343

ubuntu@filebeats-2:/etc/filebeat$ sudo filebeat setup --e  
2018-10-16T01:13:50.719Z INFO instance/beat.go:544 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]  
2018-10-16T01:13:50.720Z INFO instance/beat.go:551 Beat UUID: c3de4395-5173-4333-8a8a-b13e2ff374c9  
2018-10-16T01:13:50.720Z INFO [beat] instance/beat.go:768 Beat info {"system\_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "c3de4395-5173-4333-8a8a-b13e2ff374c9"}}}  
2018-10-16T01:13:50.720Z INFO [beat] instance/beat.go:777 Build info {"system\_info": {"build": {"commit": "e193f6d68b25b7ddbe3a3ed8d60bc07fea1ef800", "libbeat": "6.4.2", "time": "2018-09-26T12:42:46.000Z", "version": "6.4.2"}}}  
2018-10-16T01:13:50.720Z INFO [beat] instance/beat.go:780 Go runtime info {"system\_info": {"go": {"os":"linux","arch":"amd64","max\_procs":1,"version":"go1.10.3"}}}  
2018-10-16T01:13:50.721Z INFO [beat] instance/beat.go:784 Host info {"system\_info": {"host": {"architecture":"x86\_64","boot\_time":"2018-10-16T00:11:53Z","containerized":false,"hostname":"filebeats-2","ips":["127.0.0.1/8","::1/128","192.168.201.19/24","fe80::f816:3eff:fe76:28e5/64"],"kernel\_version":"4.4.0-135-generic","mac\_addresses":["fa:16:3e:76:28:e5"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"16.04.5 LTS (Xenial Xerus)","major":16,"minor":4,"patch":5,"codename":"xenial"},"timezone":"UTC","timezone\_offset\_sec":0,"id":"e947bef37f61461a800be9372cd93f9e"}}}  
2018-10-16T01:13:50.721Z INFO [beat] instance/beat.go:813 Process info {"system\_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend","audit\_read"],"effective":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend","audit\_read"],"bounding":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend","audit\_read"],"ambient":null}, "cwd": "/etc/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 26533, "ppid": 26532, "seccomp": {"mode":"disabled"}, "start\_time": "2018-10-16T01:13:50.060Z"}}}  
2018-10-16T01:13:50.722Z INFO instance/beat.go:273 Setup Beat: filebeat; Version: 6.4.2  
2018-10-16T01:13:50.722Z INFO elasticsearch/client.go:163 Elasticsearch url: [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343)  
2018-10-16T01:13:50.723Z INFO pipeline/module.go:98 Beat name: filebeats-2  
2018-10-16T01:13:50.723Z INFO elasticsearch/client.go:163 Elasticsearch url: [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343)  
2018-10-16T01:13:51.008Z ERROR elasticsearch/elasticsearch.go:214 Error connecting to Elasticsearch at [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343): Get [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343): net/http: HTTP/1.x transport connection broken: malformed HTTP status code "acceptable"  
2018-10-16T01:13:51.008Z ERROR instance/beat.go:743 Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343): Get [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343): net/http: HTTP/1.x transport connection broken: malformed HTTP status code "acceptable"]  
Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343): Get [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343): net/http: HTTP/1.x transport connection broken: malformed HTTP status code "acceptable"]

---

<div class="post-metadata">

**Author:** ![JeanD-SYD](https://avatars.discourse-cdn.com/v4/letter/j/85e7bf/32.png) [@JeanD-SYD](https://discuss.elastic.co/u/JeanD-SYD)\
**Post date:** [October 16, 2018, 1:36am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/3 "2018-10-16T01:36:40Z")

</div>

this was with port :9243

ubuntu@filebeats-2:/etc/filebeat$ sudo filebeat setup --e  
2018-10-16T01:32:48.571Z INFO instance/beat.go:544 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]  
2018-10-16T01:32:48.571Z INFO instance/beat.go:551 Beat UUID: c3de4395-5173-4333-8a8a-b13e2ff374c9  
2018-10-16T01:32:48.571Z INFO [beat] instance/beat.go:768 Beat info {"system\_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "c3de4395-5173-4333-8a8a-b13e2ff374c9"}}}  
2018-10-16T01:32:48.572Z INFO [beat] instance/beat.go:777 Build info {"system\_info": {"build": {"commit": "e193f6d68b25b7ddbe3a3ed8d60bc07fea1ef800", "libbeat": "6.4.2", "time": "2018-09-26T12:42:46.000Z", "version": "6.4.2"}}}  
2018-10-16T01:32:48.572Z INFO [beat] instance/beat.go:780 Go runtime info {"system\_info": {"go": {"os":"linux","arch":"amd64","max\_procs":1,"version":"go1.10.3"}}}  
2018-10-16T01:32:48.573Z INFO [beat] instance/beat.go:784 Host info {"system\_info": {"host": {"architecture":"x86\_64","boot\_time":"2018-10-16T00:11:53Z","containerized":false,"hostname":"filebeats-2","ips":["127.0.0.1/8","::1/128","192.168.201.19/24","fe80::f816:3eff:fe76:28e5/64"],"kernel\_version":"4.4.0-135-generic","mac\_addresses":["fa:16:3e:76:28:e5"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"16.04.5 LTS (Xenial Xerus)","major":16,"minor":4,"patch":5,"codename":"xenial"},"timezone":"UTC","timezone\_offset\_sec":0,"id":"e947bef37f61461a800be9372cd93f9e"}}}  
2018-10-16T01:32:48.573Z INFO [beat] instance/beat.go:813 Process info {"system\_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend","audit\_read"],"effective":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend","audit\_read"],"bounding":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend","audit\_read"],"ambient":null}, "cwd": "/etc/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 26581, "ppid": 26580, "seccomp": {"mode":"disabled"}, "start\_time": "2018-10-16T01:32:47.910Z"}}}  
2018-10-16T01:32:48.574Z INFO instance/beat.go:273 Setup Beat: filebeat; Version: 6.4.2  
2018-10-16T01:32:48.574Z INFO elasticsearch/client.go:163 Elasticsearch url: [https://ece-3a-01.ece-elastic.xyz:9243](https://ece-3a-01.ece-elastic.xyz:9243)  
2018-10-16T01:32:48.575Z INFO pipeline/module.go:98 Beat name: filebeats-2  
2018-10-16T01:32:48.576Z INFO elasticsearch/client.go:163 Elasticsearch url: [https://ece-3a-01.ece-elastic.xyz:9243](https://ece-3a-01.ece-elastic.xyz:9243)  
2018-10-16T01:32:49.131Z ERROR elasticsearch/elasticsearch.go:214 Error connecting to Elasticsearch at [https://ece-3a-01.ece-elastic.xyz:9243](https://ece-3a-01.ece-elastic.xyz:9243): 404 Not Found: {"ok":false,"message":"Unknown cluster."}  
2018-10-16T01:32:49.132Z ERROR instance/beat.go:743 Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch [https://ece-3a-01.ece-elastic.xyz:9243](https://ece-3a-01.ece-elastic.xyz:9243): 404 Not Found: {"ok":false,"message":"Unknown cluster."}]  
Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch [https://ece-3a-01.ece-elastic.xyz:9243](https://ece-3a-01.ece-elastic.xyz:9243): 404 Not Found: {"ok":false,"message":"Unknown cluster."}]

---

<div class="post-metadata">

**Author:** ![JeanD-SYD](https://avatars.discourse-cdn.com/v4/letter/j/85e7bf/32.png) [@JeanD-SYD](https://discuss.elastic.co/u/JeanD-SYD)\
**Post date:** [October 16, 2018, 1:36am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/4 "2018-10-16T01:36:49Z")

</div>

ubuntu@filebeats-2:/etc/filebeat$ curl -sv '\*' [https://ece-3a-01.ece-elastic.xyz:9343](https://ece-3a-01.ece-elastic.xyz:9343)

- Rebuilt URL to: \*/
- Trying ::1...
- connect to ::1 port 80 failed: Connection refused
- Trying 127.0.0.1...
- connect to 127.0.0.1 port 80 failed: Connection refused
- Failed to connect to \* port 80: Connection refused
- Closing connection 0
- Rebuilt URL to: [https://ece-3a-01.ece-elastic.xyz:9343/](https://ece-3a-01.ece-elastic.xyz:9343/)
- Trying 10.243.196.19...
- Connected to ece-3a-01.ece-elastic.xyz (10.243.196.19) port 9343 (#1)
- found 148 certificates in /etc/ssl/certs/ca-certificates.crt
- found 592 certificates in /etc/ssl/certs
- ALPN, offering http/1.1
- SSL connection using TLS1.2 / ECDHE\_RSA\_AES\_256\_GCM\_SHA384
- 

```
   server certificate verification OK

```

- 

```
   server certificate status verification SKIPPED

```

- 

```
   common name: *.ece-elastic.xyz (matched)

```

- 

```
   server certificate expiration date OK

```

- 

```
   server certificate activation date OK

```

- 

```
   certificate public key: RSA

```

- 

```
   certificate version: #3

```

- 

```
   subject: CN=*.ece-elastic.xyz

```

- 

```
   start date: Sat, 13 Oct 2018 06:54:11 GMT

```

- 

```
   expire date: Fri, 11 Jan 2019 06:54:11 GMT

```

- 

```
   issuer: C=US,O=Let's Encrypt,CN=Let's Encrypt Authority X3

```

- 

```
   compression: NULL

```

- ALPN, server did not agree to a protocol

> GET / HTTP/1.1  
> Host: ece-3a-01.ece-elastic.xyz:9343  
> User-Agent: curl/7.47.0  
> Accept: _/_

- Connection #1 to host ece-3a-01.ece-elastic.xyz left intact  
\*▒▒▒▒▒No acceptable header received.

---

<div class="post-metadata">

**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\
**Post date:** [October 16, 2018, 2:33pm UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/5 "2018-10-16T14:33:34Z")

</div>

~~`9343` is the transport protocol port, can you try `9243` which is the HTTPS port? I believe `beats` uses that (and `curl` definitely does)~~

Sorry I only read the last message, reading the rest now

Alex

---

<div class="post-metadata">

**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\
**Post date:** [October 16, 2018, 2:50pm UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/6 "2018-10-16T14:50:49Z")

</div>

OK I see the problem:

(Note first that `9243` is the correct port as described above)

`Exiting: Couldn't connect to any of the configured Elasticsearch hosts. Errors: [Error connection to Elasticsearch https://ece-3a-01.ece-elastic.xyz:9243: 404 Not Found: {"ok":false,"message":"Unknown cluster."}]`

The formats of the cluster URLs are of the form `<clusterid>.<domain>` eg if your cluster Id (listed in the overview page on the UI / returned from the create API call) is `261f41b5d7114d2fb96c403bed80c148` and as you said your domain prefix is `*.ece-elastic.xyz` then your cluster URL would look like `https://261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243`

So the error you see above is expected since `ece-3a-01` is not a cluster ID

Now ... currently your 3 hosts are eg `ece-3*-01.ece-elastic.xyz` so you need to make your wildcard DNS point `*.ece-elastic.xyz` to a round-robin of the 3 ECE hosts (assuming you have a proxy role on each of them)

Once you have done that then you can check the DNS setup works by doing eg `host ANYTHING.ece-elastic.xyz` (or `dig`) and it should return one of the 3 IPs of `ece-3a-01`, `ece-3b-01` and `ece-3c-01`

(At this point you can also set the `cname` in the settings page of the UI to `ece-elastic.xyz` and it will auto-generate the correct URLs on the cluster overview pages)

Once you have confirmed that then you would just set eg:

```auto
hosts: ["https://261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243"]

protocol: "https"
username: "elastic"
password: "3VC933BDccAl8Q90iZo0yJg9"

```

(note: I'm assuming your setup is internal - otherwise you should change all the passwords etc 🙂 )

and it should connect (note I used the cluster ID of elasticsearch not Kibana)

Does that make sense?

Sorry that all the network set-up is a pain (for a quick out-of-the box experience, we provide the `ip.es.io` service which just maps `<IP>.ip.es.io` to IP `<IP>` so you can route easily to a single node of an ECE install via `<CLUSTERID>.<IP>.ip.es.io` without having to mess about with DNS

Alex

---

<div class="post-metadata">

**Author:** ![JeanD-SYD](https://avatars.discourse-cdn.com/v4/letter/j/85e7bf/32.png) [@JeanD-SYD](https://discuss.elastic.co/u/JeanD-SYD)\
**Post date:** [October 17, 2018, 12:13am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/7 "2018-10-17T00:13:05Z")

</div>

Good news

From the ECE the deployment:  
elasticsearch  
[https://261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243](https://261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243)  
kibana  
[https://2f2122e03fe0499db11412a9e0b69b4a.ece-elastic.xyz:9243](https://2f2122e03fe0499db11412a9e0b69b4a.ece-elastic.xyz:9243)

With the above added to the filebeats.yml  
sudo vi /etc/filebeat/filebeat.yml

setup.kibana:  
#host: "localhost:5601"  
host: "[https://2f2122e03fe0499db11412a9e0b69b4a.ece-elastic.xyz:9243](https://2f2122e03fe0499db11412a9e0b69b4a.ece-elastic.xyz:9243)"

output.elasticsearch:  
#Array of hosts to connect to.  
#hosts: ["localhost:9200"]  
hosts: ["261f41b5d7114d2fb96c403bed80c148.ece-elastic.xyz:9243"]  
#Optional protocol and basic auth credentials.  
protocol: "https"  
username: "elastic"  
password: "3VC933BDccAl8Q90iZo0yJg9"

sudo filebeat modules enable system  
sudo filebeat setup --e  
sudo service filebeat start

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 31, 2018, 12:13am UTC](https://discuss.elastic.co/t/ece-deployment-cannot-connect-filebeats-to-kibana/152575/8 "2018-10-31T00:13:07Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
