Unable to enroll fleet server as assinged policy does not have fleet server input

Elastic Stack Beats
1

Hi,

I am setting up fleet-server in air-gap environment. However, I am unable to enroll the fleet-server to elasticsearch. I am having the below error.

2021-07-05T22:37:11.552+0800 INFO cmd/enroll_cmd.go:300 Generating self-signed certificate for Fleet Server
2021-07-05T22:37:12.847+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:18.895+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:24.951+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:31.001+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:37.045+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:43.104+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:44.110+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:37:46.125+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:47.134+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:37:51.171+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:52.172+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:37:58.189+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:59.199+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:05.227+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:07.227+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:08.228+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:14.238+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:15.238+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:21.243+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:23.244+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:24.246+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:30.258+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:31.262+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:37.271+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:39.280+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:40.283+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:46.289+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:47.292+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:53.305+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:54.308+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:55.308+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:56.310+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:39:02.420+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:39:03.420+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:39:06.447+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:39:07.453+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
Error: fleet-server never started by elastic-agent daemon: context canceled
Error: enroll command failed with exit code: 12021-07-05T22:37:11.552+0800 INFO cmd/enroll_cmd.go:300 Generating self-signed certificate for Fleet Server
2021-07-05T22:37:12.847+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:18.895+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:24.951+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:31.001+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:37.045+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:43.104+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Starting
2021-07-05T22:37:44.110+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:37:46.125+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:47.134+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:37:51.171+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:52.172+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:37:58.189+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:37:59.199+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:05.227+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:07.227+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:08.228+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:14.238+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:15.238+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:21.243+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:23.244+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:24.246+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:30.258+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:31.262+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:37.271+0800 INFO cmd/enroll_cmd.go:648 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:39.280+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:40.283+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:46.289+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:47.292+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:53.305+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:54.308+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:38:55.308+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:38:56.310+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:39:02.420+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:39:03.420+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
2021-07-05T22:39:06.447+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Starting
2021-07-05T22:39:07.453+0800 INFO cmd/enroll_cmd.go:643 Fleet Server - Error - assigned policy does not have fleet-server input
Error: fleet-server never started by elastic-agent daemon: context canceled
Error: enroll command failed with exit code: 1

May I know where I can add the server input?

2

I'm not sure if this error is related to the air-gapped environment. Did you run the Package Registry? Is it accessible?

3

Kibana should have added the fleet-server integration to the Default Fleet Server policy. Being air-gapped you will need to run the package registry as well (as @mtojek mentions).

You can manually add the integration by going to the Default Fleet Server policy and add the fleet-server integration.

4

yes. I am running it in an air-gap environment. May I know how I can setup the package registry? The kibana and elasticsearch version i am using is 7.13.2

5

Docs: GitHub - elastic/package-storage: Package storage for packages served through the package registry service

6

I have run the docker run -it docker.elastic.co/package-registry/distribution:production
below is the message generated
...
2021/07/11 17:31:19 zscaler 0.2.0 /packages/production/zscaler/0.2.0
2021/07/11 17:31:22 368 package manifests loaded.

However, I cannot connect to the port
curl -f http://127.0.0.1:8080
curl: (7) Failed to connect to 127.0.0.1 port 8080: Connection refused

May I know what is the error?

7

Hi I have managed to get my elastic package registry up and running. I have also changed the xpack.fleet.registryUrl: "http://package-registry.corp.net:8080" in kibana config. However, I am still getting the following error:
"Error connecting to package registry: request to https://epr.elastic.co/search?package=system&internal=true&experimental=true&kibana.version=7.13.2 failed, reason: getaddrinfo ENOTFOUND epr.elastic.co"

8

below is my kibana configmap file
kibana.yml: |-
server.host: 0.0.0.0
server.ssl.enabled: true
server.ssl.certificate: /usr/share/kibana/config/certs/kibana/tls.crt
server.ssl.key: /usr/share/kibana/config/certs/kibana/tls.key
xpack.encryptedSavedObjects.encryptionKey: ${ELASTICSEARCH_OBJECT_KEY}
xpack.security.enabled: true
xpack.fleet.enabled: true
xpack.fleet.registryUrl: ${REGISTRY_HOSTS}
xpack.fleet.agents.enabled: true
xpack.fleet.agents.tlsCheckDisabled: true
elasticsearch:
hosts: ${ELASTICSEARCH_HOSTS}
username: ${ELASTICSEARCH_USER}
password: ${ELASTICSEARCH_PASSWORD}
ssl:
key: /usr/share/kibana/config/certs/es/tls.key
certificate: /usr/share/kibana/config/certs/es/tls.crt

9

@nchaulet Any ideas on why that setting would not be picked up by Kibana? ^

10

@cheapsupps are you using an enterprise licence of Kibana? currently it's only possible to configure the registry url if you have an enterprise licence.

11

I have the same issue, instead I don't have any access to the internet. Is there a way to curl the content down and move across?

12

@charliek17 The documentation linked below provides information (it was also linked above by @mtojek) on how to do it in environments without the internet.

1 Like
13

Hi, had this issue too, ive open and closed an issue on the github page.

docker run --publish 8080:8080 -it docker.elastic.co/package-registry/distribution:production

That should work

But I'm running into the same issue where regardless of setting regurl it still defaults to epr.elastic.co

1 Like
14

In update. We don't have a Elastic Enterprise to allow Custom EPR Registries. So that's why its not working.

15

I managed to get the EPR registries by enabling trial. I wonder does the integration still works after the trial has expired.

16

Hi, I've encountered the same error message.

2021-08-31T07:13:04.409Z ERROR log/reporter.go:36 2021-08-31T07:13:04Z - message: Application: fleet-server--7.14.0: State changed to FAILED: Error - assigned policy does not have fleet-server input - type: 'ERROR' - sub_type: 'FAILED'

I've added fleet-server integration to Elastic Cloud Agent policy manually. Then after restarting the APM & Fleet server, it started up successfully.

I'm using an air-gapped ECE environment. Added following two settings in kibana.yml:

xpack.ingestManager.registryProxyUrl: http://proxy-host:3128
xpack.fleet.registryProxyUrl: http://proxy-host:3128

I wonder why the agent started by APM & Fleet server uses the Elastic Cloud Agent policy. Default Fleet Server policy should be assigned to the agent instead?

1 Like
17

We suffered this issue as well because we had recently removed "xpack.ingestManager.registryProxyUrl" as it was appearing in the deprecation logs as having been replaced by "xpack.fleet.registryProxyUrl".

Turns out both are needed under the covers, in v7.14.x possibly by accident...

We also discovered if upgrading from v7.13.x to v7.14.x that if the you hit the enrollment issue make sure both the settings are available in kibana ( and that your proxy allows you to connect to the epr.elastic.co address ) that you may also need to run the 3 lines from this " Fleet Server fails with assigned policy does not have fleet-server input log error" from Troubleshoot common problems | Fleet and Elastic Agent Guide [master] | Elastic

1 Like
18

@Jugsofbeer I'm curious, do you have any other configs still under the xpack.ingestManager.* key. I believe we do have a bug where if any configs are still under this old name, it will overwrite any configs under xpack.fleet.* when we translate the old settings over.

I've opened an issue with a test case to track the root problem here, but we may need to workaround this in the meantime on the Fleet side.

19

Hi Josh, I have been working through this with a Elastic Support Case the past week, send me a PM if you want the number.

20

Hi,

notice Elasticsearch v 7.15.0 is out.

Based on below link, is enterprise license still required for the custom epr url?

thanks