Elastic Agent - problems following the documentation, stumbled upon dead links and more

In the documentation for setting up Elastic Agent the following is stated:

Kibana user with All privileges on Fleet and Integrations. Since many Integrations assets are shared across spaces, users need the Kibana privileges in all spaces.

But when I select the user kibana, it shows that kibana is deprecated - that's fine, I check kibana_system, but I can't edit anything with it because it's a built-in user... I tried ignoring that thinking it's outdated and carried on... (please note that I did that AFTER I did the paragraph below)

I installed the Elastic Agent and started it with systemctl start elastic-agent and enabled it on boot with systemctl start elastic-agent.

But then I noticed the included documentation of Elastic Agent with a dead link... I tried stopping and disabling the Elastic Agent and I succeeded after a surprisinging big time leap. Then I followed the included README.md and ran

./elastic-agent -c elastic-agent.yml -e

which gave me the following output:

Error: could not read configuration file /var/lib/elastic-agent/elastic-agent.yml: open /var/lib/elastic-agent/elastic-agent.yml: no such file or directory
For help, please see our troubleshooting guide at https://www.elastic.co/guide/en/fleet/8.5/fleet-troubleshooting.html

Next I'm supposed to run this:
./elastic-agent setup -e

...but it doesn't make sense, since the config for Elastic Agent is clearly missing.

  • How do I solve these issues?..

My goal is to setup Elastic Agent with sample data for simple SIEM analysis (I'm still a beginner).

Hi @SirMuffington

Can you show us the document you're referring to?

Are you trying to install elastic agent as standalone or fleet managed?

The file is located in /usr/share/elastic-agent/README.md from the newest Elastic Agent registration. It was installed from a DEB package (running on Debian 11). The file's contents are:

# Welcome to Elastic-Agent 8.5.1

Agent manages other beats based on configuration provided.

## Getting Started

To get started with Elastic-Agent, you need to set up Elasticsearch on
your localhost first. After that, start Elastic-Agent with:

     ./elastic-agent -c elastic-agent.yml -e

This will start Elastic-Agent and send the data to your Elasticsearch
instance. To load the dashboards for Elastic-Agent into Kibana, run:

    ./elastic-agent setup -e

For further steps visit the
[Quick start](https://www.elastic.co/guide/en/beats/elastic-agent/main/elastic-agent-installation-configuration.html) guide.

## Documentation

Visit [Elastic.co Docs](https://www.elastic.co/guide/en/beats/elastic-agent/main/index.html)
for the full Elastic-Agent documentation.

## Release notes


I found the config in /etc/elastic-agent/elastic-agent.yml though. I guess that's my ticket to success?

I am trying to install it as fleet managed.

I would not use that Read Me (It looks to be more of a leftover from Beats)

That is not referring to Fleet managed

I would use the official documentation that I linked you to... That is why I provided it.

There's very easy steps to get fleet managed agent.

In fact, if you log into Kibana into Fleet, there are step by step instructions.

So I don't even need to grant these rights as told in the docs everywhere (including the link you sent in here)?:

A Kibana user with All privileges on Fleet and Integrations. Since many Integrations assets are shared across spaces, users need the Kibana privileges in all spaces.

I am just saying follow the documentation on our site here not the readme file.

With respect to the Role Based Control... you can start out using the default elastic user which has that role /privilege and it should all work or you can set up your own users and roles following the documentation that is up to you and then yes the user would need a role as described.

Alright, thank you for this information.

elastic works fine for now.

The problem I am now facing is that Installed as a system package, installation will not be altered. gets printed out when I try to run all of the steps in the setup of the Fleet Server, which apparently needs to be setup by me. I did apt-get remove elastic-agent because I used a DEB package beforehand and then executed the elastic-agent from the tar and then this got printed out and it got stuck:

{"log.level":"info","@timestamp":"2022-11-22T21:50:28.912Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":403},"message":"Generating self-signed certificate for Fleet Server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-22T21:50:31.313Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":792},"message":"Fleet Server - Starting","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-22T21:50:35.315Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":792},"message":"Fleet Server - Error - EOF","ecs.version":"1.6.0"}

Why is it indicating an end of file?

Hi @SirMuffington

Apologies I am not following I think we need to back up a bit I may have made an assumption... I did not realize you are actually trying to install the Fleet Server itself... which is a particular policy / type of elastic agent ... I thought you were just trying to install the normal agent to collect / logs metrics etc. after you already. had a fleet server...

Installing the Fleet server itself has a specific set of install instructions.

So a couple questions to confirm ...

What does that mean.....

Do you have a self managed Elasticsearch and Kibana Running with Authentication and TLS enabled and the version is 8.5.1? i.e. all the prerequisites? here

And next you want to install a Fleet Server (elastic-agent with the fleet server policy? Is that what you are trying to accomplish?

I just followed the in the Docs below and in Kibana and installed and connected the fleet server immediately took about 5 mins...

If you want to install your own fleet server you need to follow the instructions here

Are those the instructions you followed? Did you use the Quickstart or Advanced ... notice that it show installing with the tar.gz and if you follow the steps it should give the correct commands to run.

So to help I will need to know what you have exactly what steps you followed.

Here is what I did .. I just copied the commands from Kibana then ran them and it installed and connected. It will not like it if you already have an agent running you will want to uninstall that and install with this command (you can then later add other policies to this agent)

Note this is on an Elasticsarch / Kibana 8.5.2 on Ubuntu 20.04 , Default Setup with Self Signed Certs

curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.5.2-linux-x86_64.tar.gz
tar xzvf elastic-agent-8.5.2-linux-x86_64.tar.gz
cd elastic-agent-8.5.2-linux-x86_64
sudo ./elastic-agent install \
  --fleet-server-es= \
  --fleet-server-service-token=AdsfgdsfgdsfgdsfgdsfgZXQtc2VydmVyL3Rva2VuLTE2NjkxNjM2NDk0Njg6Tks2bmpwN0ZTWmk5N3dBYm04eG0wZw \
  --fleet-server-policy=fleet-server-policy \
Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:Y
{"log.level":"info","@timestamp":"2022-11-23T00:37:04.252Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":403},"message":"Generating self-signed certificate for Fleet Server","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-23T00:37:06.792Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":792},"message":"Fleet Server - Starting","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-23T00:37:10.795Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":773},"message":"Fleet Server - Running on policy with Fleet Server integration: fleet-server-policy; missing config fleet.agent.id (expected during bootstrap process)","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-23T00:37:11.204Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":471},"message":"Starting enrollment to URL: https://stephenb-es-8-test:8220/","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-23T00:37:12.559Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":273},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.

I did not get any messages about generating self signed cert...

So let me know the exact steps you are taking, show the exact commands and what is not working... and perhaps me or someone else can help