Elastic-agent.exe not running on target

Elastic Security
1

I have set up a virtual machine to install McAfee ePO and solidcore for testing. So I am taking the opportunity to connect my VM to the Elastic stack. I copied over the Agent and the agent.yml file. Made the changes to ensure the IP and port are correct. Made sure that all the firewalls were open in terms of Ports and that it was set for bi-directional to avoid having to worry about that.

Now that I am trying to run the elastic-agent.exe on its own and with the whole string that it says to use. It will not install. With the string when I enter it in Powershell or command prompt (as administrator) it errors out with not knowing what the -f is (.\elastic-agent.exe install -f --url=https://ip address:8220 --

It didn't show up in the Kibana interface so I am pretty sure it didn't install.

Where should I start for working on this?

2

Hi, @MKirby Thanks for testing it and reaching out. the product has a Troubleshooting guide here:

As well as a FAQ besides, please review and we're always open to feedback on them to help future users get stated faster.

for your case, do you mind sharing what version you are using? And I'd like to know if you are using the stand-alone Agent or attempting to enroll it in Fleet?

If you are not using the 7.13 release, please do check the guides / info for the prior releases as the architecture (and start up) has changed significantly between 7.12 to 7.13.

If Agent didn't start with the -f paramater, you can omit it. it saves you one prompt in the command line, and if it fails again you may get a more detailed error message. Please do post screenshots or text copy of what you're seeing!

I can offer up, too, that if you are using stand-alone mode the Docs should help elaborate the configuration a lot, and which ports to use to cite creds and get the Agent's sub-processes access to post to ES. This is just a guess as you reference copying over specific single files, which is not recommended if you are NOT using stand-alone mode.

Happy to try to help more, cheers.

3

Thank you for the assistance. I was able to get the agent installed on the "host" server that is running my McAfee security suite. It turns out that the biggest issue was that I had it configured to run an IPV6 not v4. Once I disabled the IPV6 the agent installed.

In answer to your other questions I seem to be running a mix between 7-12 and 7-13. The only beats that are the 7-13 version are: WinlogBeat which is at 7-13.0, Auditbeat 7-13.2. The other beats I have installed are FileBeat 7-12.1, and MetricBeat 7-12.1. My ElasticAgent is 7-13.1. The trunk of the ELK stack I am running Logstash, ElasticSearch and Kibana are all at 7-12.1

I am now waiting for the communication to start so that my McAfee server will show up in the Kibana console.

4

Glad you are making progress, and happy to help. I think, if you are running the 7.12.x ELK Stack, you may run into a problem with the 7.13 Elastic Agent. There were breaking changes between 7.12 and 7.13 with how Agent runs and its requirements (a new 'stack' component called the Fleet Server is required and is expected by a running 7.13 Agent). I think the other beats in place are ok with the 7.13 agent on the same host, but you'll need to update the stack to get that version of Agent working - you could use 7.12.x Agent I expect (latest patch release).

Let us know if you want any more help or want to share the success, we appreciate the testing and feedback.

5

Thank you for all the advice it is really helping.
In order to upgrade from the current version of ELK that I am running, can I simply upgrade from the Kibana UI when I log on, or do I need to download the ES, LS and Kibana, then re-install?

6

Good Morning;

Taking your advice Eric and I am trying to upgrade the ELK stack to 7-13. Following the steps it suggests that I backup (take a snapshot) before going ahead. I understand that and even though this is a test for me, it is a good process to get used to. When I attempt to run the snapshot and add the code to the Dev Tools console

PUT /_snapshot/my_backup
{
 "type": "fs"
 "settings": {
   "location": "d:\Elastic\snapshots"
  }
}

I get an error message indicating that the root cause is
`Preformatted textUnrecognized character escape 'e' (code 101)\n at [Source: (org.elasticsearch-common.io.stream.InputStreamStreamInput); line: 4, column: 22]"'

I have looked through the elastic search logs and configs but am unsure where I would need to add an API to point to a snapshot storage location.

7

Thanks MKirby, we're taking a look to see if we can easily reproduce this on our side to help the team fix it. I'll make sure we capture it in a ticket, though it is beyond my expertise to help much.

If you are ok to skip that part of the upgrade process for the sake of reviewing Agent install it may allow progress. Sorry for the issues, and let us know! I'll post back about any work-arounds to the above that I find, there may be more on it on-line that I haven't dug into yet.

8

Thank you for getting back to me on this. I did some more research and went through some of the other discussions that have taken place on the forum. I found one that indicated I needed to setup the backup in the ElasticSearch.yml file and then run the command without specifying exactly where it was saving the snapshot. When I did that I was able to create a snapshot and run the Put command. However, I am still unsure where the snapshot went. If there is a way that I don't know to specify where that snapshot goes it would be very helpful.

I did manage to upgrade the ElasticSearch and Kibana. My next step is to upgrade the Logstash as the Beats i am using are already at the 7.13.2 level.

Thank you for looking into this

9

Hi again. I'm going to look into the Documentation for that stack upgrade support and see if we can make it better. I don't know the ES side API off the top of my head, but can look it up for us if you don't find it first!

10

Thank you for continuing your research on this.

As a note, even after the upgrade I am still unable to see my ePO server (new Windows virtual machine) in the list of Hosts. My next step is to upgrade its agent to the 7.13.4 and add the config from my SIEM to see what happens.

I will post if it works.

11

HI hi. If you wanted to post a [redacted] config config and/or the string used on the command line to install or enroll the Agent it could help us review, if needed.

12

Hello Eric;
I have tried it on two different virtual machines. Both of which are able to ping back to the Host_Elk_box (the system that is hosting the two virtual machines.) I know that the Windows 2016 VM can connect to my ELK stack as it is running a McAfee agent and is communicating and recieving new virus definitions daily.
I have tried two means of connecting the Elastic Agent

  1. from the directory it is loaded I have run elastic-agent install -- Choose not to enroll and the Agent installs, but doesn't connect back to the ELK Stack. In the agent.yml file under Output I have used the following:
outputs:
  default:
    type: elasticsearch
    hosts:
      - 'http://192.168.56.4:9200'
    username: elastic
    password: Itsmikekirby

This is a test environment so username and IP's don't need to be hidden. I can destroy and rebuild as needed

On my CentOS7, I have run out of space on the hdd so I need to tear it down and rebuild with more space to test properly.

13

Eric;
I forgot to mention that when I do attempt to enroll in fleet I receive the following:

C:\elkagent\7.13.4\elastic-agent-7.13.4-windows-x86_64>elastic-agent.exe install -f --fleet-server-es=http://192.168.2.34:9200 --fleet-server-service-token=AAEAAWVsYXN0aWMvlZXQtc2VydmVyL3Rva2VuLTE2Mjc1NzE3ODkzMTY6a2xOSTR1RXBSsmFhc2RzMmNjNVNhQQVsYXN0aWMvZxlZXQtc2VydmVyL3Rva2VuLTE2Mjc1NzE3ODkzMTY6a2XOSTR1RXBSamFhc2RzMmNjNVNhQQ
The Elastic Agent is currently in BETA and should not be used in production

2021-07-29T12:04:14.815-0400    INFO    cmd/enroll_cmd.go:302   Generating self-signed certificate for Fleet Server
2021-07-29T12:04:24.121-0400    INFO    cmd/enroll_cmd.go:613   Waiting for Elastic Agent to start Fleet Server
2021-07-29T12:04:30.197-0400    INFO    cmd/enroll_cmd.go:618   Waiting for Elastic Agent to start Fleet Server: no fleet-server application running
2021-07-29T12:04:36.309-0400    INFO    cmd/enroll_cmd.go:618   Waiting for Elastic Agent to start Fleet Server: no fleet-server application running
2021-07-29T12:04:42.418-0400    INFO    cmd/enroll_cmd.go:618   Waiting for Elastic Agent to start Fleet Server: no fleet-server application running
2021-07-29T12:04:48.586-0400    INFO    cmd/enroll_cmd.go:618   Waiting for Elastic Agent to start Fleet Server: no fleet-server application running
2021-07-29T12:04:52.637-0400    INFO    cmd/enroll_cmd.go:596   Waiting for Elastic Agent to start
2021-07-29T12:04:57.604-0400    INFO    cmd/enroll_cmd.go:646   Fleet Server - Starting
2021-07-29T12:04:59.682-0400    INFO    cmd/enroll_cmd.go:646   Fleet Server - Error - dial tcp 192.168.2.34:9200: connectex: A socket operation was attempted to an unreachable network.
2021-07-29T12:05:05.807-0400    INFO    cmd/enroll_cmd.go:651   Fleet Server - Error - dial tcp 192.168.2.34:9200: connectex: A socket operation was attempted to an unreachable network.
2021-07-29T12:05:09.854-0400    INFO    cmd/enroll_cmd.go:646   Fleet Server - Restarting
2021-07-29T12:05:10.871-0400    INFO    cmd/enroll_cmd.go:646   Fleet Server - Error - dial tcp 192.168.2.34:9200: connectex: A socket operation was attempted to an unreachable network.
2021-07-29T12:05:17.073-0400    INFO    cmd/enroll_cmd.go:651   Fleet Server - Error - dial tcp 192.168.2.34:9200: connectex: A socket operation was attempted to an unreachable network.

thanks for looking and assisting.

14

Disclaimer: I have no experience running Fleet-Server on Windows. Only on Linux. Agents are Windows are my primary.

Add ( --insecure ) behind your connection when you do the agent install. I'm going to guess that dev lab isn't running full SSL setup. Noticed in my case in 7.13.3 and .4 that the agent is not correctly pulling from the Windows certificate store so you will need to included the CA file on each agent and reference it in the Fleet settings. For example in Fleet Settings under Elasticsearch output configuration (YAML) at the following line:
ssl.certificate_authorities: ["C:/Program Files/Elastic/Agent/ca.crt"]

ElasticFleetSettings

"Please note this is only really helpful if your agents are Windows based as that path is useless in Linux/Mac."

Change to the CA chain you use. Copy your CA or the public cert of your dev elastic node and name it ca.crt. This will correct 95% of your future issues.

Make sure the firewall on the elastic box has port 8220/TCP "Fleet-Server" and 9200/TCP.

The changes between 7.12 agent and 7.13 are not compatible. 7.12 Elastic-Agent will not work with 7.13 Fleet server. It's a massive change between the two and for the better! One thing to note that is not documented is go back and check for old inject pipelines and remove them when all your agents are updated. This bite me hard very hard.

Sanity check for me. Are you using Fleet-Server on Windows "That's what it looks like" or on Cent 7? Is Elastic on Windows or Linux?

15

@MKirby HI - I am hoping you got some more success with Agent. I think the post above has good merit in reviewing, if you want to post back the noted details. Best regards.

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.