Everytime when I start elastic-endpoint.exe it overwrites the file elastic-endpoint.yaml with:
hosts:
- http://localhost:9200
Because of that the Elastic agent is not sending any data to my Elasticsearch cluster that has another IP adres.
I have the same issue, you beat me to the post.
https://discuss.elastic.co/t/missing-elastic-security-and-endpoint-integration-data/246601
I'm confused as to what is wrong.
1 Like
I think you're looking for the "Settings" on the top right of the Management -> Ingest Manager -> Fleet section. It will have a section for input on where the elasticsearch and kibana urls are set for the config of the fleet.
Thanks, I've tried that, the Kibana URL was correct, I've changed the Elasticsearch URL but still no joy.
Hello @Zawadidone
You and @OntheHighSeas have helped identify a bug that we've filed to fix that workflow. @_joel is correct about where the URLs are set, but the bug causes them to not propagate down to the fleet as expected. The issue to fix this here: https://github.com/elastic/kibana/issues/76136
In the meantime in order to update the elastic search address endpoint is using please try the following workaround:
- In ingest manager, under the main settings menu, you can update,add,change the Kibana and Elasticsearch URLs. Click save.
- Afterward, under the Configurations tab of ingest manager, click on the Configuration assigned to the endpoint you want to update.
- On the Configuration page, in the integrations tab, click the actions "..." for the Elastic Endpoint Security integration and select "Edit integration"
- On this next page, click "Save integration" in the bottom right (you do not need to make any changes).
This should trigger an update to the configuration for the agent which will propagate down with the new global settings applies.
Please let me know if this doesn't resolve the issue for you.
-Nick Fritts
Same issue. After following the steps above the agent does change but data is not sent to elastic cluster.
On the workstation side I do see a connection is being established but dies very quickly. From the log
//
{"@timestamp":"2020-08-27T20:06:36.8385000Z","agent":{"id":"removed","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"info","origin":{"file":{"line":1392,"name":"HttpLib.cpp"}}},"message":"HttpLib.cpp:1392 Establishing GET connection to [https://myclustername:9200/_cluster/health]","process":{"pid":4140,"thread":{"id":7080}}}
{"@timestamp":"2020-08-27T20:06:36.55357800Z","agent":{"id":"removed","type":"endpoint"},"ecs":{"version":"1.5.0"},"log":{"level":"notice","origin":{"file":{"line":65,"name":"BulkQueueConsumer.cpp"}}},"message":"BulkQueueConsumer.cpp:65 Elasticsearch connection is down","process":{"pid":4140,"thread":{"id":7080}}}
\
Connection is dead before anything is sent.
Thanks your instruction fixed the issue. Will this also be added to the documentation of the Elastic agent?
@Zawadidone I'm unsure if the steps I listed will be added to the documentation. We consider it a bug that you have to go through those steps. If you change the Elasticsearch URL, it should automatically update the Endpoint. My guess is that it will be patched, but I'll ask around to see if people want to document the work around in the mean time.
-nf
Are those log entries in the same order they appeared in the log? Could you provide any additional log entries?
Are you using self signed SSL certificates? There's a separate issue with self signed SSL certificates currently that could be causing what you're seeing.
-nf
That is copy/paste in order from the log file. The time stamps seemed off to me as well. Time stamps are only in order by hour/min/sec as soon as they hit ms they are off. Might be a little better to slow the log writes down or you'll be killing some ssd drives.
Cert is signed by our internal CA. Cert has client/server auth as a usage as well. Had to double check to make sure it wasn't trying to hit a cert that was server auth only.
Could you check to make sure that your internal CA is added to the computer trusted root certificate store (not just the user store)?
Trusted root and intermediate are in the correct location under system. Checked to make sure my deltas where being updated on my ca as well just because I haven't in longer then I should admit...
Tested with adding to cert in user store for giggles and still no data. "EDIT: Removed from user"
What I have noticed "at least on 2 clusters" is if I install the agent with a custom profile no logs are shipped this includes elastic.agent.metricbeat, elastic.agent.filebeat and elastic.agent.endpont-security. If I leave it at default I do get metricbeat and filebeat. Updating the agent from the gui after the install will continue to send. I do not how ever get endpoint-security no matter what I do.
I have not added endpoint-security to the default integration to see if that will work yet.
EDIT:
Elastic-Endpoint shows up in the streaming logs after installing in default after adding Endpoint to the integration. " I do believe this is in the documentation about not supporting custom ones yet" Moving to a policy afterwards data still shows up. Now the fun testing Endpoint and seeing how to view the detection's in dashboards and SIEM. And re-writing the install and uninstall scripts. Mostly the uninstall as it leaves Elastic-Endpoint installed after you run it. Install script to avoid having techs install to the wrong location a copy to Elastic folder should be in place vs installing from the current working directory. That's purely a use case for my environment and may not be yours.
Just saw your edit, I'm glad to see that you got it working. Can you give me more details on your uninstall process?
The endpoint should not remain installed after performing the agent uninstall. It may just be an artifact of your use-case as you said, but I'd like to confirm that if possible. If something isn't working as expected I want to make sure an issue is opened to track getting it fixed.
-nf
@NickFritts
To not highjack the thread and for future searches see here: " Uninstall/Install Elastic-Endpoint. Endpoint stays after uninstall "
1 Like
Mine still doesn't work after touching the configurations. The URLs do update.
I'm looking at the elastic-agent-json.log but can't see any errors which would indicate something incorrect.
Any suggestions, thanks
Yes I'm using self-signed certs, and still struggling.
@OntheHighSeas lets use the post you started to discuss further: Missing Elastic Security and endpoint integration data
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.