I created a new 7.9 installation on my lab environment.
I tried to create apikeys on APM Server, using documented command, but I got a couple of problems:
apm-server now runs under apm-server user, and launch apikey commands under root gives error (I needed to switch user to apm-server. That's ok, I suggest to align documentation)
The commands give me the following ouptut, under every condition (apm-server up/down, elasticsearch server reachable/not reachable)
-bash-4.2$ apm-server apikey create -e -v --ingest --agent-config --name java-002
2020-08-27T15:43:47.214Z INFO instance/beat.go:640 Home path: [/usr/share/apm-server] Config path: [/etc/apm-server] Data path: [/var/lib/apm-server] Logs path: [/var/log/apm-server]
2020-08-27T15:43:47.214Z INFO instance/beat.go:648 Beat ID: 2f71fc0a-f7a0-4a40-ac46-e556fa186810
2020-08-27T15:43:47.215Z INFO [config] config/api_key.go:50 Falling back to elasticsearch output for API Key usage
Is that the complete output? I just tested with 7.9.0 and it worked for me. I get that output, and then shortly after:
API Key created:
Name ........... java-002
Expiration ..... never
Id ............. <ID>
API Key ........ <APIKey> (won't be shown again)
Credentials .... <Credentials> (use it as "Authorization: APIKey <credentials>" header to communicate with APM Server, won't be shown again)
Are you not seeing any further output? No errors? Does the command even complete?
No further output, and the command completes immediately
I'm using Vagrant with CentOs 8.
Is it correct to execute it with apm-server user (and not root)?
Running with apm-server user is fine; what is important is that the user configured for the ES connection has the required application privileges, described in creating an API Key via APM Server.
Hi @simitt,
I reconfigured everything.
I'm able to see a go apm-agent on Kibana, but when I submit the api key command, I have: dial tcp 127.0.0.1:9200: connect: connection refused
apm-server.yml has a different output.elasticsearch.hosts configuration (see my first post), so I don't understand why it tries to connect to localhost.
It sounds like apm-server.yml isn't being picked up. Does apm-server export config show the same output.elasticsearch configuration as in your config file?
Hi @axw,
I tested more carefully:
If apm-server.api_key is set to false I have the tcp error.
Otherwise, if it is set to true, I got success as in my first post.
Below the export config, with the correct elasticsearch host.
I take advantage of your help and kindness.
Is there a tutorial with instructions on how to create valid apmagent key using elasticsearch rest api? (so, without using apmserver api/command line at all)
Thank you
$ vagrant ssh -- sudo -u apm-server apm-server test output
elasticsearch: http://192.168.121.1:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 192.168.121.1
dial up... OK
TLS... WARN secure connection disabled
talk to server... OK
version: 7.9.0
"apm-server apikey create" works:
$ vagrant ssh -- sudo -u apm-server apm-server apikey create -e -v --ingest --agent-config --name java-002
2020-09-02T02:24:46.032Z INFO instance/beat.go:640 Home path: [/usr/share/apm-server] Config path: [/etc/apm-server] Data path: [/var/lib/apm-server] Logs path: [/var/log/apm-server]
2020-09-02T02:24:46.032Z INFO instance/beat.go:648 Beat ID: a11d18fb-1631-4e91-b000-d9b0c8a9a652
2020-09-02T02:24:46.033Z INFO [config] config/api_key.go:50 Falling back to elasticsearch output for API Key usage
API Key created:
Name ........... java-002
Expiration ..... never
Id ............. geehTHQBLIloR_Qkfysa
API Key ........ 31Ot-6i_QByrw_deFfjPpA (won't be shown again)
Credentials .... Z2VlaFRIUUJMSWxvUl9Ra2Z5c2E6MzFPdC02aV9RQnlyd19kZUZmalBwQQ== (use it as "Authorization: APIKey <credentials>" header to communicate with APM Server, won't be shown again)
Hi @axw,
I resolved the issue.
I had an incorrect proxy configuration with no_proxy for "192.168.*", but not for "*.elk.svi".
The error was quite subtle, since o.s. ping worked correctly, and apmserver too!
Thank you for helping me in the issue, since I discovered the problem using the test output command (see incorrect output below).
Anyway, is there a way to create the couple Api Key:Credentials directly using Elasticseatch Rest Api, and not apmserver?
[vagrant@apmserver ~]$ sudo -u apm-server apm-server test output -v -e -d "*"
2020-09-02T08:19:07.470Z INFO instance/beat.go:640 Home path: [/usr /share/apm-server] Config path: [/etc/apm-server] Data path: [/var/lib/apm-serve r] Logs path: [/var/log/apm-server]
2020-09-02T08:19:07.470Z DEBUG [beat] instance/beat.go:692 Beat met adata path: /var/lib/apm-server/meta.json
2020-09-02T08:19:07.470Z INFO instance/beat.go:648 Beat ID: 344fff2 4-8338-4285-9377-ca4ecefbaeec
2020-09-02T08:19:07.470Z INFO [index-management] idxmgmt/std.go:1 84 Set output.elasticsearch.index to 'apm-server-7.9.0' as ILM is enabled.
2020-09-02T08:19:07.471Z DEBUG [tls] tlscommon/tls.go:155 tls%!(EX TRA string=successfully loaded CA certificate: %v, string=/etc/apm-server/certs/ ca.pem)
2020-09-02T08:19:07.471Z INFO eslegclient/connection.go:99 elastics earch url: https://esnode.elk.svi:9200
elasticsearch: https://esnode.elk.svi:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 192.168.9.90
dial up... OK
TLS...
security: server's certificate chain verification is enabled
handshake... OK
TLS version: TLSv1.3
dial up... OK
2020-09-02T08:19:18.537Z DEBUG [esclientleg] eslegclient/connection.g o:290 ES Ping(url=https://esnode.elk.svi:9200)
2020-09-02T08:19:18.545Z DEBUG [esclientleg] eslegclient/connection.g o:294 Ping request failed with: Get https://esnode.elk.svi:9200: Success
talk to server... ERROR Get https://esnode.elk.svi:9200: Success
[vagrant@apmserver ~]$ ping esnode1.elk.svi
PING esnode1.elk.svi (192.168.9.90) 56(84) bytes of data.
64 bytes from esnode1.elk.svi (192.168.9.90): icmp_seq=1 ttl=64 time=0.898 ms
64 bytes from esnode1.elk.svi (192.168.9.90): icmp_seq=2 ttl=64 time=0.787 ms
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.