# Auditbeat vs testing ES output **URL:** \ **Category:** Beats\ **Tags:** auditbeat\ **Created:** [February 27, 2020, 2:24pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252 "2020-02-27T14:24:15Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [February 27, 2020, 2:24pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/1 "2020-02-27T14:24:16Z")
trying to test auditbeat connectivity to elastic, connectivity seems fine only service is unavailable. Wondering where I possible could see the RC, tried ingesting nodes' elasticsearch + audit logs, beat log, but nothing seems logged regarding this and tcp/ssldump is no good as I'm using https. Appreciate any hints, TIA! ``` # /usr/share/auditbeat/bin/auditbeat test output -c /etc/auditbeat/auditbeat.yml elasticsearch: https://... parse url... OK connection... parse host... OK dns lookup... OK addresses: dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.3 dial up... OK talk to server... ERROR Get https://: Service Unavailable ``` ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [February 28, 2020, 8:30am UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/2 "2020-02-28T08:30:02Z")
Wondering if would be an auditing issue... though I'm confident I'm using properly credentials, but maybe I should turn on auditing event logging, as I read from doc: `xpack.security.audit.enabled` Set to `true` to enable auditing on the node. The default value is `false` . This puts the auditing events in a dedicated file named `_audit.json` on each node. ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [February 28, 2020, 10:13am UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/3 "2020-02-28T10:13:20Z")
Increased beat's log level from warning to debug and just gets this: ``` 2020-02-28T11:07:42.651+0100 DEBUG [keystore] keystore/keystore.go:125 accessing key 'INGEST_PROTOCOL' from the keystore 2020-02-28T11:07:42.651+0100 DEBUG [keystore] keystore/keystore.go:125 accessing key 'INGEST_USER' from the keystore 2020-02-28T11:07:42.651+0100 DEBUG [keystore] keystore/keystore.go:125 accessing key 'INGEST_PWD' from the keystore 2020-02-28T11:07:42.652+0100 DEBUG [keystore] keystore/keystore.go:125 accessing key 'INGEST_URL' from the keystore 2020-02-28T11:07:42.652+0100 INFO elasticsearch/client.go:174 Elasticsearch url: https:// elasticsearch: https://... parse url... OK connection... parse host... OK dns lookup... OK addresses: 62.243.41.249 dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.3 dial up... OK 2020-02-28T11:07:55.188+0100 DEBUG [elasticsearch] elasticsearch/client.go:733 ES Ping(url=https://) 2020-02-28T11:07:55.196+0100 DEBUG [elasticsearch] elasticsearch/client.go:737 Ping request failed with: Get https://: Service Unavailable talk to server... ERROR Get https://: Service Unavailable ``` ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 3, 2020, 7:53am UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/4 "2020-03-03T07:53:43Z")
Anyone? 'Service unavailable' wouldn't that possible imply some kind of permission issue on elastic side, and if how to get details logged somehow? I'm puzzled by this, user attempted with has a role which got create\_index,index,write on auditbeat-\* same role which work for another user working for Windows auditbeats. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 3, 2020, 7:05pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/5 "2020-03-03T19:05:44Z")
Hey @stefws, "Service unavailable" seems like an error on the server side, it doesn't look like a permissions issue. Could you check the Elasticsearch logs? ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 3, 2020, 7:57pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/6 "2020-03-03T19:57:04Z")
Thanks, already tried ingesting nodes' elasticsearch + audit logs, but nothing in these, wondering if I should enable auditing logs as I think it mighty be a permission issue for the user... ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 3, 2020, 8:20pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/7 "2020-03-03T20:20:59Z")
🤔 I would expect a different error if the user wouldn't have enough privileges. Do you have any proxy between auditbeat and elasticsearch? ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 4, 2020, 6:58am UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/8 "2020-03-04T06:58:38Z")
Yes a HAproxy, but got +350 Windows using the same LB VIP through same HAproxy-\>ES ingest nodes without any issue. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 4, 2020, 2:16pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/9 "2020-03-04T14:16:26Z")
Can you see anything in the haproxy access logs? Maybe it is the one generating this "Service unavailable" errors? ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 5, 2020, 11:52am UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/10 "2020-03-05T11:52:21Z")
No nothing in HAproxy log, also got +350 Windows Computer running both Winlog+Audit beats through same HAproxy just fine, so it's seems weird... ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 5, 2020, 12:08pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/11 "2020-03-05T12:08:48Z")
Do you mean that normal ingestion works, but `auditbeat test output` fails? ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 5, 2020, 12:16pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/12 "2020-03-05T12:16:38Z")
Normal ingestion from other [windows] clients works through the same HAproxy, now I'm trying from a Linux client which fails. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 5, 2020, 12:17pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/13 "2020-03-05T12:17:43Z")
Are both Windows and Linux machines using the same Beats versions and output configs? ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 5, 2020, 12:34pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/14 "2020-03-05T12:34:36Z")
Same beat versions 7.6.0 and same output config except with different credentials (as I have forgot the password using in Windows keystores). Keystores doesn't seem compatible among Wintel/Lintel clients 🙂 So I created another User with same roles as the working Wintel clients' User and entered this in a Lintel keystore. If I hardwire known wrong credentials in yml' output section, I get the same error, so I think it's some kind on authentication error from Elastic. Also tried with known 'Super User' credentials hardwired into output, but gives the same error with Service Unavailable. ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 5, 2020, 1:13pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/15 "2020-03-05T13:13:04Z")
Found the Wintel credential, but even that isn't wokring on my Lintel client. If I enter bad password in Wintel I get this error: ``` talk to server... ERROR 401 Unauthorized: ``` But not on my Lintel, hm wondering why, netstat shows an established tcp socket to expected destination as also indicated by test output, so what may hinder the 'ping' test to fail with 'Service Unavailable' if it's not a authentication failure... ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 5, 2020, 1:25pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/16 "2020-03-05T13:25:54Z")
😉 but running auditbeat it sends data just fine, so is it only the test output that fails... ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 5, 2020, 3:34pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/17 "2020-03-05T15:34:40Z")
This 401 error is what I would expect if the problem were in the credentials used. `Service Unavailable` looks like a network or server-side problem, in Elasticsearch, or in Haproxy. Would it be possible to try to do the output test directly from auditbeat to Elasticsearch? So we can discard some problem with the connectivity through haproxy. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 5, 2020, 3:35pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/18 "2020-03-05T15:35:44Z")
> [@stefws](#): > > Also tried with known 'Super User' credentials hardwired into output, but gives the same error with Service Unavailable. This also points to some problem in the connectivity from auditbeat to Elasticsearch through haproxy. ---
**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\ **Post date:** [March 5, 2020, 4:03pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/19 "2020-03-05T16:03:05Z")
No no, if I actually just starts the Auditbeat service on my Lintel box then it actually ships data just fine to Elastic through HAproxy like my Wintel clients do, even that 'test output' says 'Service Unavailable', very weird. What does test output ping actually do compared to sending data that might be different? ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [March 5, 2020, 4:11pm UTC](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252/20 "2020-03-05T16:11:02Z")
Oh ok, that's interesting, because actually any connection does the same initial ping test, also when connecting to ship data. There might be some difference in the way Beats sets up the connection when testing, but this is going to require further investigation. Could you share your output configuration? In any case it'd be good if you could make the test with the same configuration but directly to Elasticsearch, without Haproxy. [Next page](https://discuss.elastic.co/t/auditbeat-vs-testing-es-output/221252.md?page=2)