Read timeout warning when Follow referrals is true

jfreeman · March 13, 2018, 10:13pm

First time AD authentication using https://elasticsearchurl/_xpack/security/_authenticate takes around 3 mins to return a successful result. I am using x-pack 6.0 and below is the realms setting in elasticsearch.yml file

xpack:
  security:
    authc:
      realms:
        active_directory:
          type: "active_directory"
          bind_dn: user@company.com
          bind_password: mypassword
          order: 0
          domain_name: corp.company.com
          follow_referrals: true
          timeout.tcp_read: 25s
          timeout.tcp_connect: 25s
          timeout.ldap_search: 25s
          url: "ldaps://corp.company.com:3211"
          ssl:
            certificate_authorities: ["/etc/elasticsearch/certs/ca.pem"]
            verification_mode: none
          user_search:
            base_dn: "OU=UserAccounts,DC=domain,DC=corp,DC=company,DC=com"
          group_search:
            base_dn: "DC=domain,DC=corp,DC=company,DC=com"
          files:
            role_mapping: "/etc/elasticsearch/role_mapping.yml"
          unmapped_groups_as_roles: false

Below is the log file - Detailed error log removed for brevity

    [2018-03-13T21:43:40,524][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://ForestDnsZones.corp.company.com/DC=ForestDnsZones,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server ForestDnsZones.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
Caused by: java.net.SocketTimeoutException: Read timed out
	... 12 more
[2018-03-13T21:44:26,725][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://DomainDnsZones.corp.company.com/DC=DomainDnsZones,DC=corp,DC=company,DC=com]
	... 12 more
[2018-03-13T21:45:57,211][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://DomainDnsZones.redmond.corp.company.com/DC=DomainDnsZones,DC=redmond,DC=corp,DC=company,DC=com]
	... 12 more
[2018-03-13T21:46:27,448][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region1.corp.company.com/DC=region1,DC=corp,DC=company,DC=com]
	... 12 more
[2018-03-13T21:47:00,697][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region2.corp.company.com/DC=region2,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server region2.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
	... 12 more
[2018-03-13T21:48:16,366][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region3.corp.company.com/DC=region3,DC=corp,DC=company,DC=com]
... 12 more
[2018-03-13T21:48:33,306][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region4.corp.company.com/DC=region4,DC=corp,DC=company,DC=com]
[2018-03-13T21:48:53,398][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region5.corp.company.com/DC=region5,DC=corp,DC=company,DC=com]
	... 11 more
[2018-03-13T21:49:08,527][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region6.corp.company.com/DC=region6,DC=corp,DC=company,DC=com]
[2018-03-13T21:49:08,659][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region7.corp.company.com/DC=region7,DC=corp,DC=company,DC=com]
[2018-03-13T21:49:23,835][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://region8.corp.company.com/DC=region8,DC=corp,DC=company,DC=com]

If I set the follow_referrals: false then the authentication fails. LDp.exe authentication response is instant. How to reduce the latency of x-pack AD authentication ?

Yogesh_Gaikwad · March 14, 2018, 2:34am

Hi @jfreeman,

Not sure if you tried to increase the read timeout value if not could please increase it and see if that helps.

The default referral hop limit is five as per unbounded ldap connection options, but I do not see it being into effect. we can see more hops being made than 5. This setting is not configurable as of now, so you might not be able to set it.

Regards,
Yogesh Gaikwad

jfreeman · March 14, 2018, 2:44am

My realm is active_directory and I don't see timeout setting in https://www.elastic.co/guide/en/x-pack/6.0/active-directory-realm.html, however I have still used the timeouts

timeout.tcp_read: 25s
timeout.tcp_connect: 25s
timeout.ldap_search: 25s

which are meant for LDAP Authentication https://www.elastic.co/guide/en/x-pack/6.0/ldap-realm.html and I am not sure if this takes effect.

Is there timeout setting for Active directory user authentication?

Yogesh_Gaikwad · March 14, 2018, 3:04am

hmm my mistake , I read incorrectly as LDAP. Let me go through once again.

Regards,
Yogesh

Yogesh_Gaikwad · March 14, 2018, 3:20am

Hi @jfreeman,

I think the settings for the timeout should take effect for Active Directory as well as LDAP.
Could you try increasing the timeout values and see if it has any effect on the errors that you are getting?

Regards,
Yogesh Gaikwad

TimV · March 14, 2018, 3:45am

My guess is that your AD server is issuing multiple referrals, one (or more) of which is required, and one (or more) is unreachable and causes a long timeout.

We might get to an acceptable situation by fiddling with timeouts, but we're better off finding out what's going on under the covers.

I recommend that you turn on TRACE logging for the LdapUtils class, which will give us an indication of which referrals are happening, which ones return useful results, and which ones fail.

Run the following steps:
(1) Turn on TRACE logging for LDAP

PUT /_cluster/settings
{
    "transient" : {
        "logger.org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils" : "TRACE"
    }
}

(2) Clear the AD realm cache

POST _xpack/security/realm/active_directory/_clear_cache

(3) Test the AD authentication

GET _xpack/security/_authenticate

(4) Turn logging back to INFO

PUT /_cluster/settings
{
    "transient" : {
        "logger.org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils" : "INFO"
    }
}

Then check the elasticsearch log for trace records from LdapUtils.

You should see messages like:

LDAP Search {{request}} => {{result}} ({{entries}})

LDAP referred elsewhere {{request}} => {{referral}}

That should give us a better idea of what's going on, and how we can avoid the problem.

jfreeman · March 14, 2018, 5:46am

Enabled Trace logging and got this response

{
    "acknowledged": true,
    "persistent": {},
    "transient": {
        "logger": {
            "org": {
                "elasticsearch": {
                    "xpack": {
                        "security": {
                            "authc": {
                                "ldap": {
                                    "support": "TRACE"
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

The successful auth response took 4 mins using
GET _xpack/security/_authenticate
and I dont see messages like

LDAP Search {{request}} => {{result}} ({{entries}})
LDAP referred elsewhere {{request}} => {{referral}}

The log message is here - To fit the 7000 character limit I removed duplicate lines and have put the remaining log as new post.

[2018-03-14T04:40:13,661][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://subdomain1.corp.company.com/DC=subdomain1,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server subdomain1.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
	at com.unboundid.ldap.sdk.LDAPConnectionInternals.sendMessage(LDAPConnectionInternals.java:574) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
	at com.unboundid.ldap.sdk.LDAPConnection.sendMessage(LDAPConnection.java:4249) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
	at com.unboundid.ldap.sdk.SimpleBindRequest.process(SimpleBindRequest.java:551) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
	at com.unboundid.ldap.sdk.LDAPConnection.bind(LDAPConnection.java:2143) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
	at com.unboundid.ldap.sdk.LDAPConnection.getReferralConnection(LDAPConnection.java:4573) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
	at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.lambda$followReferral$11(LdapUtils.java:601) ~[x-pack-security-6.2.2.jar:6.2.2]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_151]
	at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.privilegedConnect(LdapUtils.java:87) ~[x-pack-security-6.2.2.jar:6.2.2]
	at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.followReferral(LdapUtils.java:601) ~[x-pack-security-6.2.2.jar:6.2.2]
	at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.access$300(LdapUtils.java:66) ~[x-pack-security-6.2.2.jar:6.2.2]
	at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils$LdapSearchResultListener.searchResultReceived(LdapUtils.java:533) [x-pack-security-6.2.2.jar:6.2.2]
	at com.unboundid.ldap.sdk.AsyncSearchHelper.responseReceived(AsyncSearchHelper.java:240) [unboundid-ldapsdk-3.2.0.jar:3.2.0]
	at com.unboundid.ldap.sdk.LDAPConnectionReader.run(LDAPConnectionReader.java:569) [unboundid-ldapsdk-3.2.0.jar:3.2.0]
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_151]
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_151]
	at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_151]
	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_151]
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) ~[?:?]
	at sun.security.ssl.InputRecord.read(InputRecord.java:503) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:757) ~[?:?]
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[?:?]
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[?:1.8.0_151]
	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[?:1.8.0_151]
	at com.unboundid.ldap.sdk.LDAPConnectionInternals.sendMessage(LDAPConnectionInternals.java:543) ~[?:?]
	... 12 more

jfreeman · March 14, 2018, 5:48am

[2018-03-14T04:41:28,913][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://subdomain2.corp.company.com/DC=subdomain2,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server subdomain2.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
[2018-03-14T04:41:59,331][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://DomainDnsZones.southpacific.corp.company.com/DC=DomainDnsZones,DC=southpacific,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server DomainDnsZones.southpacific.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
[2018-03-14T04:42:01,152][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://subdomain3.corp.company.com/DC=subdomain3,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server subdomain3.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
[2018-03-14T04:43:07,132][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://subdomain4.corp.company.com/DC=subdomain4,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server subdomain4.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
[2018-03-14T04:43:23,898][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://subdomain5.corp.company.com/DC=subdomain5,DC=corp,DC=company,DC=com]

jfreeman · March 14, 2018, 5:49am

[2018-03-14T04:43:44,025][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://Managed.corp.company.com/DC=Managed,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to resolve address 'Managed.corp.company.com':  UnknownHostException(message='Managed.corp.company.com: Name or service not known', trace='lookupAllHostAddr(Inet6AddressImpl.java:native) / lookupAllHostAddr(InetAddress.java:928) / getAddressesFromNameService(InetAddress.java:1323) / getAllByName0(InetAddress.java:1276) / getAllByName(InetAddress.java:1192) / getAllByName(InetAddress.java:1126) / getByName(InetAddress.java:1076) / connect(LDAPConnection.java:750) / connect(LDAPConnection.java:710) / <init>(LDAPConnection.java:534) / getReferralConnection(LDAPConnection.java:4545) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:755) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:710) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnection.<init>(LDAPConnection.java:534) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnection.getReferralConnection(LDAPConnection.java:4545) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
Caused by: java.net.UnknownHostException: Managed.corp.company.com: Name or service not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_151]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[?:1.8.0_151]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_151]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[?:1.8.0_151]
at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:750) ~[?:?]
... 11 more
[2018-03-14T04:43:53,709][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://DomainDnsZones.subdomain6.corp.company.com/DC=DomainDnsZones,DC=subdomain6,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server DomainDnsZones.subdomain6.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
[2018-03-14T04:44:29,196][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://exchange.corp.company.com/DC=Exchange,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to send the LDAP message to server exchange.corp.company.com:636:  SocketTimeoutException(message='Read timed out', trace='socketRead0(SocketInputStream.java:native) / socketRead(SocketInputStream.java:116) / read(SocketInputStream.java:171) / read(SocketInputStream.java:141) / readFully(InputRecord.java:465) / read(InputRecord.java:503) / readRecord(SSLSocketImpl.java:983) / performInitialHandshake(SSLSocketImpl.java:1385) / writeRecord(SSLSocketImpl.java:757) / write(AppOutputStream.java:123) / flushBuffer(BufferedOutputStream.java:82) / flush(BufferedOutputStream.java:140) / sendMessage(LDAPConnectionInternals.java:543) / sendMessage(LDAPConnection.java:4249) / process(SimpleBindRequest.java:551) / bind(LDAPConnection.java:2143) / getReferralConnection(LDAPConnection.java:4573) / lambda$followReferral$11(LdapUtils.java:601) / doPrivileged(AccessController.java:native) / privilegedConnect(LdapUtils.java:87) / followReferral(LdapUtils.java:601) / access$300(LdapUtils.java:66) / searchResultReceived(LdapUtils.java:533) / responseReceived(AsyncSearchHelper.java:240) / run(LDAPConnectionReader.java:569)', revision=24201)
at com.unboundid.ldap.sdk.LDAPConnectionInternals.sendMessage(LDAPConnectionInternals.java:574) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnection.sendMessage(LDAPConnection.java:4249) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.SimpleBindRequest.process(SimpleBindRequest.java:551) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnection.bind(LDAPConnection.java:2143) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_151]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_151]
at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_151]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_151]
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) ~[?:?]
at sun.security.ssl.InputRecord.read(InputRecord.java:503) ~[?:?]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) ~[?:?]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) ~[?:?]
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:757) ~[?:?]
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[?:?]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[?:1.8.0_151]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[?:1.8.0_151]
at com.unboundid.ldap.sdk.LDAPConnectionInternals.sendMessage(LDAPConnectionInternals.java:543) ~[?:?]
... 12 more

jfreeman · March 14, 2018, 5:50am

[2018-03-14T04:44:29,443][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://NTDev.corp.company.com/DC=NTDev,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPBindException: 80090308: LdapErr: DSID-0C09041C, comment: AcceptSecurityContext error, data 52e, v42cd
[2018-03-14T04:44:30,592][WARN ][o.e.x.s.a.l.s.LdapUtils  ] caught exception while trying to follow referral [ldaps://winse.corp.company.com/DC=winse,DC=corp,DC=company,DC=com]
com.unboundid.ldap.sdk.LDAPBindException: 80090308: LdapErr: DSID-0C09042A, comment: AcceptSecurityContext error, data 52e, v3839
at com.unboundid.ldap.sdk.LDAPConnection.bind(LDAPConnection.java:2171) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnection.getReferralConnection(LDAPConnection.java:4573) ~[unboundid-ldapsdk-3.2.0.jar:3.2.0]
at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.lambda$followReferral$11(LdapUtils.java:601) ~[x-pack-security-6.2.2.jar:6.2.2]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_151]
at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.privilegedConnect(LdapUtils.java:87) ~[x-pack-security-6.2.2.jar:6.2.2]
at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.followReferral(LdapUtils.java:601) ~[x-pack-security-6.2.2.jar:6.2.2]
at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.access$300(LdapUtils.java:66) ~[x-pack-security-6.2.2.jar:6.2.2]
at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils$LdapSearchResultListener.searchResultReceived(LdapUtils.java:533) [x-pack-security-6.2.2.jar:6.2.2]
at com.unboundid.ldap.sdk.AsyncSearchHelper.responseReceived(AsyncSearchHelper.java:240) [unboundid-ldapsdk-3.2.0.jar:3.2.0]
at com.unboundid.ldap.sdk.LDAPConnectionReader.run(LDAPConnectionReader.java:569) [unboundid-ldapsdk-3.2.0.jar:3.2.0]

system · April 11, 2018, 5:50am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Warning Observed after integrating with Active Directory Elasticsearch elastic-stack-security , docker	2	272	May 23, 2023
Execute wizard watcher: Error: input search: Error: Request Timeout after 60000ms Kibana	4	824	February 22, 2019
java.net.SocketTimeoutException: Read timed out when nodes leave the cluster Elasticsearch	1	89	May 1, 2024
Elastice ECE Watcher SMTP timeout Kibana	3	651	July 30, 2019
Elasticsearch Watcher - socket_timeout_exception when trying to send a .pdf email attachment generated from a Dashboard Elasticsearch elastic-stack-alerting	5	18	September 30, 2024

Read timeout warning when Follow referrals is true

Related Topics