# Use azure active directory with NEST/Elasticsearch.net

**URL:** <https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [August 10, 2021, 8:49am UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893 "2021-08-10T08:49:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhishek\_Tripathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tripathi/32/92750_2.png) [@Abhishek\_Tripathi](https://discuss.elastic.co/u/Abhishek_Tripathi)\
**Post date:** [August 10, 2021, 8:49am UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/1 "2021-08-10T08:49:52Z")

</div>

We are working on a project where we are authenticating users with Azure Active Directory. Upon the successful authentication, the user's browsers receive an Id and Access token, and then we use the same access token to query other Microsoft products (Sharepoint, OneDrive, etc).

We are planning to use Elastic Search for our search need. We have already set up SAML/ OpenId realms on our ECE Deployment Portal and Cluster. So if any users try to access ECE deployment portal/ Kibana, they will be prompted to authenticate against Microsoft Azure AD, and upon successful authentication, they get redirected to ECE or Kibana.

We are using C# and NEST dll ([ElasticSearch.Net](http://ElasticSearch.Net)) to create queries and search the elastic search endpoint. We are not sure how exactly should we use the access token received on the UI side with Elastic Search to query out indices. We know, we can use native user credentials or API keys to access the elastic search but we want to use the same azure ad authentication flow(SAML/OpenID) to access Elastic Search.

**Is it possible to use the Azure AD access token received on the UI side to access & query Elastic Search Clusters or is there any other way to re-authenticate users while they try to access Elastic Search Cluster?  
Is there a way to authenticate the users with the elastic search endpoint and generate an access token that can be used to query elastic search further?**

In short, we want to re-authenticate users with Elastic Search while querying the data?

```
var settings = new ConnectionSettings(new Uri(mEsQuerySource.Url));
settings.BasicAuthentication("user", "plain text password");
mClient = new ElasticClient(settings);
```

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [August 11, 2021, 1:47am UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/2 "2021-08-11T01:47:07Z")

</div>

In current versions of Elasticsearch (as I write this, 7.14 is the latest version) there is no way to use an Azure AD access token to directly access Elasticsearch.  
That is, you cannot have your application authenticate directly to AAD and then use the tokens you receive from AAD as a credential to authenticate to Elasticsearch.  
There is no authentication provider in Elasticsearch that works with arbitrary tokens from an external issuer.

You can however do the same thing that ECE and Kibana do and perform SAML or OpenID Connect authentication via Elasticsearch, in order to generate Elasticsearch access & refresh tokens (which are separate from the Azure AD tokens).

There is documentation on how to perform [SAML](https://www.elastic.co/guide/en/elasticsearch/reference/current/saml-guide-stack.html#saml-no-kibana) or [OIDC](https://www.elastic.co/guide/en/elasticsearch/reference/current/oidc-guide.html#oidc-without-kibana) authentication to Elasticsearch via a custom application.

The high level overview would be (I assume SAML here, but OIDC would be similar):

- When a user accesses your application they would authenticate against AzureAD as normal
- Then, you would use the Elasticsearch APIs to perform an additional authentication against an Elasticsearch SAML realm with Elasticsearch as the service provider and AzureAD as the Identity Provider.
- Since the user is already authenticated within Azure AD, that second authentication process should be transparent to the user - AAD will simply issue a new SAML assertion with Elasticsearch as the recipient.
- Those Elasticsearch APIs will accept the SAML assertion, and return a pair of tokens (access + refresh) that can be used to authenticate to Elasticsearch
- Your application will retain the access + refresh tokens for the user's session
- The access token will be used to [authenticate](https://www.elastic.co/guide/en/elasticsearch/reference/current/token-authentication-services.html#token-authentication-services) when accessing Elasticsearch APIs
- The refresh token will be used to [generate a new access token](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-get-token.html) when the old one expires (or is about to expire).

If your users are in an identity store that Elasticsearch can query (e.g. something that supports LDAP search), then another option is to use the Elasticsearch [`run-as` capability](https://www.elastic.co/guide/en/elasticsearch/reference/current/run-as-privilege.html).  
In this case your application would authenticate to Elasticsearch using a single system credential (probably a user in the native realm). That user would have permission to `run-as` all other users and this can be used to perform searches on behalf of your end users without needing them to authenticate directly to Elasticsearch.

The final option would be to implement a custom realm, if you have engineers who are comfortable writing Java.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tripathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tripathi/32/92750_2.png) [@Abhishek\_Tripathi](https://discuss.elastic.co/u/Abhishek_Tripathi)\
**Post date:** [August 11, 2021, 7:15am UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/3 "2021-08-11T07:15:51Z")

</div>

Hi Tim,

Thanks for responding!

I do have a follow-up question on point 2 but first, let me explain how point 1 works in our case -

- **When a user accesses your application they would authenticate against AzureAD as normal**  
_We are using C# and Angular on the front end and we would be using MSAL libraries to authenticate our users. For authentication, we are opening the Microsoft Authentication page for users so that they can use their Azure AD credentials to authenticate themselves._

- **Then, you would use the Elasticsearch APIs to perform an additional authentication against an Elasticsearch SAML realm with Elasticsearch as the service provider and AzureAD as the Identity Provider.**  
_When you say we would need to use the Elasticsearch APIs to perform an additional authentication, do you mean we will have to redirect the user to the Microsoft Authentication page again?_

When I use a service account to prepare my realm for user authentication and execute the below request, It returns a redirect URL and the documentation says we need to redirect our application to this URL for authentication to happen.

```auto
POST /_security/oidc/prepare
{
  "realm" : "oidc1"
}

```

```auto
{
  "redirect" : "https://login.microsoftonline.com/{tenant-id}/oauth2/v2.0/authorize?scope=openid+email&response_type=code&redirect_uri=https://xyz.azure.elastic.bain.com%3A9243%2Fapi%2Fsecurity%2Foidc%2Fcallback&state=zoTG-Iaj20hS9ag3TbRlae6woNtPv5mE1Elku6q5UVg&nonce=Y-BkrE2IE2LZ5MgKzKJZIZENrWdDAMty1DlHPk7bV5w&client_id={client_id}",
  "state" : "zoTG-Iaj20hS9ag3TbRlae6woNtPv5mE1Elku6q5UVg",
  "nonce" : "Y-BkrE2IE2LZ5MgKzKJZIZENrWdDAMty1DlHPk7bV5w",
  "realm" : "oidc1"
}

```

You see, we have already redirected our users (on point 1) to authenticate themselves and if we follow this documentation we would have to redirect the users again. It would not be a very good user experience if we ask them to authenticate themselves twice for the same application.

Is there a way to authenticate users against the elastic realm without redirecting them again to the Microsoft authentication page as we have already authenticated the user (in point 1)?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [August 13, 2021, 1:34am UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/4 "2021-08-13T01:34:50Z")

</div>

> [@Abhishek\_Tripathi](#):
>
> When you say we would need to use the Elasticsearch APIs to perform an additional authentication, do you mean we will have to redirect the user to the Microsoft Authentication page again?

Yes, that's exactly what I mean.  
However, per my earlier comment:

> Since the user is already authenticated within Azure AD, that second authentication process should be transparent to the user - AAD will simply issue a new SAML assertion with Elasticsearch as the recipient.

Depending on exactly how AAD is configured (sorry, I'm not an expert on the specific of AAD configuration, so I can't be more precise than that), the user is unlikely to even see the a login page. Because they have a valid AAD session, they don't need to enter their credentials again, we simply need to trigger a SAML assertion with the necessary details to be sent to Elasticsearch.

> [@Abhishek\_Tripathi](#):
>
> You see, we have already redirected our users (on point 1) to authenticate themselves and if we follow this documentation we would have to redirect the users again

Yes, that is exactly what you need to do. Elasticsearch cannot use your existing tokens because they have nothing to do with Elasticsearch. You need to generate new ones.

> [@Abhishek\_Tripathi](#):
>
> we have already authenticated the user (in point 1)?

You have, but Elasticsearch hasn't.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tripathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tripathi/32/92750_2.png) [@Abhishek\_Tripathi](https://discuss.elastic.co/u/Abhishek_Tripathi)\
**Post date:** [August 17, 2021, 7:13am UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/5 "2021-08-17T07:13:08Z")

</div>

Hi Tim,

Following your suggestions and elastic documentation from [OIDC](https://www.elastic.co/guide/en/elasticsearch/reference/current/oidc-guide.html#oidc-without-kibana), I registered two azure ad applications (1 for custom web app and 1 for elastic search realm setup) on my tenant.

Here are my realm configurations -

```auto
xpack:
  security:
    authc:
      realms:
        oidc:
          oidc1:
            order: 2
            rp.client_id: "{client-id}"
            rp.response_type: "code"
            rp.requested_scopes: ["openid", "email"]
            rp.redirect_uri: "https://customwebapp.azurewebsites.net"
            op.issuer: "https://login.microsoftonline.com/{tenanet-id}/v2.0"
            op.authorization_endpoint: "https://login.microsoftonline.com/{tenanet-id}/oauth2/v2.0/authorize"
            op.token_endpoint: "https://login.microsoftonline.com/{tenanet-id}/oauth2/v2.0/token"
            op.userinfo_endpoint: "https://graph.microsoft.com/oidc/userinfo"
            op.endsession_endpoint: "https://login.microsoftonline.com/{tenanet-id}/oauth2/v2.0/logout"
            rp.post_logout_redirect_uri: "https://customwebapp.azurewebsites.net"
            op.jwkset_path: "https://login.microsoftonline.com/{tenanet-id}/discovery/v2.0/keys"
            claims.principal: email
            claims.groups : groups

```

I am able to prepare the OIDC realm and getting the redirect, state, and nonce values as a response from elastic. When I redirect the browser to the redirect URL, it asks me to authenticate with Microsoft azure ad. It does ask me to re-authenticate but doesn't ask for a password as I am already logged in to the browser session. Upon successful authentication, it returns the browser window to my web app.

Here is how I am doing the prepare api call from my angular app -

```auto
PrepElastic() {
    var url = "https://{appid}.eastus2.azure.elastic.xyz.com:9243/_security/oidc/prepare"
    let authorizationData = 'Basic ' + btoa('facilitator' + ':' + 'somePasswordHere');

    const headers = { 'Authorization': authorizationData, 'Content-Type': 'application/json' };
    const body = { "realm": "oidc1" };
    return this.http.post<any>(url, body, { headers });
  }

```

I store the state and nonce returned from the prepare call and use the same to authenticate with authenticate API but it gives me an error in the response.

```auto
AuthenticateElastic() {
    var url = "https://{appid}.eastus2.azure.elastic.xyz.com:9243/_security/oidc/authenticate"
    let authorizationData = 'Basic ' + btoa('facilitator' + ':' + 'somePasswordHere');

    const headers = { 'Authorization': authorizationData, 'Content-Type': 'application/json' };
    const body = {
      "redirect_uri": this.dataShareService.getAuthCodeUrl(),
      "state": this.dataShareService.getstate(),
      "nonce": this.dataShareService.getnonce(),
      "realm": "oidc1"
    };
    return this.http.post<any>(url, body, { headers });
  }

```

Here is the error message I am getting on the browser side in response -

`{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [<OIDC Token>] for action [cluster:admin/xpack/security/oidc/authenticate]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"unable to authenticate user [<OIDC Token>] for action [cluster:admin/xpack/security/oidc/authenticate]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}`

I turned on tracing on elastic search realm in order to find out the actual error -

```auto
PUT /_cluster/settings
{
  "transient": {
    "logger.org.elasticsearch.xpack.security.authc.oidc": "trace"
  }
}

```

and I see below error message logged-

`[elasticsearch.server][WARN] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to validate the response, the response did not contain a state parameter])`

Reading the above message seems that we are not passing state but I can assure that we are passing all the correct values with the authenticate request.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/7/57b48e3ec14047fb00afa27e7456515e1e19988e.png)

Could you please help here?

Thanks,  
Abhishek

---

<div class="post-metadata">

**Author:** ![Abhishek\_Tripathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek_tripathi/32/92750_2.png) [@Abhishek\_Tripathi](https://discuss.elastic.co/u/Abhishek_Tripathi)\
**Post date:** [August 19, 2021, 6:23pm UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/6 "2021-08-19T18:23:52Z")

</div>

So the problem was with the Auth-Code redirect URI returned from prepare call after the authentication. It has session\_id appended to it at the end. Once I remove it, it worked. Thanks Tim for the support.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 16, 2021, 6:24pm UTC](https://discuss.elastic.co/t/use-azure-active-directory-with-nest-elasticsearch-net/280893/7 "2021-09-16T18:24:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.