Sharepoint online connector content source - 403 after few days of connecting and validating

Elastic Search
1

I am using Elasticsearch 8.6 and have used Sharepoint enterprise source to connect our organization sharepoint online instance.
I was able to successfully connect and sync the contents of the desired sites using the Azure AD admin. However, after 3 days, the incremental and deletion syncs started failing with the following error in enterprise-search/app-server.log
##################################################
/usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/office365/custom_client.class:265:in `raise_any_errors': got a 403 from https://graph.microsoft.com/v1.0/groups/338e4b86-7cfa-493a-993e-fa3132c1fa21/sites/root with query {:$select=>"id,name"}
##################################################
This error is coming now on all the sync requests. Please note that this same account and permissions as described in the document "Connecting SharePoint Online | Workplace Search Guide [8.6] | Elastic" were configured but later this error starts happening. Could someone please help in this issue? I can provide the logs as needed,

Thanks,
Amol

2

Hi @amolpathak224 ,

Is there any chance that your configured credentials for the connector have expired or were revoked? Using the same credentials and the same user, do you also get a 403 when making the same request through the Graph API Explorer?

1 Like
3

Thanks @Sean_Story Sean for your reply, it seems that there was some issue with O365 credentials temporarily which was disallowing the user login. After 2 days it automatically started to work, it seems some issue with our enterprise O365 login. Thank you again!

1 Like
4

Hello @Sean_Story ,
I copied this ELK instance machine and moved the image to the instance to production, as is. After making the relevant IP related changes, the applications were up and everything is working fine. However, after moving the Sharepoint Connector stopped working. I have updated the redirect URI in the Azure AD and changed the respective content details. Still after connecting the Sharepoint connector fails with
#######################################
Error after less than 5 seconds
Updated 0 items
Failure caused by: Connectors::ContentSources::Office365::CustomClient::ClientError: got a 403 from https://graph.microsoft.com/v1.0/sites/ with query {:$select=>"id,name", :search=>"", :top=>10}
########################################
All required accesses are given from Azure App, and also one more thing noticeable in enterprise-search logs
########################################
ce365/custom_client.class:265:in raise_any_errors': got a 403 from https://graph.microsoft.com/v1.0/me with query (Connectors::ContentSources::Office365::CustomClient::ClientError) from /usr/share/enterprise-search/lib/war/gems/gems/actionpack-5.2.8.1/lib/action_controller/metal/mime_responds.rb:203:in respond_to'
from /usr/share/enterprise-search/lib/war/gems/gems/actionpack-5.2.8.1/lib/action_controller/metal/basic_implicit_render.rb:6:in send_action' from /usr/share/enterprise-search/lib/war/gems/gems/actionpack-5.2.8.1/lib/action_controller/metal/rendering.rb:30:in process_action'
from /usr/share/enterprise-search/lib/war/gems/gems/actionpack-5.2.8.1/lib/action_controller/metal/rescue.rb:22:in process_action' from /usr/share/enterprise-search/lib/war/gems/gems/actionpack-5.2.8.1/lib/action_controller/metal/instrumentation.rb:34:in block in process_action'
from /usr/share/enterprise-search/lib/war/gems/gems/actionpack-5.2.8.1/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
########################################################

Due to this the error, the Configuration section in sharepoint connector is also not coming up it seems, which was visible in the older setup.

5

How did you do this?

We've seen similar issues before if you use a Snapshot/Restore, but do not keep the exact same secret_management.encryption_keys (configured in your enterprise-search.yml).

Because Enterprise Search encrypts your OAuth ClientId and ClientSecret, if you change the underling encryption key, those values become "garbage", and then you can't go through the OAuth flow to authenticate or refresh your token. Our UIs also sometimes fail, because they expect an object in a payload, but get a string due to the bad decryption.

6

We copied the VM image and moved it a new cluster in prod. So all the config, indices remained same. Once the image is copied, I changed all the config files for IP address and URL changes and after that started ELK services and enterprise-search. Everything worked fine, except the sharepoint connector. I removed the source, thereby the index also got deleted and tried reconnecting it using the same admin user of Azure but no luck. Still it's seeing 403 issues on all the URIs.
Is it possible to check if the Elastic is receiving the token from Azure on redirect URI? I can't find any relevant failure in logs except this 403 error. The encryption key is same as the older machine.
Thanks

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.