# Unknown problem occurred when syncing

**URL:** <https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483>\
**Category:** Elastic Search\
**Tags:** elastic-workplace-search\
**Created:** [June 21, 2021, 6:31am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483 "2021-06-21T06:31:35Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [June 21, 2021, 6:31am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/1 "2021-06-21T06:31:35Z")

</div>

I added GoogleDrive as a source for WorkplaceSearch.  
After a few hours, I was able to retrieve the data from GoogleDrive and search for documents from WorkplaceSearch, but suddenly I got the following message.

```auto
Unknown problem occurred when syncing. [Check diagnostic bundle for details - Message id: 60d02e4984c212b38a33d4cd].

```

How do I resolve this?

I'm posting the contents of a json file that I got by pressing the `Download diagnostics data` button in `Content Sources > Google Drive > Source Settings`.

```auto
{
  "general": {
    "version": "7.13.2",
    "revision": "af74d68aafaab071eafa00c2ab628ecf5a1daa27",
    "elasticsearchVersion": "7.11.0"
  },
  "source": {
    "id": "60c09c5b84c21256a3a17b64",
    "serviceType": "google_drive",
    "name": "Google Drive",
    "context": "organization",
    "dlpEnabled": false,
    "recentJobs": {
      "incremental": [
        {
          "createdAt": "2021-06-21T06:14:28Z",
          "status": "error",
          "fatalException": {
            "friendly_message": "Unknown problem occurred when syncing.",
            "stack_trace": "/usr/share/enterprise-search/lib/war/frito_pie/lib/secret_keeper/google.class:35:in `refresh_client\u0027: undefined method `[]\u0027 for nil:NilClass\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/lib/secret_keeper/google.class:19:in `refresh\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/lib/secret_keeper.class:274:in `ensure_valid!\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/lib/secret_keeper.class:249:in `fetch_secret_details\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/lib/secret_keeper.class:115:in `fetch_content_source_secret_details!\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/app/models/frito_pie/content_source.class:663:in `authorization_details\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/app/models/frito_pie/content_source.class:671:in `authorization_data\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/lib/frito_pie/content_source_service/google_drive.class:141:in `block in web_client\u0027\n\tfrom org/jruby/RubyKernel.java:1897:in `tap\u0027\n\tfrom /usr/share/enterprise-search/lib/war/frito_pie/lib/frito_pie/content_source_service/google_drive.class:140:in `web_client\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:93:in `client\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:193:in `get_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:123:in `block in yield_files\u0027\n\tfrom org/jruby/RubyKernel.java:1442:in `loop\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:122:in `yield_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:109:in `yield_actions_and_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:97:in `yield_domain_actions_and_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:21:in `yield_document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:151:in `block in document_changes\u0027\n\tfrom org/jruby/RubyGenerator.java:104:in `each\u0027\n\tfrom org/jruby/RubyEnumerator.java:396:in `each\u0027\n\tfrom org/jruby/RubyEnumerator.java:414:in `each\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:158:in `block in document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `block in measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:53:in `duration\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/stats.class:4:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:148:in `block in document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:109:in `block in with_auth_tokens_and_retry\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:84:in `convert_rate_limit_errors\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:108:in `block in with_auth_tokens_and_retry\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:223:in `convert_transient_server_errors\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:107:in `with_auth_tokens_and_retry\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:147:in `block in document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `block in measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:53:in `duration\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/stats.class:4:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:146:in `document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/index.class:13:in `block in run\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/abstract_extractor_work.class:83:in `run_with_suspension\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/index.class:12:in `run\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/abstract_extractor_work.class:121:in `execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/workers/extract_worker.class:7:in `block in run\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:24:in `block in execute\u0027\n\tfrom com/concurrent_ruby/ext/SynchronizationLibrary.java:232:in `synchronize\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:19:in `execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/config/initializers/concurrent.class:16:in `block in realize\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/java_executor_service.rb:79:in `run\u0027\n",
            "id": "60d02e4884c212d93133d463",
            "message": "undefined method `[]\u0027 for nil:NilClass",
            "class": "NoMethodError"
          },
          "errorReason": "unknown",
          "completedAt": "2021-06-21T06:14:32Z",
          "durationSeconds": 4.0,
          "documentErrorCount": 0,
          "documentErrors": []
        },
... snip ...

```

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [June 21, 2021, 3:08pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/2 "2021-06-21T15:08:47Z")

</div>

Hi @its-ogawa ,

Looking at the `fatalException.stack_trace`, it looks like Workplace Search can no longer find the authorization information needed to communicate with Google Drive. Can you check a few things?

- make sure that your Connector Configuration (OAuth Client ID and Client Secret) is still set and correct.
- make sure that if you are using multiple Enterprise Search servers that they all have the same `secret_management.encryption_keys` value in your `config/enterprise-search.yml`
- make sure that you haven't changed the value of `secret_management.encryption_keys` since configuring your Connector Configuration for Google Drive.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [June 22, 2021, 1:09am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/3 "2021-06-22T01:09:10Z")

</div>

> [@Sean\_Story](#):
>
> Looking at the `fatalException.stack_trace` , it looks like Workplace Search can no longer find the authorization information needed to communicate with Google Drive. Can you check a few things?

Thanks for the answer.

> - make sure that your Connector Configuration (OAuth Client ID and Client Secret) is still set and correct.

As far as I can see in GoogleCloudPlatform, I can confirm that the Client ID and Client Secret are set.  
Can I check them from the WorkplaceSearch administrator screen? I believe I have set those information for sure.

> - make sure that if you are using multiple Enterprise Search servers that they all have the same secret\_management.encryption\_keys value in your config /enterprise-search.yml

You are using one EnterpriseSearch server.

> - make sure that you haven't changed the value of secret\_management.encryption\_keys since configuring your Connector Configuration for Google Drive .

The value of secret\_management.encryption\_keys has not been changed.  
It's not a very good configuration, but since you have only one value of secret\_management.encryption\_keys set, I don't think it will switch.

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [June 22, 2021, 3:30pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/4 "2021-06-22T15:30:21Z")

</div>

> [@its-ogawa](#):
>
> Can I check them from the WorkplaceSearch administrator screen? I believe I have set those information for sure.

Yes, you should validate that the Client ID and Client Secret that you see in the Google Cloud Platform match what you have configured for your connector. You can get to that view by clicking Settings,

 ![Screen Shot 2021-06-22 at 10.22.25 AM](https://us1.discourse-cdn.com/elastic/original/3X/8/5/85ec220daa5eacfe00ef07d9de354d3fd9cdd1d9.png)

Then click Content source connectors:

 ![Screen Shot 2021-06-22 at 10.22.52 AM](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e5261611ade8be01ddc299a9371972673e77b75f.png)

Then click the link by Google Drive:

 ![Screen Shot 2021-06-22 at 10.23.22 AM](https://us1.discourse-cdn.com/elastic/original/3X/6/3/630f504d190f02253f1097b504d5ba4bc0e5ea1c.png)

Then validate that these fields match what you have in the Google Cloud Platform.

 ![Screen Shot 2021-06-22 at 10.23.55 AM](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0bf728ee54d2e9bff3bee0ac0a456d85d73951e2.png)

> [@its-ogawa](#):
>
> The value of secret\_management.encryption\_keys has not been changed.  
> It's not a very good configuration, but since you have only one value of secret\_management.encryption\_keys set, I don't think it will switch.

It's not uncommon for customers to make a mistake when manually updating their version of Enterprise Search to not carefully copy their value of `secret_management.encryption_keys` from the old config file to the new config file. This means that any secrets added in the old version are now un-encryptable in the new version.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [June 23, 2021, 3:38am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/5 "2021-06-23T03:38:04Z")

</div>

Thank you for the clear explanation.  
(I'd like to put this explanation in the official reference!)

When I followed the steps and checked the Client ID and Client Secret, they were blank for some reason.

After checking them in GoogleCloudPlatform and running Save configuration again, another GoogleDrive content was created and the motivation started.

After a while, the synchronization stops and the same message as before appears.

```auto
An unknown problem occurred during synchronization. [Check the diagnostic bundle for more information - Message ID: 60d2991884c2129788374894].

```

The content will be similar, but we will also describe the content of the newly obtained json file.

```auto
{
  "general": {
    "version": "7.13.2",
    "revision": "af74d68aafaab071eafa00c2ab628ecf5a1daa27",
    "elasticsearchVersion": "7.11.0"
  },
  "source": {
    "id": "60d2938884c21260d5349327",
    "serviceType": "google_drive",
    "name": "Google Drive (1)",
    "context": "organization",
    "dlpEnabled": false,
    "recentJobs": {
      "incremental": [
        {
          "createdAt": "2021-06-23T01:51:05Z",
          "status": "complete",
          "fatalException": null,
          "errorReason": null,
          "completedAt": "2021-06-23T01:51:20Z",
          "durationSeconds": 15.0,
          "documentErrorCount": 0,
          "documentErrors": []
        }
      ],
      "full": [
        {
          "createdAt": "2021-06-23T01:51:05Z",
          "status": "error",
          "fatalException": {
            "friendly_message": "Unknown problem occurred when syncing.",
            "stack_trace": "/usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/http_command.rb:235:in `check_status\u0027: Unknown error\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/api_command.rb:136:in `check_status\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/http_command.rb:194:in `process_response\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/http_command.rb:310:in `execute_once\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/http_command.rb:113:in `block in execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/retriable-3.1.2/lib/retriable.rb:61:in `block in retriable\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/retriable-3.1.2/lib/retriable.rb:56:in `retriable\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/http_command.rb:110:in `block in execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/retriable-3.1.2/lib/retriable.rb:61:in `block in retriable\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/retriable-3.1.2/lib/retriable.rb:56:in `retriable\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/http_command.rb:102:in `execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-apis-core-0.2.1/lib/google/apis/core/base_service.rb:377:in `execute_or_queue_command\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/google-api-client-0.53.0/generated/google/apis/drive_v3/service.rb:1215:in `list_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:193:in `get_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:123:in `block in yield_files\u0027\n\tfrom org/jruby/RubyKernel.java:1442:in `loop\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:122:in `yield_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:109:in `yield_actions_and_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:105:in `yield_user_actions_and_files\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:22:in `yield_document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:151:in `block in document_changes\u0027\n\tfrom org/jruby/RubyGenerator.java:104:in `each\u0027\n\tfrom org/jruby/RubyEnumerator.java:396:in `each\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:158:in `block in document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `block in measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:53:in `duration\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/stats.class:4:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:148:in `block in document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:109:in `block in with_auth_tokens_and_retry\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/google_drive/extractor.class:84:in `convert_rate_limit_errors\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:108:in `block in with_auth_tokens_and_retry\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:223:in `convert_transient_server_errors\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:107:in `with_auth_tokens_and_retry\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:147:in `block in document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `block in measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:53:in `duration\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/statsd-instrument-2.1.1/lib/statsd/instrument.rb:284:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/stats.class:4:in `measure\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/content_sources/base/extractor.class:146:in `document_changes\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/index.class:13:in `block in run\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/abstract_extractor_work.class:83:in `run_with_suspension\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/index.class:12:in `run\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/work/abstract_extractor_work.class:121:in `execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/lib/connectors/workers/extract_worker.class:7:in `block in run\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:24:in `block in execute\u0027\n\tfrom com/concurrent_ruby/ext/SynchronizationLibrary.java:232:in `synchronize\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/safe_task_executor.rb:19:in `execute\u0027\n\tfrom /usr/share/enterprise-search/lib/war/connectors/config/initializers/concurrent.class:16:in `block in realize\u0027\n\tfrom /usr/share/enterprise-search/lib/war/gems/gems/concurrent-ruby-1.1.8/lib/concurrent-ruby/concurrent/executor/java_executor_service.rb:79:in `run\u0027\n",
            "id": "60d2991884c2129788374894",
            "message": "Unknown error",
            "class": "Google::Apis::TransmissionError"
          },
          "errorReason": "unknown",
          "completedAt": "2021-06-23T02:14:49Z",
          "durationSeconds": 1424.0,
          "documentErrorCount": 0,
          "documentErrors": []
        }
      ]
    },
    "oauthTokenAgeSeconds": 2384,
    "oauthScopes": [
      "email",
      "profile",
      "https://www.googleapis.com/auth/drive.readonly",
      "https://www.googleapis.com/auth/drive.metadata.readonly",
      "https://www.googleapis.com/auth/admin.directory.group.readonly",
      "https://www.googleapis.com/auth/admin.directory.user.readonly"
    ]
  }
}

```

Is there a solution for these?

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [June 23, 2021, 6:39pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/6 "2021-06-23T18:39:22Z")

</div>

> [@its-ogawa](#):
>
> `Google::Apis::TransmissionError`

Has it continued to fail with this error? This tells us that Google had a problem, and is outside of the control of our code. Likely, Google's API had a brief outage of some sort.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [June 25, 2021, 1:45am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/7 "2021-06-25T01:45:07Z")

</div>

> [@Sean\_Story](#):
>
> Has it continued to fail with this error? This tells us that Google had a problem, and is outside of the control of our code. Likely, Google's API had a brief outage of some sort.

I see.

It's a problem with the Google API, isn't it?  
Indeed, when I checked again a few days later, the number of items in the content had increased, so in the case of this error, there is nothing wrong with WorkplaceSearch, so it might be a good idea to give it some time and see what happens.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [June 25, 2021, 1:45am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/8 "2021-06-25T01:45:13Z")

</div>

Sorry again and again, but I am getting a different error.  
How should I interpret this?

```auto
{
  "general": {
    "version": "7.13.2",
    "revision": "af74d68aafaab071eafa00c2ab628ecf5a1daa27",
    "elasticsearchVersion": "7.11.0"
  },
  "source": {
    "id": "60d2938884c21260d5349327",
    "serviceType": "google_drive",
    "name": "Google Drive (1)",
    "context": "organization",
    "dlpEnabled": false,
    "recentJobs": {
      "incremental": [
        {
          "createdAt": "2021-06-24T23:53:54Z",
          "status": "complete",
          "fatalException": null,
          "errorReason": null,
          "completedAt": "2021-06-24T23:53:58Z",
          "durationSeconds": 4.0,
          "documentErrorCount": 0,
          "documentErrors": []
        },
        ... snip ...
      ],
      "delete": [
        {
          "createdAt": "2021-06-24T23:05:52Z",
          "status": "error",
          "fatalException": {
            "friendly_message": "Unknown problem occurred when syncing.",
            "stack_trace": "0: Read timed out\n",
            "id": "60d52cd784c212d931aaeb22",
            "message": "Read timed out",
            "class": "Faraday::TimeoutError"
          },
          "errorReason": "unknown",
          "completedAt": "2021-06-25T01:09:43Z",
          "durationSeconds": 7431.0,
          "documentErrorCount": 0,
          "documentErrors": []
        },
        ... snip ...
      ],
      "full": [
        {
          "createdAt": "2021-06-24T19:29:44Z",
          "status": "error",
          "fatalException": {
            "friendly_message": "Failed to save data to search index.",
            "stack_trace": ... snip ...,
            "message": "Connectors::Work::AbstractExtractorWork::PublishingFailedError",
            "class": "Connectors::Work::AbstractExtractorWork::PublishingFailedError"
          },
          "errorReason": "indexing_failed",
          "completedAt": "2021-06-24T23:16:00Z",
          "durationSeconds": 13576.0,
          "documentErrorCount": 1001,
          "documentErrors": [
            {
              "error_id": "60d5122284c212342b9ecaf7",
              "error_class": "Connectors::Work::AbstractExtractorWork::PublishingFailedError",
              "error_message": "Exception: Connectors::Work::AbstractExtractorWork::PublishingFailedError: Connectors::Work::AbstractExtractorWork::PublishingFailedError: Context: {:message_id\u003d\u003e\"60d5122284c212342b9ecaf7\"}",
              "stack_trace": ... snip ...
            },
            ... snip ...
          ]
        },
        ... snip ...
      ]
    },
    "oauthTokenAgeSeconds": 153,
    "oauthScopes": [
      "email",
      "profile",
      "https://www.googleapis.com/auth/drive.readonly",
      "https://www.googleapis.com/auth/drive.metadata.readonly",
      "https://www.googleapis.com/auth/admin.directory.group.readonly",
      "https://www.googleapis.com/auth/admin.directory.user.readonly"
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [June 25, 2021, 2:58pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/9 "2021-06-25T14:58:03Z")

</div>

Can you re-send with the stacktraces included, instead of `...snip....`? That will help me to identify the root cause. Don't worry about it being unreadable, the discuss formatting will put all these on a single line.

From what you've shared, all I can be sure of is that something went wrong while attempting to index data into elasticsearch. It could be an Elasticsearch health issue, or it could be a bug in Workplace Search, or it could be transient. Does the error continue to happen? It looks like the most recent Incremental job completed successfully.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [June 28, 2021, 12:44am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/10 "2021-06-28T00:44:11Z")

</div>

My apologies.  
There was a lot of text, so I took the liberty of cutting it out.  
The stack\_trace is as follows.

> <https://github.com/its-ogawa/Elastic/blob/main/60d2938884c21260d5349327_google_drive_1624583680_diagnostics.json>

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 9, 2021, 1:37am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/11 "2021-07-09T01:37:09Z")

</div>

After watching for a while, the error was no longer displayed and the status of the sync changed to Complete.  
I was relieved.

* * *

@Sean_Story

Is it correct to assume that WorkplaceSearch will run the sync repeatedly until the status becomes complete?

Also, what should I look for to distinguish between an error that time will resolve and an error that needs some action, like this one?

I'd like to close this topic soon. Thank you in advance.

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [July 9, 2021, 4:12pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/12 "2021-07-09T16:12:46Z")

</div>

Hi @its-ogawa ,

> [@its-ogawa](#):
>
> The stack\_trace is as follows.

Sorry that I missed responding to your full post of the connector diagnostics. There's unfortunately not a smoking gun in there that I can see. I would have recommended that you take the contextual message IDs you can find in there (like `Context: {:message_id\u003d\u003e\"60d4dc4b84c2129788871790\"}"`) and grep for those ids in your logs, looking for other stack traces or errors immediately before those. Something was causing your asynchronous publishing tasks to fail semi-regularly, which tripped the Max Error Ratio circuit breaker: [Common source synchronization issues | Workplace Search documentation [8.11] | Elastic](https://www.elastic.co/guide/en/workplace-search/current/workplace-search-common-source-sync-issues.html#max-error-ratio). If you see this again, you could try increasing either of:

```yml
workplace_search.content_source.max_error_ratio #defaults to 0.15
# OR
workplace_search.content_source.error_ratio_window_size #defaults to 100

```

> [@its-ogawa](#):
>
> Is it correct to assume that WorkplaceSearch will run the sync repeatedly until the status becomes complete?

Sort of. Workplace search will continually sync with your standard content sources. Full re-syncs every 72 hours, "latest changes" every 2 hours, "deleted documents" every 6 hours, and "permissions updates" (if document level permissions are enabled for your source) every 5 minutes. If one of those jobs fails, another job of the same type will restart sooner than it would have otherwise. So yes, sometimes jobs fail for transient reasons (network issues, service unavailaility, etc) and will succeed on a consecutive retry. Sometimes the issue is not transient, and you will have the same failure again and again until the underlying issue is resolved. I suggest you consult [Common source synchronization issues | Workplace Search documentation [8.11] | Elastic](https://www.elastic.co/guide/en/workplace-search/current/workplace-search-common-source-sync-issues.html) with each specific failure to get a feeling for what the issue may be. If it's not in that list or you are still having trouble resolving an issue, you may want to consider talking to us about a [subscription](https://www.elastic.co/subscriptions) in order to access full-time support and troubleshooting.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 12, 2021, 9:41am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/13 "2021-07-12T09:41:15Z")

</div>

@Sean_Story ,

Thank you for your answer.

I am sorry for rushing to answer.  
I am certainly not in a position to take on full time support.  
I would appreciate it if you could answer when you feel like it.  
(I hope you will except the forum from time to time.......)

> [@Sean\_Story](#):
>
> I would have recommended that you take the contextual message IDs you can find in there (like `Context: {:message_id\u003d\u003e\"60d4dc4b84c2129788871790\"}"` ) and grep for those ids in your logs, looking for other stack traces or errors immediately before those.

What are the logs you are referring to here?  
There are some logs in `/var/log/enterprise-search`, but they do not contain document IDs (e.g. `60d4dc4b84c2129788871790`).

In particular, the document ID does not seem to be available through the API for the basic license, so we do not know what document the document ID refers to.  
Is this error something that basic license users can't handle? If so, I am very disappointed.

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [July 12, 2021, 2:28pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/14 "2021-07-12T14:28:00Z")

</div>

Hi @its-ogawa,

> [@its-ogawa](#):
>
> What are the logs you are referring to here?  
> There are some logs in `/var/log/enterprise-search` , but they do not contain document IDs (e.g. `60d4dc4b84c2129788871790` ).

Are you using docker? If so, then yes, the logs would be in `/var/log/enterprise-search`. However, the logs "roll over", so since it has been quite a while since you first posted about this error, those log files have likely removed themselves. Something to just know for next time, since it sounds like your syncs are working now?

> [@its-ogawa](#):
>
> In particular, the document ID does not seem to be available through the API for the basic license, so we do not know what document the document ID refers to.  
> Is this error something that basic license users can't handle? If so, I am very disappointed.

You are correct that the basic license does not include the Search API, which is how you would use a document ID to look up a document. However the `message_id` we were discussing above is not a "document ID" but a "log message ID". That ID is used to help you tie your error message in the UI to the stack trace in your connector diagnostics JSON, to the relevant lines in your logfile.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 13, 2021, 1:09am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/15 "2021-07-13T01:09:35Z")

</div>

> [@Sean\_Story](#):
>
> Are you using docker? If so, then yes, the logs would be in `/var/log/enterprise-search` .

I installed the package on my own CentOS instead of Docker.  
(I guess the configuration isn't too different from Docker)

> [@Sean\_Story](#):
>
> However, the logs "roll over", so since it has been quite a while since you first posted about this error, those log files have likely removed themselves.

Your guess is quite correct. Too much time has passed since the first error.

> [@Sean\_Story](#):
>
> Something to just know for next time, since it sounds like your syncs are working now?

What I need to know is how to read the json file that WorkplaceSearch issues on error.  
This is where the log message ID (which I mistook for the document ID in my last post) shows up in which log file, and where the error details are listed.

In my `/var/log/enterprise-search`, I have 8 different logs being output.

- app-server.log
- connectors.log
- filebeat
- gc.log
- init.log
- stats.log
- system.log
- worker.log

Can I look for the log message ID in the json file in `app-server.log` (?) and assume that the error details are in there? and assume that the error details are listed there?

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [July 13, 2021, 5:06pm UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/16 "2021-07-13T17:06:37Z")

</div>

Hi @its-ogawa ,

> [@its-ogawa](#):
>
> Can I look for the log message ID in the json file in `app-server.log` (?) and assume that the error details are in there? and assume that the error details are listed there?

For errors you see in the content source UI and the connectors diagnostics json, you're most likely to see the corresponding message IDs in `connectors.log` or `worker.log`. But you can use `grep` to search over all the log files just to be sure. `app-server.log` will typically have the logs relevant to client network traffic to Enterprise Search, which can be helpful with troubleshooting any errors you get when interacting with our APIs, or if web pages are not loading correctly.

---

<div class="post-metadata">

**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\
**Post date:** [July 14, 2021, 1:00am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/17 "2021-07-14T01:00:24Z")

</div>

@Sean_Story  
I see!

Thanks for the good quality answers.

If there is any problem, I will first and foremost refer to `connectors.log`, `worker.log` and `app-server.log` to pursue the cause.  
And if I can't figure it out, I would like to consult the Elastic forum (by starting a new topic).

So far, there seems to be no errors or warnings in these logs.  
I am no longer able to reproduce the situation when I started the topic. My apologies.  
I would like to close this topic.

Thank you for your help in getting this resolved.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 31, 2022, 2:48am UTC](https://discuss.elastic.co/t/unknown-problem-occurred-when-syncing/276483/18 "2022-10-31T02:48:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.