Elastic self managed connector client - source code issue

Elastic Search
1

Trying to install and configure elastic enterprise search connector(self managed) to connect with share point online.
Using source code to run the connector client.
While setup can be able to create the executable files for the connector client, but while start the connector(./bin/connector) getting the below exception
Exception:

Traceback (most recent call last):
  File "./bin/connectors", line 33, in <module>
    sys.exit(load_entry_point('elasticsearch-connectors', 'console_scripts', 'connectors')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/./bin/connectors", line 22, in importlib_load_entry_point
    for entry_point in distribution(dist_name).entry_points
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/importlib/metadata/__init__.py", line 861, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/importlib/metadata/__init__.py", line 399, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for elasticsearch-connectors

Reference link:
Connector client - Connector clients (self-managed) | Enterprise Search documentation [8.12] | Elastic
Connector CLI used to install - connectors/docs/CLI.md at main · elastic/connectors · GitHub

Require some help on the above exception, please let me know whether you faced the issue.

Thanks.

2

Hi @Arun_Kumar8 ,

Sorry you're having issues running the connectors from source.

Have you tried running make clean install since hitting this issue? This looks like the type of error that we sometimes see when we change branches or make code changes but forget to re-install packages.

Make sure that make clean install completes without errors.

If you're still having the issue after that, can you tell me:

  1. what branch you're running from
  2. what code changes you've made (if any)
  3. what operating system you're running on
  4. what bin/pip freeze outputs for you
3

Hi @Sean_Story,

Thanks for your quick response.
With the make clean install, it is working fine.

we can be able to create connector and job for the service type sharepoint_online.
If we view the Job from the Job id, the status showing is pending, no documents getting indexed but index is created.

Can you please help us on how to debug more or how to check the logs for the same.

Note: We are using Elastic trial version.

Is this due to any licensing issue?

command used to create Job - ./bin/connectors job start -i GAhKZo4B8S1noEQReDNs -t full -o json

Job status

job_id                    GQhMZo4B8S1noEQRDTPe
 connector_id              GAhKZo4B8S1noEQReDNs
 index_name                sharepoint_onlineifc
 job_status                pending
job_type                  full
documents_indexed         0
volume_documents_indexed  0
documents_deleted         0

Thanks,
Arun

4

Is this due to any licensing issue?

Shouldn't be, no. The only things tied to a paid license would be Document Level Security for the SPO connector, but even then I don't believe it would just hang for you.

the status showing is pending, no documents getting indexed but index is created.

To diagnose this, you'll want to look at the connector framework's logs. You may also want to consider changing the log level to DEBUG if you don't initially have enough information.

One thing it might be (can't be sure without logs) is that you may have a LOT of sites that the connector is taking a long time paging through. Try disabling the Enumerate all sites? configuration, and specifying just a single site to start with. See the configuration docs for details: Elastic SharePoint Online connector reference | Enterprise Search documentation [8.12] | Elastic

5

Thanks Sean.

We did enable the DEBUG logging in config.yml(service.log_level:DEBUG) inside the connectors directory. However, we do not see any logs being generated.

We disabled Enumerate all sites, Subsites but still no luck.

6

How are you running connectors? Are you running from the source code with make run or are you using docker images?
Do you not see any logs, or just didn't see DEBUG logs after making the change?
Did you restart the connectors process?

The connector process produces logs. So we just need to validate that you're actually starting it. If it's not running, that would explain why your jobs always stay pending

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.