HAYSTACK_CONNECTIONS

[sdarwin (Sam Darwin)]

Hi,
Not sure which category to post this in. Django Haystack GitHub - django-haystack/django-haystack: Modular search for Django supports Elasticsearch as a backend search engine. The connection is specified this way:

HAYSTACK_CONNECTIONS = {
    "default": {
        "ENGINE": "haystack.backends.elasticsearch7_backend.Elasticsearch7SearchEngine",
        "URL": "http://127.0.0.1:9200/",
        "INDEX_NAME": my_index,
    },
}

Next, considering Elastic Cloud. Which is hosted Elasticsearch.

1. Is Elastic Cloud "secure by default" in the sense of requiring the usage of an API key or token? Or does it start out as clear-text and no password?

2. What are examples of a formatted HAYSTACK_CONNECTIONS that would work with Elastic Cloud?

3. Would it make sense for someone from the Elastic development team to reach out to django-haystack, and provide authentication examples for the documentation? Assuming auth is needed. Maybe it isn't.

[carly.richmond]

Hi @sdarwin,

Welcome to the community! Looking at django-haystack it looks to be a community driven project, and although it supports Elasticsearch we don't have any oversight of it. I see that the Haystack docs state that Elasticsearch versions 1.x, 2.x, 5.x and 7.x are supported, and no mention of the latest 8.x.

In terms of the specific questions you ask:

1. Elastic Cloud has security explicitly enabled by default as of version 8.x, including authentication and TLS. If you check our Python client connection examples you'll see connections can be initialised a few ways including using either an API key or username, password combination.

2. Looking at the Haystack examples there is a 7.x example in the docs, but not a latest example using 8.x. It's not clear if 8.x support has been worked on having a quick peek at the open GitHub issues. I don't see support for an API Key in the Elasticsearch backends having a quick peek at the backend Python code on GitHub (disclaimer I'm more of a dabbler in Python).

3. I'm not sure on that one. It's a community driven project and they do accept contributions from others as well as the raising of issues. It can be contributed to by anyone, not just Elastic.

Have you raised an issue on the project based on their contribution guidelines to see if someone with knowledge of the project can help?

[sdarwin (Sam Darwin)]

Hi Carly,

I don't see support for an API Key in the Elasticsearch backends

So maybe I was right in imagining there are blockers to using django-haystack with Elastic Cloud.

API Keys, and 8.x support.

I signed up at Elastic Cloud, and was going to be a paid subscriber. But then, ran into these issues. Switched back to 7.x self-hosted. Now, not using Cloud.

Haystack is open source, anyone can send pull requests. If Haystack had excellent integration with Elastic Cloud, in general, would a few more django developers choose the hosted service? Possibly.

[sdarwin (Sam Darwin)]

Have you raised an issue on the project?

Yes. I commented in an open issue "haystack elasticsearch 2.4 - how to secure ?" Issue #1695. A problem with the django-haystack project is there are 430 open issues, 747 closed issues. (That ratio seems to show they aren't solving issues quickly)