How the users can access my Elasticsearch database in my Django SaaS?

[whiterabbit]

Let's say that I have a SaaS based on Django backend that processes the data of the users and write everything to the Elasticsearch. Now I would like to give users access to search and request their data stored in ES using all possible search requests available in ES. Obviously the user should have only access to his data, not to other user's data. I am aware that it can be done in a lot of different ways but I wonder what is safe and the best solution? At this point I store everything in one index and type in the way shown below but I can do this in any way.

            "_index": "example_index",
            "_type": "example_type",
            "_id": "H2s-lGsdshEzmewdKtL",
            "_score": 1,
            "_source": {
                "user_id": 1,
                "field1": "example1",
                "field2": "example2",
                "field3": "example3"
            }

I think that the best way would be to associate every document with the user_id. The user would send for example GET request with body and authorization header with Token. I would use Token to extract id of the user for example in this way

                    key = request.META.get('HTTP_AUTHORIZATION').split()[1]
                    user_id = Token.objects.get(key=key).user_id

After this I would redirect his request to ES and only data that meet requirements and belongs to this user would be returned. Of course I could do this like shown above where I also add field user_id. For example I could use post_filter in this way:

To every request I would add something like this:

,
    "post_filter": {
        "match": {
            "user_id": 1
        }
    }

For example the user sends GET with body

{
    "query": {
        "regexp": {
            "tag": ".*example.*"
        }
    }
}

and I change this in my backend and redirect request to ES with body:

{
    "query": {
        "regexp": {
            "tag": ".*example.*"
        }
    },
    "post_filter": {
        "match": {
            "user_id": 1
        }
    }
}

but it doesn't seem to me that including this field in _source is a good idea. I am almost sure that it can be solved in a more optimal way than post_filtering. I see a lot of information about authorization in ES however I can’t find how can I associate document with user_id and then search only his documents without post_filtering. Any ideas?

[whiterabbit]

My current solution looks in they way shown below however as I mentioned I believe that it is not optimal way. If anyone has an idea how can I solve this in the way described above I will be grateful for help.

I send for example

{
  "query": {
    "regexp": {
      "tag": ".*test.*"
    }
  }
}

In Django backend I just do

        key = request.META.get('HTTP_AUTHORIZATION').split()[1]

        user_id = Token.objects.get(key=key).user_id

        body = json.loads(request.body)

        body['post_filter'] = {"match": {"user_id": user_id}}

        res = es.search(index="pictures", doc_type="picture", body=body)

        output = []

        for hit in res['hits']['hits']:
            output.append(hit["_source"])

        return Response(
            {'output': output},
            status=status.HTTP_200_OK)

[gabriel_tessier]

Hi,

Not a solution but a proposition that may help depend on your context, but I didn't use is alias filtering
https://www.elastic.co/guide/en/elasticsearch/reference/7.2/indices-aliases.html#filtered

You can generate your alias when you save your user on signals call:
https://docs.djangoproject.com/en/2.2/topics/signals/

As you load your user_id before making you request you can overwrite the index name.
index_name = 'pictures_%s' % (user_id)

res = es.search(index=index_name, doc_type="picture", body=body)

Another remark if you don't use alias, I think it's more safe to use term instead of match as term is exact match.
https://www.elastic.co/guide/en/elasticsearch/reference/7.2/query-dsl-term-query.html

and add the term in a bool must query as post filter is not applied on aggregations.
https://www.elastic.co/guide/en/elasticsearch/reference/7.2/search-request-post-filter.html

Hope it can help.