Hi @sgasse Couldn't help but poking in (interesting topic)...
and welcome to the community...
Document and Field level security is a platinum feature see here, but yes you should be OK to trial this.
So good news I just tested and for me it worked as expected in cloud with platinum license.
So I think we just need dig in a bit more.
Perhaps we should try a term query .... which can only match exact terms. its possible your match query is a full text query is is matching more than you think as from the Dev Tools it only returns the top 10 results.
"query": {
"term": {
"project.keyword": {
"value": "my_project"
}
}
}
Here is my example
In my example I have many apps that are in a field cloudfoundry.app.name that field is a keyword if it was using the default mapping I would use cloudfoundry.app.name.keyword
POST /_security/api_key
{
"name": "my-api-key",
"expiration": "1d",
"role_descriptors": {
"role-a": {
"cluster": [
"all"
],
"index": [
{
"names": [
"filebeat-*"
],
"privileges": [
"read"
],
"query": {
"term": {
"cloudfoundry.app.name": {
"value": "spring-music"
}
}
}
}
]
}
}
}
Result
{
"id" : "sadfasdfkBsdfsdfl0p6VzfPd",
"name" : "my-api-key",
"expiration" : 1620139846091,
"api_key" : "asdadadsasdRde14awKNAgRZA"
}
converted to base64
echo -n "sadfasdfkBsdfsdfl0p6VzfPd:asdadadsasdRde14awKNAgRZA" | base64
then ran some queries
curl -H "Authorization: ApiKey sadfsdfsadfsadf2VnpmUGQ6Ukl6WTN6RmhSZGUxNGF3S05BZ1JaQQ==" -H "Content-Type: application/json" -d '{"size":3,"query":{"term":{"cloudfoundry.app.name":{"value":"spring-music"}}}}' https://myelasticurl.es.us-west1.gcp.cloud.es.io/filebeat-7.11.2-2021.04.28-000011/_search | jq
Worked as expected I got results for spring-music
Then I specifically tried to get results that we not matched the the role query i.e. search for other app specificallyy
sbrown$ curl -H "Authorization: ApiKey asdfsadfasdfasdfMHA2VnpmUGQ6Ukl6WTN6RmhSZGUxNGF3S05BZ1JaQQ==" -H "Content-Type: application/json" -d '{"size":3,"query":{"term":{"cloudfoundry.app.name":{"value":"scheduler-200ms"}}}}' https://myelasticurl.es.us-west1.gcp.cloud.es.io/filebeat-7.11.2-2021.04.28-000011/_search | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 242 100 161 100 81 540 271 --:--:-- --:--:-- --:--:-- 812
{
"took": 19,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}
ceres-2:filebeat-7.11.2-linux-x86_64 sbrown$
Then I tried a match_all and just got the results I would expect
ceres-2:filebeat-7.11.2-linux-x86_64 sbrown$ curl -H "Authorization: ApiKey sadfsadfsadfVnpmUGQ6Ukl6WTN6RmhSZGUxNGF3S05BZ1JaQQ==" -H "Content-Type: application/json" -d '{"size":3,"query":{"match_all":{}}}' https://myelasticurl.es.us-west1.gcp.cloud.es.io/filebeat-7.11.2-2021.04.28-000011/_search | jq
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6256 100 6221 100 35 19440 109 --:--:-- --:--:-- --:--:-- 19489
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": 1,
"hits": [
{
"_index": "filebeat-7.11.2-2021.04.28-000011",
"_type": "_doc",
"_id": "MLbKGXkB_INU45ioej7I",
"_score": 1,
"_source": {
"agent": {
"hostname": "e8b823dc-4c45-4a00-7db7-95a4",
"name": "e8b823dc-4c45-4a00-7db7-95a4",
"id": "3e72667d-dbb0-46f1-a3cf-5460ee786dd9",
"type": "filebeat",
"ephemeral_id": "b3cf31bd-349c-42f6-b81c-b1ddaaab1cd9",
"version": "7.11.2"
},
"message": "2021-04-28 18:42:49.910 INFO 20 --- [nio-8080-exec-1] o.c.samples.music.web.AlbumController : Deleting album 7a41e2d3-689c-4f4f-a028-62ff5e30f10b",
"input": {
"type": "cloudfoundry"
},
"@timestamp": "2021-04-28T18:42:49.910Z",
"ecs": {
"version": "1.6.0"
},
"stream": "stdout",
"host": {
"name": "e8b823dc-4c45-4a00-7db7-95a4"
},
"cloudfoundry": {
"app": {
"name": "spring-music",
"id": "ec3e9ff9-ff03-412e-98a9-40ed8d537241"
},
...
Hope This Helps...