Best approach to apply Fine Grained Access Control to index documents

I have a service that apply fine grained access control to ES index document and I am facing some ES limitation / performance issue. For this reason I am not sure if my approach is correct, of if there is a better way to handle that. I am using ES 6.8

I have an Apollo JS server that run a query towards a Postgres DB gathering a list of all the document/resources the user is allowed to access. This is a list of strings and it it pretty long, about 58k strings. This is applied to the query adding a filter to the other filters that are part of the query, if any:

    IN: {
          ["auth_resource_path"]: [...resourceList],
        },

an example of a piece of the query body is:

query body:  {"size":0,"query":{"bool":{"must":[{"terms":{"auth_resource_path":["/programs/pcdc/projects/20210325/persons/person_156977/subjects/0x92DDDD10F2A2645764CE4E15619924CE","/programs/pcdc/projects/20210223/persons/person_156789/subjects/0xA4CE42BAEAFFD85A5A573F7C0488647D","/programs/pcdc/projects/20210223/persons/person_156791/subjects/0xE23B0F16F4B158D1A417B2B422AEB303","/programs/pcdc/projects/20210223/persons/person_156792/subjects/0xECCB3D6714826A61C09404348C2A9C1F","/programs/pcdc/projects/20210223/persons/person_156793/subjects/0xE24EE86EB9F21939CD169B92BCD78F34","/programs/pcdc/projects/20210223/persons/person_156794/subjects/0x51220822D576DDC4F1F9B68425E8005A","/programs/pcdc/projects/20210223/persons/person_156795/subjects/0x9F76342930F67A310A173B007E3256BD","/programs/pcdc/projects/20210223/persons/person_156796/subjects/0xDC7CFD23C465D23E38FE1245874B8863","/programs/pcdc/projects/20210223/persons/person_156797/subjects/0x6EE4D7DE109189B2C3A322F0D675D15F","/programs/pcdc/projects/20210223/persons/person_156798/subjects/0x3146E12F9120E91DE60FD2F15DCBE315","/programs/pcdc/projects/20210223/persons/person_156799/subjects/0xB6144F3315F8CC556E5C9F953DDABAD7","/programs/pcdc/projects/20210223/persons/person_156800/subjects/0x1CFFE383FB26D75741347B9D0A881EA1","/programs/pcdc/projects/20210223/persons/person_156801/subjects/0x224ACD5DFFFA0403F7CC3DE9278A11F7","/programs/pcdc/projects/20210223/persons/person_156802/subjects/0x4E55992122D9B32CBA7BEFCCCAD2526D","/programs/pcdc/projects/20210223/persons/person_156803/subjects/0x6AE372BFAB833154B799961B8A725B41","/programs/pcdc/projects/20210223/persons/person_156804/subjects/0x13F02A02FE0E1F8A892D43503DF9CD71","/programs/pcdc/projects/20210223/persons/person_156805/subjects/0xB14DDE5D87CB597DA6D97C89BE685074","/programs/pcdc/projects/20210223/persons/person_156806/subjects/0xF3F2708741598DB2CA39D8B501C09717","/programs/pcdc/projects/20210223/persons/person_156807/subjects/0x45BF61D24C666B3C9878EAA5335A7032","/programs/pcdc/projects/20210223/persons/person_156808/subjects/0x2CC7088D4E5A45691B17E750BE080FE7","/programs/pcdc/projects/20210223/persons/person_156809/subjects/0x3A8A95E30C07B394B8623196DB935704","/programs/pcdc/projects/20210223/persons/person_156810/subjects/0xB7A4047EC240A80AF6DDEA908D8B855B","/programs/pcdc/projects/20210223/persons/person_156811/subjects/0x05C74F58C1B84338607C318554707210","/programs/pcdc/projects/20210223/persons/person_156812/subjects/0x56F475FF2C671D510955E5AD69FAB00D","/programs/pcdc/projects/20210325/persons/person_156789/subjects/0x2B1D2E3C4648236211D982AA60BAC9BD","/programs/pcdc/projects/20210325/persons/person_156790/subjects/0xE23B0F16F4B158D1A417B2B422AEB303","/programs/pcdc/projects/20210325/persons/person_156791/subjects/0xECCB3D6714826A61C09404348C2A9C1F","/programs/pcdc/projects/20210325/persons/person_156792/subjects/0xE24EE86EB9F21939CD169B92BCD78F34","/programs/pcdc/projects/20210325/persons/person_156793/subjects/0x51220822D576DDC4F1F9B68425E8005A","/programs/pcdc/projects/20210325/persons/person_156794/subjects/0x9F76342930F67A310A173B007E3256BD","/programs/pcdc/projects/20210325/persons/person_156795/subjects/0xDC7CFD23C465D23E38FE1245874B8863","/programs/pcdc/projects/20210325/persons/person_156796/subjects/0x6EE4D7DE109189B2C3A322F0D675D15F","/programs/pcdc/projects/20210325/persons/person_156797/subjects/0x3146E12F9120E91DE60FD2F15DCBE315","/programs/pcdc/projects/20210325/persons/person_156798/subjects/0xB6144F3315F8CC556E5C9F953DDABAD7","/programs/pcdc/projects/20210325/persons/person_156799/subjects/0x1CFFE383FB26D75741347B9D0A881EA1","/programs/pcdc/projects/20210325/persons/person_156800/subjects/0x224ACD5DFFFA0403F7CC3DE9278A11F7","/programs/pcdc/projects/20210325/persons/person_156801/subjects/0x4E55992122D9B32CBA7BEFCCCAD2526D","/programs/pcdc/projects/20210325/persons/person_156802/subjects/0x6AE372BFAB833154B799961B8A725B41","/programs/pcdc/projects/20210325/persons/person_156803/subjects/0x13F02A02FE0E1F8A892D43503DF9CD71","/programs/pcdc/projects/20210325/persons/person_156804/subjects/0xB14DDE5D87CB597DA6D97C89BE685074","/programs/pcdc/projects/

When the 'resourceList' was about 10k items the query was slow but resolving, but now, with 58k items it runs out of time most of the times.

Do you have any suggestion on the best way to solve this issue? My goal is to give every user their specific list of documents they are allowed to access.

Welcome to our community! :smiley:

Might it make more sense to create documents in Elasticsearch that are user centric then? That way you can just query by userid and return things, rather than trying to have a huge query.

Hi @warkolm Thank you for welcoming me.

Wouldn't doing that mean to run a new ETL every time a user registers? Like every time a new user registers to use the platform I would have to load the entire new set of data with the id of the user, not sure how sustainable that would be.