Support for indexing Windows fileshare

Elastic Stack Elasticsearch
1

Hi,

Is it possible to use Elasticsearch to index WIndows filshares? If so: are search results security trimmed based on the user executing the query?

2

Elasticsearch doesn't index any file systems at all. It indexes what you feed it. If you use an external program to gather file system contents and pass to Elasticsearch you can probably build something that takes ACLs into account but it probably won't be obvious.

1 Like
3

Thanks a lot, Magnus! Excately what I needed to know.

4

You can have a look at https://github.com/dadoonet/fscrawler. Might help.

1 Like
5

Thank you for the tip, David! Does fscrawler support security trimming based on ACL out of the box or is this something I must implement myself?

6

No. I believe it will simply fail by going in given dirs and won't index the content then. Never tested but I'll be happy to have a feedback or even better a PR on the doc if useful :slight_smile:

7

Thanks for such quick responses here! :slight_smile:

Just to be clear: I was thinking about a scenario where the crawler account have full read permission to everything on the file share. Does fscrawler then look at the ACL for every file and trim search results based on permissions? I.e. if user Bob has access to file A,B but not C on the filshare, he should not get file C in search results.

8

Ha sorry. I misunderstood.

So the attributes_support option collects such information about user and groups: https://github.com/dadoonet/fscrawler#adding-file-attributes

But IIRC the code only users are added for Windows FS.

Then, you need to filter out the results in your queries depending on that information.
Of course X-Pack and its security module helps simplifying that. It's a commercial plugin.

That's not exactly what you are looking for. I don't know if something is doable on FSCrawler side but I'll be happy if you have any idea to solve such a use case with it. Feel free to open an issue in the project with any proposal you can have. :stuck_out_tongue:

1 Like
9

Thanks again, David. This is really helpfull. I'll look into it and see if this is an advantageous strategy compared to other search solutions.

10

I should have mentioned this earlier, but the solution must support free text search - which it seems like fscrawler doesn't (correct me if I'm wrong).

11

Do you mean fulltext search?

12

Yes, sorry, I meant full-text search. "Free text search" is a little too directly translated from norwegian.

13

:smile:

So why do you think it's not supported?

14

I couldn't find anything about it, but that's probably because I used the wrong term for it... :smirk: I do see that it's supported here: https://github.com/dadoonet/fscrawler#simple-search. Is there a complete list somewhere showing supported file formats?

15

This list: http://tika.apache.org/1.14/formats.html

I believe I should link from the README to this doc. (PR are warmly welcomed :stuck_out_tongue: )

16

Yes, that would be nice. Thanks again, David! :relaxed:

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.