Create a new indices for each random directory

That's simple enough. Parse [log][file][path] using grok to extract the directory name (e.g. see this thread).

However, creating an index per day per directory can be a performance issue in elasticsearch because it can create a large number of small indexes/shards. (An index is stored in one or more shards.)

The documentation recommends using multi-gigabyte shards.