Index design for hardware test results with custom doc type. Need help

[maddysqlpro]

Hi,

I am very new to ES. We are planning to store hardware test results in the ES instance. We use this results for visualization.
Each test run produces 10 json files, each file with different structure.
Some files will be small and some files might be big.
The size of all these result files is around 5 to 10 MB for each test run.

So my question is,
Since _doc types are being removed in future releases of ES, is it good to have once single index for the test run, with a custom key "type" to differentiate each file. Is it a good option ? or is it better to have separate indexes for each file since the structure is different?

On what basis, we should decide on the number of indexes. Please guide me.

Thanks

[mjunaidmuzammil (Junaid)]

Since multiple doc types per index are not going to be support moving on, it is better to use a single type per index. In order to differentiate between files, you can either use a separate index per file or use a custom field (such as file_id) for differentiating between files in the ingested JSON doc.

In my opinion, the decision is based on the data volume in an index. Too small index shards are not recommended and optimally a shard should fall in the 5G - 40G limit.