I would like to know the best practices when designing data model for indices in elasticsearch. We have a system in which we will need to pull data from cloud storage systems(eg:dropbox),social media(eg:twitter),articles from web etc.
The design issue am facing is that each type of docs have different fields/mapping.
Some of the options I've explored.
Use different types under single index since it is having different doc structure(eg : elastic types facebook,twitter,drobpox,googledrive etc.).This will tend to add lot of types under an index.
Use dynamic mapping for index to add fields whenever necessary. And use same mapping for all docs. In this case,most of the fields will be null.(eg: for a elastic document for storage social media specific fields will be null).
Use different indices for different data points. In this case, there will be lots of indices.
I would like to know which of these options will be best in our use-case. My consideration is search and indexing performance and scalability.