Thank you very much for your advises! Great! Brilliant! Your advises is exactly i am looking for.
So, last portion of questions inline.
A common pattern for a centralised logging solution is to gather logs using Beats and then enrich and parse it in Logstash and/or the new Elasticsearch ingest node feature.
ER: Fine, i'll will use Beats or Logstash shippers itself.
These log r records are then typically indexed into Elasticsearch, where time-based indices generally are used.
ER: Great. How i can configure Elastcsearch to create only time-based indices properly? Where i can config it? For example, i have a lot of application logs every day. ES will create indices for every day? If yes where i can config shippers or forwarders to put log into appropriate index?
Data that is similar and have the same retention period is often kept in the same index, so you may end up with a time-based index per application. Each time-based index is usually associated with an index template. Types can be used sparingly, but in most cases you can just have a single type.
ER: Great and acceptable. What is standard retention for logs?
In almost all cases where data is immutable and not updated, Elasticsearch is left to assign the document ID. As Elasticsearch is a search engine, it is easy to search based on the content of the logs, and I never see the key used for search.