What is the best way to index of documents when aggregating logs?

Elastic Stack Elasticsearch
1

I want to use on elasticsearch for saving access log datas from WEB APP or Database

Web app logs and Database logs have difference fields respectively.
for example,
was-access log :
"url" : xxx,
"session-id" : xxx,
"status-code" : xxx,
...
db-access log :
"query" : xxx
...

So, i thought 2 ways to save access logs.

  • first :
    (access datas from WEB APP)
    index : was-access
    type : logs
    fields..
    (access datas from DB)
    index : db-access
    type : logs
    fields..

  • second :
    index : 2017-08-20 (i.e created index everyday)
    type : was-access || db-access
    fields..

what is the best way to save datas?

2

If they are different formats, then different indices is the best option.

3

thx for reply : )

that is my second focus about my question

The first is that it is best option to save index for every day.

for example
first option :
index : 2017-08-21
type : was
format..

index : 2017-08-20
type : was
format..

second option :
index : was
format..

what is the better option ??

4

We recommend time based indices, but daily or not depends on the volumes that you are ingesting.

5

thx for reply :slight_smile:
It will be great idea to implement my app :smiley: