Error/Info log separation

Elastic Stack Logstash
#1

Hi,
We have some log files where INFO/ERROR/WARN are recorded in single file.

  1. What is the approach generally in ELK to handle such files?
  2. What if we want error, info, warn logs separately under separate type mappings?
  3. is this very heavy or expensive operation to filer out and send under diff type/index? how to do that?
  4. OR we have to separately logs those at application's level using some logginf framework before we get it into central logging?

br,
Sunil

#2

You could create a separate index, or you could put it into the same index and have put the error level into it's own field.
If you want separate mappings you would probably be better off doing separate indexes.

Logstash has a way of dynamically creating separate indexes.

For example, let's say you put the logging level into a field called "LogLevel". In your Logstash output you could put something like the below. This will create a dynamic index with the loglevel included in the index name. For example, today's index would be "Logfiles-INFO-2016.04.11", "Logfiles-ERROR-2016.04.11", and so on.
You would probably want to do some verification that LogLevel doesn't ever get weird values in it, otherwise you might end up with some strange index names.

output {
    elasticsearch {
        hosts => "ElasticClusterName"
        index => "Logfiles-%{LogLevel}-%{+YYYY.MM.dd}"
    }
}
#3

FYI the Logstash forwarder is deprecated, you should move away from it ASAP.

#4