Make Elasticsearch understand date/time field Nginx/apache logs

Elastic Stack Elasticsearch
1

I am sending my logs from apache and nginx to elasticsearch via rsyslog.

Parsing to json:

version=2
rule=:%remote_addr:word% %ident:word% %auth:word% [%@timestamp:char-to:]%] "%method:word% %request:word% HTTP/%httpversion:float%" %status:number% %requesttime:float% "%referrer:char-to:"%" "%agent:char-to:"%"

.

template(name="apache-nginx" type="list"){
  property(name="$!all-json")
}

The problem is nginx logs like: [14/Jan/2020:08:48:23 +0100] so when send to elastic search it does not recognise it as date/time

Any tips tricks to fix it ? (no filebeats.) Has to be done with rsyslog/elasticsearch.

2

You can use an ingest pipeline so parse this as a proper date. Check out the ingest node in general, and the grok processor specifically, which also has predefined patterns for apache logs, that you might be able to reuse.

--Alex

3

Ty ! love the flexibilityof the system but i'll have to study it more thanks for pointing me into correct direction.

For now i made all json template to import everything from rsyslog downside is can't use the premade dashboard oh wel all in time.