Hello,
I have an elasticsearch output working without any problem
elasticsearch {
hosts => [ "${LOGS_ELASTICSEARCH_CLUSTER}:443" ]
ssl => true
index => "cwl-%{+YYYY.MM.dd}"
document_type => "log"
}
I wanted to redirect some type of logs to a new elasticsearch index but I'm getting this error
[2019-09-03T14:39:00,242][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<LogStash::Error: timestamp field is missing>, :backtrace=>["org/logstash/ext/JrubyEventExtLibrary.java:202:in `sprintf'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/common.rb:172:in `event_action_params'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/common.rb:48:in `event_action_tuple'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/common.rb:42:in `multi_receive'", "org/jruby/RubyArray.java:2414:in `map'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.3.8-java/lib/logstash/outputs/elasticsearch/common.rb:42:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:47:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:420:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:419:in `output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:365:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:330:in `start_workers'"]}
No newer events found at the moment. Retry.
The new output is:
if "http" in [tags] {
elasticsearch {
hosts => [ "${LOGS_ELASTICSEARCH_CLUSTER}:443" ]
ssl => true
index => "cwl-http-%{+YYYY.MM.dd}"
document_type => "log"
}
}
else {
elasticsearch {
hosts => [ "${LOGS_ELASTICSEARCH_CLUSTER}:443" ]
ssl => true
index => "cwl-%{+YYYY.MM.dd}"
document_type => "log"
}
}
I modified my filter like this
### HTTP logs
if [logger_name] == "HttpLoggerService" {
mutate {
add_tag => [ "http" ]
}
}
any thing wrong in my configuration ?
Thanks!