@timestamp error

Elastic Stack Beats
1

I have filebeat sending Apache logs to logstash which is located on another server.

Logstash shows the following error:

{:timestamp=>"2015-12-16T15:32:05.053000+0000", :message=>"Beats input: unhandled exception", :exception=>#<TypeError: The field '@timestamp' must be a (LogStash::Timestamp, not a String (2015-12-16T15:31:03.326Z)>

I have verified the connection between the servers and, as logstash is generating this error, it must be receiving data from filebeat.

running filebeat with -d publish I can see:

2015/12/16 15:18:35.584066 publish.go:100: DBG  Publish: {
  "@timestamp": "2015-12-16T15:18:35.324Z",
  "beat": {
      "hostname":  
      ....

I have minimal conf for filebeat:

############################# Filebeat ######################################

filebeat:
  # List of prospectors to fetch data.
  prospectors:
    # Apache logs
    -
      paths:
        - /var/log/apache2/*.log

############################# Output ##########################################

output:
  logstash:
    enabled: true
    hosts: ["xx.xx.xx.xx:5043"]

and minimal conf for log stash:

input {
    beats {
        port => 5043
    }
}
output {
    elasticsearch {
        hosts => "localhost:9200"
    }
}

I have tried using the date match filter (with various formats) in the logstash conf ie:

date {
      match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
}

I'm all google out now, anyone know how I can fix this?

Thanks - Paul

2

which logstash version and plugin have you installed? I think it was fixed in logstash-input-beats plugin version 0.9.4.

3

Thanks for the reply Steffen and yes, it was a version issue, I was running 0.9.2

I found this post which gave me a clue and I tried updating the plugin to the latest version. This led to dependency issues so I installed the latest version of logstash (all plugins) over the top and it worked.

Thanks - Paul