Replacing @timestamp with a custom field does not occur


I am currently using the HTTP-IN plugin to receive HTTP PUT requests from various systems.

The systems have a field called CreationTime which I am trying to use instead of the @timestamp field.

I have read few threads and none of the fixes seemed to help and the @timestamp is not being replaced.

The CreationTime test value is:

21/10/2016 15:13:06

My configuration file contents:

input {
   port => 8080
filter {
   date {
        match => ["CreationTime", "dd/MM/YYYY HH:mm:ss"]
        target => "@timestamp"
output {
  elasticsearch {
    hosts => ""
    codec => "json"
    index => "logstash-%{+YYYY.MM.dd}"
  stdout { codec => rubydebug }

If someone could give me some pointers then I would very much appreciate that.

Your date filter looks correct if your events indeed have a CreationTime field containing a date in that format. Please give an example of an event, e.g. one captured by your stdout { codec => rubydebug } output.

@magnusbaeck thanks for your reply, the below is the output I am getting in logstash, as you can see the @timestamp is not replaced by CreationTime but you may be able to point out if something is incorrect:

> JobId:752fc480
> Exception: - 
> StatusDetails:None  
> Errors: - 
> Output: - 
> Status:Completed 
> Warnings: - 
> EndTime:21/10/2016-15:16:36 
> CreationTime:21/10/2016 15:13:06  
> runOn: - 
> Verbose: - 
> @version:1 
> @timestamp:October 21st 2016, 15:23:30.275 
> host:<IP>
> headers.request_method:PUT 
> headers.request_path:/azure/jobs/ 
> headers.request_uri:/azure/jobs/ 
> headers.http_version:HTTP/1.1 
> headers.http_user_agent:Mozilla/5.0 (Windows NT; Windows NT 6.1; en-GB) WindowsPowerShell/5.0.10586.117 
> headers.content_type:application/json 
> headers.http_host:<somehost>` 
> headers.content_length:673 
> _id:AVfnoBak2aiUzdk-hbNJ 
> _type:logs 
> _index:logstash-2016.10.21

Is that the payload you are receiving? What is the output from the Logstash stdout output using the ruby debug codec?

This is exactly what I get in Kibana, which I am assuming is stored in Elastic search via logstash,

If that is the raw message, you have not parsed out any fields, especially not the CreationTime that the date filter expects. You probably need to apply a grok filter or possible a kv filter to parse out the fields in Logstash before you try to use the date filter.

Thank you for your help. I am quite a novice in all honesty. I am not sure on what the grok filter is and what is its function and how to implement it.