# Using 7.1 Can't set @timestamp from message

**URL:** <https://discuss.elastic.co/t/using-7-1-cant-set-timestamp-from-message/184205>\
**Category:** Logstash\
**Created:** [June 4, 2019, 2:17pm UTC](https://discuss.elastic.co/t/using-7-1-cant-set-timestamp-from-message/184205 "2019-06-04T14:17:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pgervais](https://avatars.discourse-cdn.com/v4/letter/p/65b543/32.png) [@pgervais](https://discuss.elastic.co/u/pgervais)\
**Post date:** [June 4, 2019, 2:17pm UTC](https://discuss.elastic.co/t/using-7-1-cant-set-timestamp-from-message/184205/1 "2019-06-04T14:17:59Z")

</div>

I have a simple logstash definition file as shown below.  
I can parse my message date properly with no dateparse error shown.  
I cannot set the @timestamp with the ObsDate from my message input stream.  
As well, the ruby output shows all my csv field names in lower case which is not the case in the sql db.  
I have even tried to remove the Target statement in date filter as the documentation states its not required and becomes the default when not specified.

All output are essentially the same as:  
{  
"avgnetworktraffickbsec" =\> 0.0,  
"@timestamp" =\> 2019-06-04T14:07:17.536Z,  
"obsdate" =\> 2019-04-30T04:00:00.000Z,  
"totalcpuloadmips" =\> 118.41,  
"obshour" =\> 17,  
"@version" =\> "1"  
}

# This config file is used to parse the sql data extracted from the db entry:CB\_EZ14A

input {  
jdbc {  
jdbc\_driver\_library =\> "/home/pxg110/sqljdbc\_4.2/sqljdbc42.jar"  
jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver"  
jdbc\_user =\> "XXXXXXXXXXX"  
jdbc\_password =\> "XXXXXXXXXXXXX"  
jdbc\_connection\_string =\> "jdbc:sqlserver://SD01CUVDB0521.OMEGA.DCE-EIR.NET:1433;"  
statement =\> "SELECT ObsDate,ObsHour,TotalCPULoadMIPS,AvgNetworkTrafficKBsec FROM smg.dbo.smgdata WHERE (TransClass='CB\_EZ14A') AND (ObsDate \>= CONVERT(DATETIME, '2019-04-01', 102)) AND (ObsDate \<= CONVERT(DATETIME, '2019-04-30', 102)) ORDER BY ObsDate;"  
}  
}  
filter {  
fingerprint {  
source =\> "message"  
target =\> "[@metadata][fingerprint]"  
method =\> "SHA1"  
key =\> "Tue\_Jun\_2019\_08\_25\_CB\_EZ14A"  
base64encode =\> true  
}

# defines all the fields to be found in the csv file.

```
    csv {
            separator => " "
            columns => [
                    "ObsDate",
                    "ObsHour",
                    "TotalCPULoadMIPS",
                    "AvgNetworkTrafficKBsec"
            ]
            convert => {
                    "ObsDate" => "date"
                    "ObsHour" => "integer"
                    "TotalCPULoadMIPS" => "float"
                    "AvgNetworkTrafficKBsec" => "float"
            }
    }
    date {
            match => ["ObsDate", "ISO8601"]
            target => "@timestamp"
   }
    mutate {
            remove_field => ["ObsDate", "ObsHour"]
    }

```

}

output {

# elasticsearch {

# action =\> "index"

# hosts =\> "localhost:9200"

# document\_id =\> "%{[@metadata][fingerprint]}"

# index =\> "Tue\_Jun\_2019\_08\_25\_CB\_EZ14A"

#}  
stdout {codec =\> rubydebug}

# stdout {}

}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 4, 2019, 6:09pm UTC](https://discuss.elastic.co/t/using-7-1-cant-set-timestamp-from-message/184205/2 "2019-06-04T18:09:27Z")

</div>

> [@pgervais](#):
>
> "obsdate" =\> 2019-04-30T04:00:00.000Z,

Note that the value of obsdate does not have quotes around it, so it is not a string, but was already converted to a LogStash::Timestamp by the jdbc filter. A date filter cannot parse that. This is a [known issue](https://github.com/logstash-plugins/logstash-filter-date/issues/95) and the workaround is to mutate+convert the field to a string.

---

<div class="post-metadata">

**Author:** ![pgervais](https://avatars.discourse-cdn.com/v4/letter/p/65b543/32.png) [@pgervais](https://discuss.elastic.co/u/pgervais)\
**Post date:** [June 4, 2019, 7:56pm UTC](https://discuss.elastic.co/t/using-7-1-cant-set-timestamp-from-message/184205/3 "2019-06-04T19:56:34Z")

</div>

Badger , Thanks for the hint.  
I have made the following minor mods based on the "known issue" provided as shown below.

```
   mutate {
            convert => { "obsdate" => "string" }
    }

```

Note: It only worked when I renamed my SQL fields name to lower case. EX: ObsDate =\> obsdate.  
I suspects this is a jdc side effect.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 2, 2019, 7:56pm UTC](https://discuss.elastic.co/t/using-7-1-cant-set-timestamp-from-message/184205/4 "2019-07-02T19:56:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.