# Change @timestamp to time from actual log file **URL:** \ **Category:** Logstash\ **Created:** [September 11, 2017, 7:43am UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011 "2017-09-11T07:43:33Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Ferrow](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Ferrow](https://discuss.elastic.co/u/Ferrow)\ **Post date:** [September 11, 2017, 7:43am UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/1 "2017-09-11T07:43:33Z")
Hello everybody.. Im trying to send log files from an Ubuntu client to my ES server. That seems to work fine although I can not seem to get the time field right. (want the time stamp from the log file, not when it has been run by ES!) also I would like to know how I exclude fields like fx "host" and "path" I have made a .conf and a matching pattern file, witch I will post here, as well as a screenshot of the output shown in Kibana.. ![output](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b76d978dc1c9a6d4661c6a97003fdd84aa719e6a.png) Thanks In advance ---
**Author:** ![Ferrow](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Ferrow](https://discuss.elastic.co/u/Ferrow)\ **Post date:** [September 11, 2017, 7:44am UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/2 "2017-09-11T07:44:01Z")
LOGSTASH CONF input { file { type =\> "CISCO\_ASA\_FIREWALL" exclude =\> "_.gz" path =\> "/home/lasse/Skrivebord/3asa5525/messages-2015-10-30_" # for live data it is better to start data at "end". (fx streaming) start\_position =\> "beginning" } } filter { if [type] == "CISCO\_ASA\_FIREWALL" { grok { patterns\_dir =\> "/home/lasse/Skrivebord/patterns/CISCO\_ASA\_PATTERN.txt" match =\> ["message", "%{CISCO\_TAGGED} %{GREEDYDATA:cisco\_message}"] } ``` syslog_pri { #what is this, and why do I need it? } date { match => ["timestamp", "MMM dd HH:mm:ss", "MMM d HH:mm:ss", "MMM dd yyyy HH:mm:ss", "MMMM d yyyy HH:mm:ss"] target => "timestamp" } if "_grokparsefailure" not in [tags] { mutate { rename => ["cisco_message" , "message"] remove_field => ["timestamp"] } } grok { patterns_dir => "/home/lasse/Skrivebord/patterns/CISCO_ASA_PATTERN.txt" match => [ "message" , "%{CISCOASA106021}", "message" , "%{CISCOASA106001}", "message" , "%{CISCOASA106006_106007_106010}", "message" , "%{CISCOASA106017}", "message" , "%{CISCOASA106020}", "message" , "%{CISCOASA106014}", "message" , "%{CISCOASA305006}", "message" , "%{CISCOASA313001_313004_313008}", "message" , "%{CISCOASA710003_1}", "message" , "%{CISCOASA710003}", "message" , "%{CISCOASA746005}", "message" , "%{CISCOASA746006}", "message" , "%{CISCOASA106023}", "message" , "%{CISCOASA313004}", "message" , "%{CISCOASA313005}", "message" , "%{CISCOASA313009}", "message" , "%{CISCOASA338004_338008}", "message" , "%{CISCOASA410001}", "message" , "%{CISCOASA400028}", "message" , "%{CISCOASA400037}", "message" , "%{CISCOASA402117}", "message" , "%{CISCOASA402119}", "message" , "%{CISCOASA405104}", "message" , "%{CISCOASA419001}", "message" , "%{CISCOASA419002}", "message" , "%{CISCOASA500004}", "message" , "%{CISCOASA507003}", "message" , "%{CISCOASA733100}", "message" , "%{CISCOASA752010}", "message" , "%{CISCOASA752016}", "message" , "%{CISCOASA106100}", "message" , "%{CISCOASA106101}", "message" , "%{CISCOASA111007}", "message" , "%{CISCOASA111008}", "message" , "%{CISCOASA111010}", "message" , "%{CISCOASA305013}", "message" , "%{CISCOASA321001}", "message" , "%{CISCOASA502103}", "message" , "%{CISCOASA713041}", "message" , "%{CISCOASA713049}", "message" , "%{CISCOASA713050}", "message" , "%{CISCOASA713073}", "message" , "%{CISCOASA713074}", "message" , "%{CISCOASA713075}", "message" , "%{CISCOASA713076}", "message" , "%{CISCOASA713119}", "message" , "%{CISCOASA713120}", "message" , "%{CISCOASA713130}", "message" , "%{CISCOASA713184}", "message" , "%{CISCOASA713228}", "message" , "%{CISCOASA713902}", "message" , "%{CISCOASA713904}", "message" , "%{CISCOASA713257}", "message" , "%{CISCOASA713904}", "message" , "%{CISCOASA725001}", "message" , "%{CISCOASA725002}", "message" , "%{CISCOASA725003}", "message" , "%{CISCOASA752004}", "message" , "%{CISCOASA725006}", "message" , "%{CISCOASA725007}", "message" , "%{CISCOASA110002}", "message" , "%{CISCOASA110003}", "message" , "%{CISCOASA106015}", "message" , "%{CISCOASA106016}", "message" , "%{CISCOASA113004}", "message" , "%{CISCOASA113005}", "message" , "%{CISCOASA113008}", "message" , "%{CISCOASA113009}", "message" , "%{CISCOASA302010}", "message" , "%{CISCOASA302013_302014_302015_302016}", "message" , "%{CISCOASA302020_302021}", "message" , "%{CISCOASA303002}", "message" , "%{CISCOASA305011}", "message" , "%{CISCOASA607001}", "message" , "%{CISCOASA602101}", "message" , "%{CISCOASA602303_602304}", "message" , "%{CISCOASA605005}", "message" , "%{CISCOASA607001}", "message" , "%{CISCOASA611101}", "message" , "%{CISCOASA611103}", "message" , "%{CISCOASA622001}", "message" , "%{CISCOASA713172}", "message" , "%{CISCOASA713905_1}", "message" , "%{CISCOASA713905}", "message" , "%{CISCOASA734001}", "message" , "%{CISCOASA737006}", "message" , "%{CISCOASA737029}", "message" , "%{CISCOASA737031}", "message" , "%{CISCOASA111009}", "message" , "%{CISCOASA609001_609002}", "message" , "%{CISCOASA710001_710002_710003_710005_710006}", "message" , "%{CISCOASA713236}", "message" , "%{CISCOASA713906}", "message" , "%{CISCOASA715036_715046_715047_715075}" ] } } } ``` output { stdout { codec =\> plain } file { path =\> "/home/lasse/Skrivebord/ASA\_archive\_1/%{type}/%{YYYY-MM}/%{type}-%{+YYYY-MM-dd}.log" } elasticsearch { hosts =\> ["[http://192.168.128.46:9200](http://192.168.128.46:9200)"] document\_type =\> "text" index =\> "clientone" user =\> elastic password =\> Infowise } } ---
**Author:** ![Ferrow](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Ferrow](https://discuss.elastic.co/u/Ferrow)\ **Post date:** [September 11, 2017, 7:45am UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/3 "2017-09-11T07:45:08Z")
#== Cisco ASA == HOSTNAME \b(?:[\_0-9A-Za-z][\_0-9A-Za-z-]{0,62})(?:.(?:[_0-9A-Za-z][0-9A-Za-z-]{0,62}))\*(.?|\b) CTIMESTAMP %{YEAR}-%{MONTHNUM2}-%{MONTHDAY}[T]%{HOUR}:?%{MINUTE}(?::?%{SECOND})(?:.[0-9]+)?%{ISO8601\_TIMEZONE} CISCO\_TAGGED %{CTIMESTAMP:timestamp}( %{SYSLOGHOST:host})? %{CISCO\_TAG:ciscotag}: CISCO\_ASA\_TAGGED %{CTIMESTAMP:timestamp}( %{SYSLOGHOST:host})? %{CISCO\_ASA\_TAG:ciscotag}: CISCO\_CLASS [0-9]{3} CISCO\_STRUC [A-Z0-9]+ CISCO\_TAG %{CISCO\_STRUC:cisco\_facility}-%{INT:cisco\_severity}-%{CISCO\_STRUC:cisco\_mnemonic}|WLC[0-9]+ CISCO\_ASA\_TAG %[A-Z0-9_]+-%{INT:cisco\_severity}-%{CISCO\_CLASS:cisco\_class}[0-9]{3} # Common Particles CISCO\_ASA\_ACTION Built|Teardown|Deny|Denied|denied|requested|permitted|received|denied by ACL|discarded|est-allowed|Dropping|dropping|created|deleted|SENDING|RECEIVED|monitored|dropped|terminated|Rejected CISCO\_ASA\_REASON AAA failure|Duplicate TCP SYN|TCP Reset-O|Failed to locate egress interface|Invalid transport field|No matching connection|DNS Response|DNS Query|(?:%{WORD}\s\*)\* CISCO\_ASA\_DIRECTION Inbound|inbound|Outbound|outbound CISCO\_ASA\_INTERVAL first hit|%{INT}-second interval CISCO\_ASA\_XLATE\_TYPE static|dynamic .... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 11, 2017, 10:01am UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/4 "2017-09-11T10:01:18Z")
You have a date filter that attempts to parse the `timestamp` field, but does your event have a `timestamp` field? And what does it look like? Until you're done debugging this I suggest you don't remove the `timestamp` field with that mutate filter since you're tampering with the evidence. ---
**Author:** ![Ferrow](https://avatars.discourse-cdn.com/v4/letter/f/7bcc69/32.png) [@Ferrow](https://discuss.elastic.co/u/Ferrow)\ **Post date:** [September 11, 2017, 10:33am UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/5 "2017-09-11T10:33:24Z")
Hi Magnus not sure what your saying /asking me 😃 The time in the actual log file is under the "message" tag. (see screenshot) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 11, 2017, 1:55pm UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/6 "2017-09-11T13:55:06Z")
> The time in the actual log file is under the "message" tag. (see screenshot) I know. And you have a grok filter that parses the message to extract various fields, including `timestamp`. Then you have a date filter that parses `timestamp` and stores the result back into `timestamp`. After that you delete the `timestamp` field with a mutate filter. That's clearly not a good idea. On top of that, the field that Kibana uses as the event timestamp is by default `@timestamp`, not `timestamp`. That is, unless you've changed your index pattern configuration in Kibana it won't matter if you fix your Logstash filters so that they write to the `timestamp` field because Kibana won't use it. Suggestions: - Use `@timestamp` as your timestamp field. It's possible to change this and use `timestamp` or whatever, but don't do that until you understand things better. - Remove the `target => "timestamp"` option in your date filter. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 9, 2017, 1:55pm UTC](https://discuss.elastic.co/t/change-timestamp-to-time-from-actual-log-file/100011/7 "2017-10-09T13:55:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.