# New user confused about odd @Timestamp **URL:** \ **Category:** Elasticsearch\ **Tags:** docker\ **Created:** [September 24, 2022, 7:58pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088 "2022-09-24T19:58:23Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![nfored](https://avatars.discourse-cdn.com/v4/letter/n/9de0a6/32.png) [@nfored](https://discuss.elastic.co/u/nfored)\ **Post date:** [September 24, 2022, 7:58pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088/1 "2022-09-24T19:58:23Z")
I have 1 logs stash server that host 8 end points, outputs to elasticsearch 6 of the index correctly log @timestamp in UTC and life is lovely two index @timestamp is in the local timezone. This is the time stamp shown in json form in kibana. I am greatly confused as all 8 index work as expected on normal vm deployment with the same longstash config, but moving to k8 I did have to combine input and output into the same file and use multiple pipelines in the k8 vs multiple inputs and one output with one pipeline on normal vm. please let me know what data to show that would help in seeking help. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [September 25, 2022, 3:52pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088/2 "2022-09-25T15:52:17Z")
You need to provide more context and information as it is not clear what is your issue. How are you running Logstash? Are you using `pipelines.yml`? Share your `pipelines.yml` file. What does your configuration looks like? Do you have a `date` filter in the configuration? Which is the timezone in your logs? What does your messages looks like? Share a sample of your messages. ---
**Author:** ![nfored](https://avatars.discourse-cdn.com/v4/letter/n/9de0a6/32.png) [@nfored](https://discuss.elastic.co/u/nfored)\ **Post date:** [September 25, 2022, 5:23pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088/3 "2022-09-25T17:23:28Z")
cat pipelines.yml - pipeline.id: alpr path.config: "/opt/bitnami/logstash/config/alpr.conf" - pipeline.id: f5afm path.config: "/opt/bitnami/logstash/config/f5afm.conf" - pipeline.id: f5asm path.config: "/opt/bitnami/logstash/config/f5asm.conf" - pipeline.id: f5bot path.config: "/opt/bitnami/logstash/config/f5bot.conf" - pipeline.id: ingress path.config: "/opt/bitnami/logstash/config/ingress.conf" - pipeline.id: logstashrsyslog path.config: "/opt/bitnami/logstash/config/logstashrsyslog.conf" - pipeline.id: fortifw path.config: "/opt/bitnami/logstash/config/fortifw.conf" ```auto Wrong TIMESTAMP input { udp { port => 9904 type => syslog id => fortigate_9904 tags => ["fortigate","9904"] } } filter { if "fortigate" in [tags] { mutate { add_field => { "[@metadata][index]" => "fortigate-%{+YYYY.MM.dd}" } add_tag => ["fortigate"] add_field => ["forti_host", "fw.mcfeetershq.local"] } grok { match => ["message", "%{SYSLOG5424PRI:syslog_index}%{GREEDYDATA:message}"] overwrite => ["message"] tag_on_failure => ["failure_grok_fortigate"] } kv { } if [msg] { mutate { replace => ["message", "%{msg}"] } } if [totalsession] { mutate { convert => { "totalsession" => "integer" } } } if [cpu] { mutate { convert => { "cpu" => "integer" } } } if [mem] { mutate { convert => { "mem" => "integer" } } } if [setuprate] { mutate { convert => { "setuprate" => "integer" } } } mutate { add_field => ["logTimestamp", "%{date} %{time}"] add_field => ["loglevel", "%{level}"] replace => ["fortigate_type", "%{type}"] replace => ["fortigate_subtype", "%{subtype}"] remove_field => ["msg", "type", "level", "date", "time"] } date { locale => "en" match => ["logTimestamp", "YYYY-MM-dd HH:mm:ss"] remove_field => ["logTimestamp", "year", "month", "day", "time", "date"] add_field => ["type", "syslog"] } } } output { elasticsearch { hosts => ["https://mhqelastic-es-http:9200"] index => "%{[@metadata][index]}" user => "elastic" password => "${ELASTICSEARCH_PASSWORD}" ssl => true cacert => "/var/data/tls.crt" } } ``` Correct Timestamp ```auto input { syslog { port => 5344 tags => ["bot","5344"] type => bot id => "bot_syslog_listener_5344" } } filter { if [type] == 'bot' { mutate { add_field => { "[@metadata][index]" => "bot-%{+YYYY.MM.dd}" } } # grok { # match => ["message", 'hostname=%{QS},bigip_mgmt_ip=%{QS},(bigip_mgmt_ip2=%{QS})?,client_ip=%{QS},client_ip_geo_location="N%{URIPATHPARAM}",client_port=%{QS},client_request_uri=%{QS},configuration_date_time="%{CISCOTIMESTAMP}",context_name=%{QS},context_type=%{QS},dest_ip=%{QS},dest_port=%{QS},device_product=%{QS},device_vendor=%{QS},device_version=%{QS},errdefs_msgno=%{QS},http_method=%{QS},http_protocol_indication=%{QS},http_protocol_info=%{QS},route_domain=%{QS},timestamp=%{QS},virtual_server_name=%{QS},device_id=%{QS},host=%{QS},request_date_time="%{CISCOTIMESTAMP}",profile_name=%{QS},support_id=%{QS},request_status=%{QS},action=%{QS},reason=%{QS},previous_action=%{QS},previous_support_id=%{QS},previous_request_date_time=%{QS},bot_signature=%{QS},bot_signature_category=%{QS},bot_name=%{QS},session_id=%{QS},class=%{QS},anomaly_categories=%{QS},anomalies=%{QS},additional_bot_signatures=%{QS},micro_service_name=%{QS},micro_service_type=%{QS},micro_service_matched_wildcard_url=%{QS},micro_service_hostname=%{QS},configured_mitigation_action=%{QS},configured_mitigation_action_reason=%{QS},actual_mitigation_action=%{QS},actual_mitigation_action_reason=%{QS},browser_configured_verification_action=%{QS},browser_actual_verification_action=%{QS},browser_actual_verification_action_reason=%{QS},captcha_status=%{QS},browser_verification_status=%{QS},device_id_status=%{QS},device_id_action=%{QS},previous_initiated_action=%{QS},previous_initiated_action_status=%{QS},new_request_status=%{QS},enforced_by=%{QS},mobile_is_app=%{QS},challenge_failure_reason=%{QS},classification_reason=%{QS},client_type=%{QS},application_display_name=%{QS},application_version=%{QS},mobile_in_emulation_mode=%{QS},os_name=%{QS},jailbroken_or_rooted_device=%{QS},mobile_debugger_enabled_device=%{QS},imei=%{QS},human_behaviour=%{QS},http_request=%{QS}'] # # } kv { field_split => "," } } } output { elasticsearch { hosts => ["https://mhqelastic-es-http:9200"] index => "%{[@metadata][index]}" user => "elastic" password => "${ELASTICSEARCH_PASSWORD}" ssl => true cacert => "/var/data/tls.crt" } } ``` ![asm broken](https://us1.discourse-cdn.com/elastic/original/3X/4/a/4a86daec194bfd12324abca17807f3841a0fd805.jpeg) ![bot_working](https://us1.discourse-cdn.com/elastic/original/3X/0/d/0d5b00ee874c45714dbbc7d1a261ca721b7bf3cc.jpeg) ![forti_broken](https://us1.discourse-cdn.com/elastic/original/3X/4/9/4994071781598ecca2bdce5d2f182b69a43b53f1.jpeg) ---
**Author:** ![nfored](https://avatars.discourse-cdn.com/v4/letter/n/9de0a6/32.png) [@nfored](https://discuss.elastic.co/u/nfored)\ **Post date:** [September 25, 2022, 5:25pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088/4 "2022-09-25T17:25:50Z")
Sorry looks like the the post put both working and not working in the same code block but if you look there is a line break that says correct or wrong Sorry one last thing here is the other broken config ```auto input { syslog { port => 5244 tags => ["asm","5244"] type => asm id => "asm_syslog_listener_5244" } } filter { if [type] == 'asm' { mutate { add_field => { "[@metadata][index]" => "asm-%{+YYYY.MM.dd}" } } grok { match => { "message" => [ "attack_type=\"%{DATA:attack_type}\"", ",blocking_exception_reason=\"%{DATA:blocking_exception_reason}\"", ",date_time=\"%{DATE:date_time}\"", ",dest_port=\"%{DATA:dest_port}\"", ",ip_client=\"%{DATA:ip_client}\"", ",is_truncated=\"%{DATA:is_truncated}\"", ",method=\"%{WORD:method}\"", ",policy_name=\"%{DATA:policy_name}\"", ",protocol=\"%{DATA:protocol}\"", ",request_status=\"%{DATA:request_status}\"", ",response_code=\"%{DATA:response_code}\"", ",severity=\"%{DATA:severity}\"", ",sig_cves=\"%{DATA:sig_cves}\"", ",sig_ids=\"%{DATA:sig_ids}\"", ",sig_names=\"%{DATA:sig_names}\"", ",sig_set_names=\"%{DATA:sig_set_names}\"", ",src_port=\"%{DATA:src_port}\"", ",sub_violations=\"%{DATA:sub_violations}\"", ",support_id=\"%{DATA:support_id}\"", "unit_hostname=\"%{DATA:unit_hostname}\"", ",uri=\"%{DATA:uri}\"", ",violation_rating=\"%{DATA:violation_rating}\"", ",vs_name=\"%{DATA:vs_name}\"", ",x_forwarded_for_header_value=\"%{DATA:x_forwarded_for_header_value}\"", ",outcome=\"%{DATA:outcome}\"", ",outcome_reason=\"%{DATA:outcome_reason}\"", ",violations=\"%{DATA:violations}\"", ",violation_details=\"%{DATA:violation_details}\"", ",request=\"%{DATA:request}\"" ] } break_on_match => false } #date { # match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss +0500"] # target => "@timestamp" #} mutate { split => { "attack_type" => "," } split => { "sig_ids" => "," } split => { "sig_names" => "," } split => { "sig_cves" => "," } split => { "staged_sig_ids" => "," } split => { "staged_sig_names" => "," } split => { "staged_sig_cves" => "," } split => { "sig_set_names" => "," } split => { "threat_campaign_names" => "," } split => { "staged_threat_campaign_names" => "," } split => { "violations" => "," } split => { "sub_violations" => "," } } if [x_forwarded_for_header_value] != "N/A" { mutate { add_field => { "source_host" => "%{x_forwarded_for_header_value}"}} } else { mutate { add_field => { "source_host" => "%{ip_client}"}} } } } output { elasticsearch { hosts => ["https://mhqelastic-es-http:9200"] index => "%{[@metadata][index]}" user => "elastic" password => "${ELASTICSEARCH_PASSWORD}" ssl => true cacert => "/var/data/tls.crt" } } ``` ---
**Author:** ![nfored](https://avatars.discourse-cdn.com/v4/letter/n/9de0a6/32.png) [@nfored](https://discuss.elastic.co/u/nfored)\ **Post date:** [September 25, 2022, 5:30pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088/5 "2022-09-25T17:30:58Z")
```auto ASM Message "hostname=\"awf01.mcfeetershq.local\",bigip_mgmt_ip=\"192.168.100.14\",bigip_mgmt_ip2=\"::\",client_ip=\"172.16.1.4\",client_ip_geo_location=\"N/A\",client_port=\"34000\",client_request_uri=\"/\",configuration_date_time=\"Sep 21 2022 19:39:58\",context_name=\"/Common/tk_wp_vs\",context_type=\"Virtual Server\",dest_ip=\"172.16.1.180\",dest_port=\"443\",device_product=\"Application Security Module\",device_vendor=\"F5\",device_version=\"pgo_use x86_64 vadc TMM Version 17.0.0.1.0.0.4 \",errdefs_msgno=\"23003147\",http_method=\"GET\",http_protocol_indication=\"HTTPS\",http_protocol_info=\"HTTP/1.1\",route_domain=\"0\",timestamp=\"Sep 25 2022 17:21:54\",virtual_server_name=\"/Common/tk_wp_vs\",device_id=\"N/A\",host=\"75.81.29.97\",request_date_time=\"Sep 25 2022 12:21:54\",profile_name=\"/Common/TK_WP_Bot\",support_id=\"4145068333164682490\",request_status=\"legal\",action=\"alarm\",reason=\"\",previous_action=\"None\",previous_support_id=\"N/A\",previous_request_date_time=\"N/A\",bot_signature=\"N/A\",bot_signature_category=\"N/A\",bot_name=\"OpenNMS\",session_id=\"0\",class=\"Unknown\",anomaly_categories=\"N/A\",anomalies=\"N/A\",additional_bot_signatures=\"N/A\",micro_service_name=\"N/A\",micro_service_type=\"N/A\",micro_service_matched_wildcard_url=\"N/A\",micro_service_hostname=\"N/A\",configured_mitigation_action=\"Alarm\",configured_mitigation_action_reason=\"\",actual_mitigation_action=\"Alarm\",actual_mitigation_action_reason=\"None\",browser_configured_verification_action=\"None\",browser_actual_verification_action=\"None\",browser_actual_verification_action_reason=\"None\",captcha_status=\"None\",browser_verification_status=\"None\",device_id_status=\"None\",device_id_action=\"None\",previous_initiated_action=\"None\",previous_initiated_action_status=\"None\",new_request_status=\"Alarmed\",enforced_by=\"Profile Mitigation and Verification Settings\",mobile_is_app=\"false\",challenge_failure_reason=\"\",classification_reason=\"\",client_type=\"Uncategorized\",application_display_name=\"N/A\",application_version=\"N/A\",mobile_in_emulation_mode=\"N/A\",os_name=\"N/A\",jailbroken_or_rooted_device=\"N/A\",mobile_debugger_enabled_device=\"N/A\",imei=\"N/A\",human_behaviour=\"N/A\",http_request=\"GET / HTTP/1.1\\r\\nConnection: CLOSE \\r\\nHost: 75.81.29.97\\r\\nUser-Agent: OpenNMS HttpMonitor\\r\\n\\r\\n\"\n", Forti message firewall,info drop-all-fwd forward: in:WAN out:RT, src-mac 00:19:07:80:ac:00, proto TCP (SYN), 185.21.216.183:50358->172.16.108.250:6944, NAT 185.21.216.183:50358->(208.94.244.51:6944->172.16.108.250:6944), len 52 ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 23, 2022, 5:31pm UTC](https://discuss.elastic.co/t/new-user-confused-about-odd-timestamp/315088/6 "2022-10-23T17:31:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.