# Overwriting the @timestamp work on stdout but don't work on es

**URL:** <https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131>\
**Category:** Logstash\
**Created:** [August 7, 2020, 7:46am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131 "2020-08-07T07:46:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tristan\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tristan_d/32/73492_2.png) [@Tristan\_D](https://discuss.elastic.co/u/Tristan_D)\
**Post date:** [August 7, 2020, 7:46am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131/1 "2020-08-07T07:46:06Z")

</div>

I want to overwrite the @timestamp with date filter.  
The result in the stdout is ok, the @timestamp is written by the log time, but there is no document written into the elasticsearch.  
If I remove the date filter, the document is normally written into the elasticsearch.

The pattern  
`NGINX_ACCESS %{IPORHOST:remote_addr} - %{USERNAME:remote_user} \[%{HTTPDATE:timelocal}\] "%{WORD:request_method} %{DATA:request} %{DATA:http_version}" %{INT:status} %{NUMBER:body_bytes_sent} "%{DATA:http_refer}" "%{DATA:http_user_agent}" "%{NUMBER:request_time}" "%{DATA:ssl_protocol}" "%{DATA:ssl_cipher}" "%{DATA:http_x_forwarded_for}""%{DATA:upstream_addr}" "%{DATA:upstream_status}" "%{DATA:upstream_response_length}" "%{DATA:upstream_response_time}"`

Here is the config file of the logstash

```auto
input {
  beats {
    port => 5044
  }
}

filter {
  grok {
    patterns_dir => "/home/dpc/elk/logstash/patterns"
    match => { "message" => "%{NGINX_ACCESS}" }
  }
  mutate {
	remove_field => [
		"upstream_addr", "ssl_cipher", "ssl_protocol", "host", "ecs", "@version",    
		"input", "http_x_forwarded_for", "http_user_agent", "http_refer", "body_bytes_sent",
		"agent", "remote_user", "input"
	]   
  }
  date {
     match => ["timelocal", "dd/MMM/yyyy:HH:mm:ss Z"]
     target => "@timestamp"
  }
}

output {
  elasticsearch {
    hosts => ["http://10.193.161.30:9200"]
    index => "server_log_%{+YYYY.MM.dd}"
    codec => json
  }
  stdout {
    codec => json
  }
}

```

This is the log  
`10.192.1.144 - - [14/Jan/2020:20:34:20 +0800] "POST /msxiaobing_callback HTTP/1.1" 403 47 "-" "python-requests/2.18.4" "0.004" "-" "-" "-""10.193.161.5:8080" "403" "47" "0.004"`

The result of the stdout

```auto
{
    "http_version":"HTTP/1.1",
    "request":"/msxiaobing_callback",
    "request_method":"POST",
    "upstream_response_time":"0.004",
    "@timestamp":"2020-01-14T12:34:20.000Z",
    "upstream_status":"403",
    "message":"10.192.1.144 - - [14/Jan/2020:20:34:20 +0800] "POST /msxiaobing_callback HTTP/1.1" 403 47 "-" "python-requests/2.18.4" "0.004" "-" "-" "-""10.193.161.5:8080" "403" "47" "0.004"",
    "upstream_response_length":"47",
    "timelocal":"14/Jan/2020:20:34:20 +0800",
    "remote_addr":"10.192.1.144",
    "log":{
        "offset":28092,
        "file":{
            "path":"/data/dingpeichang/logs/server_access.log"
        }
    },
    "status":"403",
    "tags":[
        "beats_input_codec_plain_applied"
    ],
    "request_time":"0.004"
}

```

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [August 7, 2020, 8:01am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131/2 "2020-08-07T08:01:10Z")

</div>

Is there no error in the Logstash logs?

---

<div class="post-metadata">

**Author:** ![Tristan\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tristan_d/32/73492_2.png) [@Tristan\_D](https://discuss.elastic.co/u/Tristan_D)\
**Post date:** [August 7, 2020, 8:10am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131/3 "2020-08-07T08:10:47Z")

</div>

There is no any error log in the logstash logs. That's why it's hard for me to solve it.  
Thank you for your replying.

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [August 7, 2020, 8:24am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131/4 "2020-08-07T08:24:37Z")

</div>

Just to make sure: Querying your ES index like this  
`GET server_log_2020.01.14/_search?q=@timestamp:"2020-01-14T12:34:20.000Z"`  
returns nothing?

---

<div class="post-metadata">

**Author:** ![Tristan\_D](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tristan_d/32/73492_2.png) [@Tristan\_D](https://discuss.elastic.co/u/Tristan_D)\
**Post date:** [August 7, 2020, 10:36am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131/5 "2020-08-07T10:36:53Z")

</div>

I used the wrong index in the es.  
Thank you!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 4, 2020, 10:37am UTC](https://discuss.elastic.co/t/overwriting-the-timestamp-work-on-stdout-but-dont-work-on-es/244131/6 "2020-09-04T10:37:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.