# Significance of @timestamp in Index Patterns **URL:** \ **Category:** Elasticsearch\ **Created:** [November 23, 2024, 2:35pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982 "2024-11-23T14:35:50Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [November 23, 2024, 2:35pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/1 "2024-11-23T14:35:51Z")
Hi All, What is the significance of @timestamp field that populates by default in all indices in Kibana? ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/2/42c622fbf246f0dcff26569387cd869d4000177b.png) I ask this question as this time differs with the `recvdTime` field that is set in the application and gets displayed as below: ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e62f7891a89797394aa101ee9b07f2e80ee054e.png) At times the difference between the two is in hours. Does that mean there is a delay or a choke in the network on within elasticsearch i.e. between filebeat and Logstash? Thanks ---
**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\ **Post date:** [November 23, 2024, 6:36pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/2 "2024-11-23T18:36:38Z")
`recvdTime` is not a standard field from beats or logstash afaik, and is likely coming from a field in your log or from a step in your processing pipeline. @timestamp can be set by beats or logstash, for example with windows logs, the @timestamp field is set to the time from the log. If there is no @timestamp field in the document when it arrives in Elasticsearch then Elasticsearch will set the timestamp to be the time it received the document ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [November 23, 2024, 6:54pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/3 "2024-11-23T18:54:53Z")
Thanks. Yes `recvdTime ` is a field coming from the logs. So to my question. If `@timestamp` is a field introduced by ES then does a difference in 6 to 7 hours between `recvdTime ` and `@timestamp` indicate a slowness or buffering either in filebeat to Logstash communication or due to a network slowness? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 23, 2024, 7:11pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/4 "2024-11-23T19:11:41Z")
If you have Logstash in your pipeline I believe it will set the `@timestamp` field to the timestamp it received it. There are several things that can cause differencs in timestamps, e.g.: - Differences in system time on the hosts that the components run on. - Timezone issues when parsing timestamps. This often results in a large and largely static difference. - Delays in reading the initial log. - Issues sending data down the pipeline, e.g. from Filebeat to Logstash or from any of these components to Elasticsearch. This can happen if some component goes down or is temporarily unavailable. I would recommend analysing the logs and see if there is any pattern across the ones that show delay, e.g. a specific log format, source or host. That may help you narrow down where the issue is. Also look at logs from the various components to see if there are any issues reported. ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [November 23, 2024, 7:44pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/5 "2024-11-23T19:44:44Z")
Thanks. This issue is seen during performance test (sorry forgot to mention it earlier) when 10X or more load is applied to the application. Logstash and Elasticsearch instances are hosted on the same servers. Both the Filebeat and Logstash/ES servers have exactly the same date/ time. I guess I need to analyze logs of different components at the time of the issue. It surely seems a load related issue. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 23, 2024, 8:19pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/6 "2024-11-23T20:19:13Z")
If you are running a load test it is posible that some part of the chain is a bottleneck. If this was the case I would expect to see the delay grow over time. Maybe you can create an index pipeline that calculates the delay in seconds and stores this in a new field on the events. Then use this to update the data ingested as well as any new data. That way you can easily plot delay over time and slice by sources and/or components in Kibana to identify patterns. I believe this is described [in this blog post](https://www.elastic.co/blog/calculating-ingest-lag-and-storing-ingest-time-in-elasticsearch-to-improve-observability) as well as [here](https://www.elastic.co/blog/should-i-use-logstash-or-elasticsearch-ingest-nodes). ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [November 23, 2024, 8:22pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/7 "2024-11-23T20:22:12Z")
Thanks. Also could the slowness be because we are using a 7.9.3 ELK cluster? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 23, 2024, 8:23pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/8 "2024-11-23T20:23:51Z")
It could be because Elasticsearch is not able to keep up, but even though the version you are using is very old and has been EOL a long time (you should look to upgrade as soon as possible) I do not think the version in itself is the cause. I have benchmarked high ingest loads on significantly older versions. When it comes to indexing it is often disk I/O that is the limiting factor. I would recommend you monitor disk I/O using e.g. `iostat -x`, on the Elasticsearch nodes to see of this looks like a possible cause. What type of storage are you using? Do you have monitoring installed so you can see the indexing rate you are achieving? ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [November 23, 2024, 8:29pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/9 "2024-11-23T20:29:08Z")
Thanks. One side question. What is the significance of "Time filter" when setting up an index pattern or Data view? ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/3/d32374154bd6425dce9b883ba1801b20fa25d44e.png) When setting it up it says `Select a primary time field for use with the global time filter` We have our ELK hosted on cloud VMs using virtual disks and will monitor disk i/o ---
**Author:** ![zaeemmasood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaeemmasood/32/102383_2.png) [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Post date:** [November 23, 2024, 10:16pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/10 "2024-11-23T22:16:46Z")
I believe we can check indexing rate during the test from cluster monitoring as below: ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/e/aead75f9d6c7c7fc4a0b7013cd20adedc0abf8b1.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 23, 2024, 10:33pm UTC](https://discuss.elastic.co/t/significance-of-timestamp-in-index-patterns/370982/11 "2024-11-23T22:33:54Z")
> [@zaeemmasood](#): > > hosted on cloud VMs using virtual disks Where are you hosting? What type of virtual disks are you using?