Request for Updated Blog Post: Elastic Stack Monitoring with ES 8.7

Dear Sir,
@shaunak I would like to express my gratitude for your insightful blog post titled 'Elastic Stack monitoring with Metricbeat via Logstash or Kafka' It has been instrumental in helping me set up a centralized monitoring system. Thank you for sharing your expertise.
Currently, I am in the process of migrating to ES 8.7 while aiming to maintain my existing monitoring architecture. Unfortunately, I have encountered a challenge during this transition. Instead of solely seeking support for my specific issue, I wanted to reach out to you because I believe an updated blog post dedicated to ES 8.7 would immensely benefit many others. Such a post could offer a more comprehensive and structured approach for those seeking a centralized monitoring solution.
Thank you for taking the time to consider my request. I appreciate your valuable insights.


While we anticipate an update to the article mentioned above, I wanted to share a solution for a potential issue you might encounter. You may come across an error message stating '"reason"=>"only write ops with an op_type of create are allowed in data streams"'. After extensive research, including various blog posts and Elastic's documentation, I found that the most straightforward solution is to revert to using indices, as data streams were not designed to handle update operations.

Here's a step-by-step guide on how to do it:

  1. Stop any incoming monitoring data. It's crucial to do this first to prevent any data loss or corruption.
  2. Go to the index templates.
  3. Open each template that begins with '.monitoring-'.
  4. Please note that these are system templates, so you'll need to select 'show system templates' to view them.
  5. Untick the option that says "Create data stream" and then save your changes.

By following these steps, you should be able to circumvent the mentioned issue. I hope this saves you some time and effort.