Migrating daily indices to monthly


I am trying to collapse daily indices to monthly but when i try to match the count of both old and new indices, it does not match.

This is my current logstash config look like.

input {
  elasticsearch {
    hosts => [ "xx.xx.xx.xx" ]
    index => "aws-prod-2017.09.*"
    size => 1000
    scroll => "5m"
    docinfo => true
output {
  elasticsearch {
    hosts => [ "xx.xx.xx.xx" ]
    index => "%{[@metadata][_type]}-%{+YYYY-MM}"
    document_type => "%{[@metadata][_type]}"
    document_id => "%{[@metadata][_id]}"
  stdout {
    codec => "dots"

The counts seems to be off by 200-400. What am i doing wrong here? Any other way to do this?


The count of what? Are you trying to do a reindex with this config?

@warkolm I am using the count API to count the docs in it. I am trying to create monthly indices from daily indices for a particular month

Why not use the reindex API instead?

@Christian_Dahlqvist Can you give me an example for it.

Does the examples provided in the documentation not work?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.