Migrating daily indices to monthly


(Niraj Kumar) #1

Hi,

I am trying to collapse daily indices to monthly but when i try to match the count of both old and new indices, it does not match.

This is my current logstash config look like.

input {
  elasticsearch {
    hosts => [ "xx.xx.xx.xx" ]
    index => "aws-prod-2017.09.*"
    size => 1000
    scroll => "5m"
    docinfo => true
  }
}
output {
  elasticsearch {
    hosts => [ "xx.xx.xx.xx" ]
    index => "%{[@metadata][_type]}-%{+YYYY-MM}"
    document_type => "%{[@metadata][_type]}"
    document_id => "%{[@metadata][_id]}"
  }
  stdout {
    codec => "dots"
  }
}

The counts seems to be off by 200-400. What am i doing wrong here? Any other way to do this?

--
Niraj


(Mark Walkom) #2

The count of what? Are you trying to do a reindex with this config?


(Niraj Kumar) #3

@warkolm I am using the count API to count the docs in it. I am trying to create monthly indices from daily indices for a particular month


(Christian Dahlqvist) #4

Why not use the reindex API instead?


(Niraj Kumar) #5

@Christian_Dahlqvist Can you give me an example for it.


(Christian Dahlqvist) #6

Does the examples provided in the documentation not work?


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.