Migrating daily indices to monthly

Elastic Stack Logstash
1

Hi,

I am trying to collapse daily indices to monthly but when i try to match the count of both old and new indices, it does not match.

This is my current logstash config look like.

input {
  elasticsearch {
    hosts => [ "xx.xx.xx.xx" ]
    index => "aws-prod-2017.09.*"
    size => 1000
    scroll => "5m"
    docinfo => true
  }
}
output {
  elasticsearch {
    hosts => [ "xx.xx.xx.xx" ]
    index => "%{[@metadata][_type]}-%{+YYYY-MM}"
    document_type => "%{[@metadata][_type]}"
    document_id => "%{[@metadata][_id]}"
  }
  stdout {
    codec => "dots"
  }
}

The counts seems to be off by 200-400. What am i doing wrong here? Any other way to do this?

--
Niraj

2

The count of what? Are you trying to do a reindex with this config?

3

@warkolm I am using the count API to count the docs in it. I am trying to create monthly indices from daily indices for a particular month

4

Why not use the reindex API instead?

5

@Christian_Dahlqvist Can you give me an example for it.

6

Does the examples provided in the documentation not work?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.