Need to change schema/metadata for existing index with lots of data

Elastic Stack Elasticsearch
1

(x-posting from the graylog2 group since I think this is really more of an
ES question...)

I have a Graylog2 0.10 instance running on Debian Squeeze (haven't upgraded
to Wheezy yet) with Elasticsearch 0.20.4. For some reason one of my
graylog2 indexes doesn't have the right datatype, index_options or
omit_norms set for the index. Specifically, I'm interested in the
histogram_time field which I use to generate audit reports to prove that
we're collecting logs for the past 90 days.

For example:

(I know the formatting isn't valid JSON, I just copied this out of the
'head' plugin, which doesn't preserve commas for some reason...)

{

state: open
settings: {
    index.number_of_replicas: 0
    index.version.created: 200499
    index.number_of_shards: 4
}
mappings: {
    message: {
        properties: {
            _mode: {
                type: string
            }
            _xdelay: {
                type: string
            }
            ...
           * histogram_time: {
                type: string
            }*

...

vs

{

state: open
settings: {
    index.version.created: 200499
    index.number_of_replicas: 0
    index.number_of_shards: 4
}
mappings: {
    message: {
        _source: {
            compress: true
        }
        dynamic_templates: [
            {
                store_generic: {
                    mapping: {
                        index: not_analyzed
                    }
                    match: *
                }
            }
        ]
        _ttl: {
            enabled: true
        }
        properties: {
            _last_pfn: {
                index: not_analyzed
                omit_norms: true
                index_options: docs
                type: string
            }
            _xdelay: {
                index: not_analyzed
                omit_norms: true
                index_options: docs
                type: string
            }
           * histogram_time: {
                format: yyyy-MM-dd HH-mm-ss
                type: date
            }

...*

The latter is what it should be, and that's what it is in every other
index, but this one index just has "type: string"

I've tried a few methods of reindexing this data. I've created a new index
called graylog2_7_re by copying the metadata from another index
(graylog2_6) and in the head plugin I can see that it has the correct
metadata, but when I run the reindexing scripts, for some reason the
metadata for the new index is changed to the wrong schema from the old
broken index. I can't find a way to have it actually reindex with the
correct metadata.

I've tried these two scripts:


And both have the same result of overriding the metadata that I set when
creating the new index.

Any ideas, advice or pointers would be greatly appreciated.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.