PathHierarchy in index mapping no longer working in Elasticsearch 2.0

Hi,

I upgraded my elasticsearch cluster from version 1.7 to 2.0. When I tried to start the upgraded elasticsearch, it didn't initialize and it kept outputting the following message to the log file:

[2015-11-06 03:26:41,191][WARN ][cluster.action.shard     ] [Aardwolf] [demo05][0] received shard failed for [demo05][0], node[417Bdz9fR-6M0-06JAorJw], [P], v[48267], s[INITIALIZING], a[id=9K3heegXTy-_EJE7QVNPRg], unassigned_info[[reason=ALLOCATION_FAILED], at[2015-11-06T03:26:41.172Z], details[failed to create index, failure IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown Tokenizer type [PathHierarchy] for [win_path_hierarchy]]; ]], indexUUID [J2Vp_LEGQxSCYnXh_qCePw], message [failed to create index], failure [IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown Tokenizer type [PathHierarchy] for [win_path_hierarchy]]; ]
[demo05] IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown Tokenizer type [PathHierarchy] for [win_path_hierarchy]];

My index mapping file has the following definition:

 "settings" :
 {
  "analysis" :
  {
     "analyzer" :
     {
         "win_path" :
         {
            "type" : "custom",
            "tokenizer" : "win_path_hierarchy"
         },
         "backslash" :
         {
            "type" : "pattern",
            "pattern" : "\\\\"
         }
     },
     "tokenizer" :
     {
        "win_path_hierarchy" :
        {
           "type" : "PathHierarchy",
           "delimiter" : "\\"
        }
     }
  }
 }

It worked well in elasticsearch 1.7. Is this expected? Is there any way I can recover from this error?

Thanks,
Ian

You need to use path_hierarchy instead of PathHierarchy. We used to support both formats but we only documented the former one.

Thank you so much for the quick reply, David.

Do you know if there is anyway I can recover from this, like running some command or manually modifying some internal files in elasticsearch?

Thanks,
Ian

There is sadly nothing out of the box for this.
The shortest way if possible is to reindex.

It might be possible to write a Java program which reads the old mapping and transforms it to the new version.

May be you should open an issue and ask for this but I can't tell if it will be accepted.

Did the update plugin show you that error?

Thank again for the help David.

I only have one plugin: cloud-aws. I am not sure if the error came from the plugin. I simply checked out the /var/log/elasticsearch/[mycluser].log file.

Thanks,
Ian

That was not my question. I was asking if you installed the migration plugin before trying to upgrade to 2.0. (sorry I wrote update plugin but I meant migration plugin)

I thought I probably misunderstood your question. No, I didn't not the migration plugin first.

Thanks,
Ian

It was built to detect such problems. So I'm curious if it would have detect this case. Could you run it by any chance?
It does not require any restart.

David, unfortunately, I am no longer able to start my elasticsearch service. I tried downgrading my elasticsearch service and starting the service and it failed. The log file complained about something in the index unknown.

My understanding is that the migration plugin requires a running elasticsearch service. Let me know if that is not the case.

Thanks,
Ian