PathHierarchy in index mapping no longer working in Elasticsearch 2.0


#1

Hi,

I upgraded my elasticsearch cluster from version 1.7 to 2.0. When I tried to start the upgraded elasticsearch, it didn't initialize and it kept outputting the following message to the log file:

[2015-11-06 03:26:41,191][WARN ][cluster.action.shard     ] [Aardwolf] [demo05][0] received shard failed for [demo05][0], node[417Bdz9fR-6M0-06JAorJw], [P], v[48267], s[INITIALIZING], a[id=9K3heegXTy-_EJE7QVNPRg], unassigned_info[[reason=ALLOCATION_FAILED], at[2015-11-06T03:26:41.172Z], details[failed to create index, failure IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown Tokenizer type [PathHierarchy] for [win_path_hierarchy]]; ]], indexUUID [J2Vp_LEGQxSCYnXh_qCePw], message [failed to create index], failure [IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown Tokenizer type [PathHierarchy] for [win_path_hierarchy]]; ]
[demo05] IndexCreationException[failed to create index]; nested: IllegalArgumentException[Unknown Tokenizer type [PathHierarchy] for [win_path_hierarchy]];

My index mapping file has the following definition:

 "settings" :
 {
  "analysis" :
  {
     "analyzer" :
     {
         "win_path" :
         {
            "type" : "custom",
            "tokenizer" : "win_path_hierarchy"
         },
         "backslash" :
         {
            "type" : "pattern",
            "pattern" : "\\\\"
         }
     },
     "tokenizer" :
     {
        "win_path_hierarchy" :
        {
           "type" : "PathHierarchy",
           "delimiter" : "\\"
        }
     }
  }
 }

It worked well in elasticsearch 1.7. Is this expected? Is there any way I can recover from this error?

Thanks,
Ian


(David Pilato) #2

You need to use path_hierarchy instead of PathHierarchy. We used to support both formats but we only documented the former one.


#3

Thank you so much for the quick reply, David.

Do you know if there is anyway I can recover from this, like running some command or manually modifying some internal files in elasticsearch?

Thanks,
Ian


(David Pilato) #4

There is sadly nothing out of the box for this.
The shortest way if possible is to reindex.

It might be possible to write a Java program which reads the old mapping and transforms it to the new version.

May be you should open an issue and ask for this but I can't tell if it will be accepted.

Did the update plugin show you that error?


#5

Thank again for the help David.

I only have one plugin: cloud-aws. I am not sure if the error came from the plugin. I simply checked out the /var/log/elasticsearch/[mycluser].log file.

Thanks,
Ian


(David Pilato) #6

That was not my question. I was asking if you installed the migration plugin before trying to upgrade to 2.0. (sorry I wrote update plugin but I meant migration plugin)


#7

I thought I probably misunderstood your question. No, I didn't not the migration plugin first.

Thanks,
Ian


(David Pilato) #8

It was built to detect such problems. So I'm curious if it would have detect this case. Could you run it by any chance?
It does not require any restart.


#9

David, unfortunately, I am no longer able to start my elasticsearch service. I tried downgrading my elasticsearch service and starting the service and it failed. The log file complained about something in the index unknown.

My understanding is that the migration plugin requires a running elasticsearch service. Let me know if that is not the case.

Thanks,
Ian


(system) #10