Figured out why it failed. I actually ignored this warning in the log
[2015-12-15 17:50:24,207][WARN ][plugin.hadoop.hdfs ] The Java Security Manager is enabled however Hadoop is not compatible with it and thus needs to be disabled; see the docs for more information...
When I tried to create a repository, it actually threw access denied (java.io.FilePermission <<ALL FILES>> execute). Then I ran create the repository again, I got NoClassDefFoundError.
{"error":{"root_cause":[{"type":"repository_exception","reason":"[my_backup] failed to create repository"}],"type":"repository_excep
tion","reason":"[my_backup] failed to create repository","caused_by":{"type":"creation_exception","reason":"Guice creation errors:\n\n1) Error injecting constructor, java.lang.ExceptionInInitializerError\n at org.elasticsearch.repositories.hdfs.HdfsRepository.<init>(Unknown Source)\n while locating org.elasticsearch.repositories.hdfs.HdfsRepository\n while locating org.elasticsearch.repositories.Repository\n\n1 error","caused_by":{"type":"exception_in_initializer_error","reason":null,"caused_by":{"type":"access_control_exception","reason":"access denied (\"java.io.FilePermission\" \"<<ALL FILES>>\" \"execute\")"}}}},"status":500}