# Java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation when starting spark **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [July 17, 2015, 4:19am UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763 "2015-07-17T04:19:18Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sandeep\_gunnam](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sandeep\_gunnam](https://discuss.elastic.co/u/sandeep_gunnam)\ **Post date:** [July 17, 2015, 4:19am UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763/1 "2015-07-17T04:19:18Z")
Hi Below are the details of the setup i have Cluster running Hadoop 2.6, Spark 1.3.1, and Scala 2.10.4. I used the library "elasticsearch-spark\_2.10-2.1.0.jar" in my code along with other spark libraries (like spark-core\_2.10-1.3.1.jar). When i run my code in the cluster using spark-submit i get the following dump with exception 2015-07-16 10:21:14,653 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory. 10:21:14.763 [main] INFO com.philips.bda.spark.SparkDriver$ - kafkaTopics value passed is [Ljava.lang.String;@5a7169a1. 10:21:14.765 [main] WARN com.philips.bda.spark.SparkDriver$ - Spark mode value is not passed. Running in spark-standalone mode. 10:21:14.765 [main] INFO com.philips.bda.spark.SparkDriver$ - Starting the Spark Applications!!! 15/07/16 10:21:15 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Exception in thread "main" java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation at org.apache.hadoop.fs.FileSystem.getScheme(FileSystem.java:216) at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2564) at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2574) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169) at com.philips.hdfs.filesystem.FileSystemFactory$.createFileSystem(FileSystemFactory.scala:27) at com.philips.hdfs.filesystem.FileSystemFactory$.getFileSystem(FileSystemFactory.scala:20) at com.philips.bda.spark.SparkDriver$.Setup(SparkDriver.scala:201) at com.philips.bda.spark.SparkDriver$.main(SparkDriver.scala:79) at com.philips.bda.spark.SparkDriver.main(SparkDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) I did not understand why spark is looking for Tachyon File System. Any help is very much appreciated. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [July 17, 2015, 8:31pm UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763/2 "2015-07-17T20:31:51Z")
Based on your stacktrace (please use formatting to make the post readable), the culprit seems to be `com.philips.bda.spark.SparkDriver$.Setup(SparkDriver.scala:201)`. Likely this code tries to bootstrap Spark which in turn, tries to connect to the Tachyon FS (which probably is not needed but is requested by the `Setup` method or it is not available at runtime). Either way, there's no elasticsearch-spark code in the stacktrace - in fact, it looks like in your case, the code fails early before any connection to Elasticsearch is done. Investigate how Spark is started (Spark docs have plenty of examples) and also your classpath; do you add the jars by hand or rely on Maven or other building tool to create it. Typically the latter pulls in all dependencies while the former requires one to cherry pick libraries especially during version upgrades. If I'm not mistaken, Spark 1.4 now relies on Tachyon so it might just be that a jar is missing. ---
**Author:** ![sandeep\_gunnam](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sandeep\_gunnam](https://discuss.elastic.co/u/sandeep_gunnam)\ **Post date:** [July 22, 2015, 6:49am UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763/3 "2015-07-22T06:49:34Z")
Thanks for your reply costin. In first place i posted this question here because i got this problem only when i added this jar to my eclipse. When i remove this jar and run my code then i dont see the TFS error. Here's how i got around this problem. I used elasticsearch-spark\_2.10-2.1.0.jar only during compile time but excluded it during packing using SBT. When i am doing the spark-submit i gave this jar in the jars path and this error is gone. Not sure exactly why, but am atleast i am not blocked. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [July 22, 2015, 8:04am UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763/4 "2015-07-22T08:04:59Z")
Likely this is caused since Eclipse pull in _all_ dependencies including the ones marked as `provided` or `optional` which is incorrect. Not sure how you create, generate the Eclipse classpath file but do take this into account. ---
**Author:** ![govardhanraoganji](https://avatars.discourse-cdn.com/v4/letter/g/67e7ee/32.png) [@govardhanraoganji](https://discuss.elastic.co/u/govardhanraoganji)\ **Post date:** [July 31, 2015, 5:50am UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763/5 "2015-07-31T05:50:57Z")
Use **spark-assembly-1.3.1-hadoop2.6.0.jar** ,it helps you to over come the issue. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:27pm UTC](https://discuss.elastic.co/t/java-lang-unsupportedoperationexception-not-implemented-by-the-tfs-filesystem-implementation-when-starting-spark/25763/6 "2017-07-06T13:27:46Z")