# Java Serialization of Exceptions **URL:** \ **Category:** Elasticsearch\ **Created:** [March 21, 2014, 4:18pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528 "2014-03-21T16:18:38Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Chris\_Berry\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_berry_2/32/1366_2.png) [@Chris\_Berry\_2](https://discuss.elastic.co/u/Chris_Berry_2)\ **Post date:** [March 21, 2014, 4:18pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/1 "2014-03-21T16:18:38Z")
Greetings, Let me say up-front, I am a huge fan and proponent of Elasticsearch. It is a beautiful tool. So, that said, it surprises me that Elasticsearch has such a pedestrian flaw, and serializes it's Exceptions using Java Serialization. In a big shop it is quite difficult (i.e. next to impossible) to keep all the ES Clients on the same exact JVM as Elasticsearch, and thus, it is not uncommon to get TransportSerializationExceptions instead of the actual underlying problem. I was really hoping this would be corrected in ES 1.0.X, but no such luck. (As far as I can tell...) It seems that this is pretty easily fixed? Just switch to a JSON representation of the basic Exception and gracefully (forwards-compatibly) attempt to re-hydrate the actual Exception class. You'd just have to drop an additional "header" in the stream that tells the code it is a JSON response and route to the right Handler it accordingly. If the header is missing, then do things the old way with Java Serialization?? Are there any plans to fix this? Or perhaps to entertain a Pull Request? It may seem just an annoyance, but it is actually pretty bad, in that it keeps Clients from seeing their real issues. Especially in shops where it is difficult to see the Production logs of Elasticsearch itself. Thanks much, -- Chris -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1e02778a-a1d0-44b5-8b1f-e8de7de33668%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1e02778a-a1d0-44b5-8b1f-e8de7de33668%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 21, 2014, 4:49pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/2 "2014-03-21T16:49:56Z")
I would be very interested in that pull request, too. Changing every exception transport to a textual JSON error seems a proper alternative. I haven't tried Jackson ObjectMapper on the exception classes that are present in ES but it should be possible. Jörg On Fri, Mar 21, 2014 at 5:18 PM, Chris Berry [chriswberry@gmail.com](mailto:chriswberry@gmail.com) wrote: > Greetings, > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It is > a beautiful tool. > > So, that said, it surprises me that Elasticsearch has such a pedestrian > flaw, and serializes it's Exceptions using Java Serialization. > In a big shop it is quite difficult (i.e. next to impossible) to keep all > the ES Clients on the same exact JVM as Elasticsearch, and thus, it is not > uncommon to get TransportSerializationExceptions instead of the actual > underlying problem. > I was really hoping this would be corrected in ES 1.0.X, but no such luck. > (As far as I can tell...) > > It seems that this is pretty easily fixed? > Just switch to a JSON representation of the basic Exception and gracefully > (forwards-compatibly) attempt to re-hydrate the actual Exception class. > You'd just have to drop an additional "header" in the stream that tells > the code it is a JSON response and route to the right Handler it > accordingly. If the header is missing, then do things the old way with Java > Serialization?? > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > It may seem just an annoyance, but it is actually pretty bad, in that it > keeps Clients from seeing their real issues. Especially in shops where it > is difficult to see the Production logs of Elasticsearch itself. > > Thanks much, > -- Chris > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1e02778a-a1d0-44b5-8b1f-e8de7de33668%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1e02778a-a1d0-44b5-8b1f-e8de7de33668%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/1e02778a-a1d0-44b5-8b1f-e8de7de33668%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1e02778a-a1d0-44b5-8b1f-e8de7de33668%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFxv%3D%3DNFxXkLaT7%2BKcuBApO0NZfwN77mi\_BS\_BhNeG0tA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFxv%3D%3DNFxXkLaT7%2BKcuBApO0NZfwN77mi_BS_BhNeG0tA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 21, 2014, 5:18pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/3 "2014-03-21T17:18:39Z")
I wonder why you are asking for this feature? If its because Java broke backward comp. on serialization of InetAddress that we use in our exceptions, then its a bug in Java serialization, hard for us to do something about it. You will loose a lot by trying to serialize exceptions using JSON, and we prefer not to introduce dependency on ObjectMapper in Jackson, or try and serialize exceptions using Jackson. I would be very careful in introducing this just because of a (one time bug) in Java. On Friday, March 21, 2014 5:18:38 PM UTC+1, Chris Berry wrote: > Greetings, > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It is > a beautiful tool. > > So, that said, it surprises me that Elasticsearch has such a pedestrian > flaw, and serializes it's Exceptions using Java Serialization. > In a big shop it is quite difficult (i.e. next to impossible) to keep all > the ES Clients on the same exact JVM as Elasticsearch, and thus, it is not > uncommon to get TransportSerializationExceptions instead of the actual > underlying problem. > I was really hoping this would be corrected in ES 1.0.X, but no such luck. > (As far as I can tell...) > > It seems that this is pretty easily fixed? > Just switch to a JSON representation of the basic Exception and gracefully > (forwards-compatibly) attempt to re-hydrate the actual Exception class. > You'd just have to drop an additional "header" in the stream that tells > the code it is a JSON response and route to the right Handler it > accordingly. If the header is missing, then do things the old way with Java > Serialization?? > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > It may seem just an annoyance, but it is actually pretty bad, in that it > keeps Clients from seeing their real issues. Especially in shops where it > is difficult to see the Production logs of Elasticsearch itself. > > Thanks much, > -- Chris -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Chris\_Berry\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_berry_2/32/1366_2.png) [@Chris\_Berry\_2](https://discuss.elastic.co/u/Chris_Berry_2)\ **Post date:** [March 21, 2014, 6:12pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/4 "2014-03-21T18:12:02Z")
If it happened once, then by definition it will happen again. History repeats itself. 😉 What exactly would you lose? You are simply trading one rigid serialization scheme for another more lenient one. Yes, you would have to introduce something like Jackson's Object Mapper, but that seems to be the defacto standard today and with your use of the Shade Plugin it wouldn't really be a burden on the Client anyway. With all due respect, you may be trivializing the impact of this one time bug. It is difficult, at best, to inform all the Clients of your Cluster; "Hey, if you want to see what your Exceptions really are, then upgrade your JVM" Especially in large SOA shops This just decouples the Client and Server deployments. Thanks much, -- Chris On Mar 21, 2014, at 12:18 PM, kimchy [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > I wonder why you are asking for this feature? If its because Java broke backward comp. on serialization of InetAddress that we use in our exceptions, then its a bug in Java serialization, hard for us to do something about it. > > You will loose a lot by trying to serialize exceptions using JSON, and we prefer not to introduce dependency on ObjectMapper in Jackson, or try and serialize exceptions using Jackson. > > I would be very careful in introducing this just because of a (one time bug) in Java. > > On Friday, March 21, 2014 5:18:38 PM UTC+1, Chris Berry wrote: > Greetings, > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It is a beautiful tool. > > So, that said, it surprises me that Elasticsearch has such a pedestrian flaw, and serializes it's Exceptions using Java Serialization. > In a big shop it is quite difficult (i.e. next to impossible) to keep all the ES Clients on the same exact JVM as Elasticsearch, and thus, it is not uncommon to get TransportSerializationExceptions instead of the actual underlying problem. > I was really hoping this would be corrected in ES 1.0.X, but no such luck. (As far as I can tell...) > > It seems that this is pretty easily fixed? > Just switch to a JSON representation of the basic Exception and gracefully (forwards-compatibly) attempt to re-hydrate the actual Exception class. > You'd just have to drop an additional "header" in the stream that tells the code it is a JSON response and route to the right Handler it accordingly. If the header is missing, then do things the old way with Java Serialization?? > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > It may seem just an annoyance, but it is actually pretty bad, in that it keeps Clients from seeing their real issues. Especially in shops where it is difficult to see the Production logs of Elasticsearch itself. > > Thanks much, > -- Chris > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/DF3191F4-8B07-4DCB-920A-AFD35F2FECF1%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/DF3191F4-8B07-4DCB-920A-AFD35F2FECF1%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 21, 2014, 7:07pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/5 "2014-03-21T19:07:07Z")
Not trivializing the bug at all, god knows I spend close to a week tracing it down to a JVM backward incompatibility change, but this happened once over the almost 5 years Elasticsearch existed. To introduce a workaround to something that happened once, compared to potential bugs in the workaround (Jackson is great, but what would happen if there was a bug in it for example) is not a great solution. Obviously, if this happened more often, then this is something we need to address. On Friday, March 21, 2014 7:12:02 PM UTC+1, Chris Berry wrote: > If it happened once, then by definition it will happen again. History > repeats itself. 😉 > > What exactly would you lose? > You are simply trading one rigid serialization scheme for another more > lenient one. > Yes, you would have to introduce something like Jackson’s Object Mapper, > but that seems to be the defacto standard today and with your use of the > Shade Plugin it wouldn’t really be a burden on the Client anyway. > > With all due respect, you may be trivializing the impact of this one time > bug. > It is difficult, at best, to inform all the Clients of your Cluster; “Hey, > if you want to see what your Exceptions really are, then upgrade your JVM” > Especially in large SOA shops > > This just decouples the Client and Server deployments. > > Thanks much, > — Chris > > On Mar 21, 2014, at 12:18 PM, kimchy \<[kim...@gmail.com](mailto:kim...@gmail.com) \\> > wrote: > > I wonder why you are asking for this feature? If its because Java broke > backward comp. on serialization of InetAddress that we use in our > exceptions, then its a bug in Java serialization, hard for us to do > something about it. > > You will loose a lot by trying to serialize exceptions using JSON, and we > prefer not to introduce dependency on ObjectMapper in Jackson, or try and > serialize exceptions using Jackson. > > I would be very careful in introducing this just because of a (one time > bug) in Java. > > On Friday, March 21, 2014 5:18:38 PM UTC+1, Chris Berry wrote: > > > Greetings, > > > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It > > is a beautiful tool. > > > > So, that said, it surprises me that Elasticsearch has such a pedestrian > > flaw, and serializes it's Exceptions using Java Serialization. > > In a big shop it is quite difficult (i.e. next to impossible) to keep all > > the ES Clients on the same exact JVM as Elasticsearch, and thus, it is not > > uncommon to get TransportSerializationExceptions instead of the actual > > underlying problem. > > I was really hoping this would be corrected in ES 1.0.X, but no such > > luck. (As far as I can tell...) > > > > It seems that this is pretty easily fixed? > > Just switch to a JSON representation of the basic Exception and > > gracefully (forwards-compatibly) attempt to re-hydrate the actual Exception > > class. > > You'd just have to drop an additional "header" in the stream that tells > > the code it is a JSON response and route to the right Handler it > > accordingly. If the header is missing, then do things the old way with Java > > Serialization?? > > > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > > It may seem just an annoyance, but it is actually pretty bad, in that it > > keeps Clients from seeing their real issues. Especially in shops where it > > is difficult to see the Production logs of Elasticsearch itself. > > > > Thanks much, > > -- Chris > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)\ **Post date:** [March 21, 2014, 7:47pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/6 "2014-03-21T19:47:02Z")
If this is a concern, why not have your client's use the REST api so they don't need to worry about matching their java version with the java version of the search cluster? Thanks, Matt Weber On Fri, Mar 21, 2014 at 12:07 PM, kimchy [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Not trivializing the bug at all, god knows I spend close to a week tracing > it down to a JVM backward incompatibility change, but this happened once > over the almost 5 years Elasticsearch existed. To introduce a workaround to > something that happened once, compared to potential bugs in the workaround > (Jackson is great, but what would happen if there was a bug in it for > example) is not a great solution. Obviously, if this happened more often, > then this is something we need to address. > > On Friday, March 21, 2014 7:12:02 PM UTC+1, Chris Berry wrote: > > > If it happened once, then by definition it will happen again. History > > repeats itself. 😉 > > > > What exactly would you lose? > > You are simply trading one rigid serialization scheme for another more > > lenient one. > > Yes, you would have to introduce something like Jackson’s Object Mapper, > > but that seems to be the defacto standard today and with your use of the > > Shade Plugin it wouldn’t really be a burden on the Client anyway. > > > > With all due respect, you may be trivializing the impact of this one time > > bug. > > It is difficult, at best, to inform all the Clients of your Cluster; > > “Hey, if you want to see what your Exceptions really are, then upgrade your > > JVM” > > Especially in large SOA shops > > > > This just decouples the Client and Server deployments. > > > > Thanks much, > > — Chris > > > > On Mar 21, 2014, at 12:18 PM, kimchy [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > I wonder why you are asking for this feature? If its because Java broke > > backward comp. on serialization of InetAddress that we use in our > > exceptions, then its a bug in Java serialization, hard for us to do > > something about it. > > > > You will loose a lot by trying to serialize exceptions using JSON, and we > > prefer not to introduce dependency on ObjectMapper in Jackson, or try and > > serialize exceptions using Jackson. > > > > I would be very careful in introducing this just because of a (one time > > bug) in Java. > > > > On Friday, March 21, 2014 5:18:38 PM UTC+1, Chris Berry wrote: > > > > > Greetings, > > > > > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It > > > is a beautiful tool. > > > > > > So, that said, it surprises me that Elasticsearch has such a pedestrian > > > flaw, and serializes it's Exceptions using Java Serialization. > > > In a big shop it is quite difficult (i.e. next to impossible) to keep > > > all the ES Clients on the same exact JVM as Elasticsearch, and thus, it is > > > not uncommon to get TransportSerializationExceptions instead of the > > > actual underlying problem. > > > I was really hoping this would be corrected in ES 1.0.X, but no such > > > luck. (As far as I can tell...) > > > > > > It seems that this is pretty easily fixed? > > > Just switch to a JSON representation of the basic Exception and > > > gracefully (forwards-compatibly) attempt to re-hydrate the actual Exception > > > class. > > > You'd just have to drop an additional "header" in the stream that tells > > > the code it is a JSON response and route to the right Handler it > > > accordingly. If the header is missing, then do things the old way with Java > > > Serialization?? > > > > > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > > > It may seem just an annoyance, but it is actually pretty bad, in that it > > > keeps Clients from seeing their real issues. Especially in shops where it > > > is difficult to see the Production logs of Elasticsearch itself. > > > > > > Thanks much, > > > -- Chris > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/) > > topic/elasticsearch/7bpam7mWjY8/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0% > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoDB%2BVqpKvc3BPbnX2COxpwMi35GmON1bBKD74u9APJ2HQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoDB%2BVqpKvc3BPbnX2COxpwMi35GmON1bBKD74u9APJ2HQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 21, 2014, 7:47pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/7 "2014-03-21T19:47:19Z")
What I'm interested in would be a perspective that ES nodes could communicate with other ES nodes by transparent (readable) data streams specified by an "ES node protocol", independent of Java serialization. So, ES nodes in the long run could be implemented in even another language on the JVM that may not be able to handle the internals of Java serialization stream protocol. I know this could be implemented by a plugin. But a standard, transparent node communication protocol in the core would even be better. Looking back in Java history there were two visible breaking serialization changes. First was the jump from 1.1 to 1.2, when 1.1 JVMs couldn't read objects from 1.2 JVMs over the network any more. And second was for the 1.5 JVMs when enums were added to the serialization. This broke the communication with 1.4 JVMs. That was all before ES, now we don't have to care about any JVM \<6. And recently there was the invisible, hard to reveal breakage, the InetAddress class, due to to an undisclosed "security fix", and yes, it's a bug that can not be fixed from within ES. Let's cross fingers the next serialization break will not take place so soon. Maybe when Project Jigsaw has been addressed in Java 9 and ES must move to Java 9. That feels like a very long way ahead. So I'm fine with waiting for when the time is right. Jörg On Fri, Mar 21, 2014 at 8:07 PM, kimchy [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Not trivializing the bug at all, god knows I spend close to a week tracing > it down to a JVM backward incompatibility change, but this happened once > over the almost 5 years Elasticsearch existed. To introduce a workaround to > something that happened once, compared to potential bugs in the workaround > (Jackson is great, but what would happen if there was a bug in it for > example) is not a great solution. Obviously, if this happened more often, > then this is something we need to address. > > On Friday, March 21, 2014 7:12:02 PM UTC+1, Chris Berry wrote: > > > If it happened once, then by definition it will happen again. History > > repeats itself. 😉 > > > > What exactly would you lose? > > You are simply trading one rigid serialization scheme for another more > > lenient one. > > Yes, you would have to introduce something like Jackson's Object Mapper, > > but that seems to be the defacto standard today and with your use of the > > Shade Plugin it wouldn't really be a burden on the Client anyway. > > > > With all due respect, you may be trivializing the impact of this one time > > bug. > > It is difficult, at best, to inform all the Clients of your Cluster; > > "Hey, if you want to see what your Exceptions really are, then upgrade your > > JVM" > > Especially in large SOA shops > > > > This just decouples the Client and Server deployments. > > > > Thanks much, > > -- Chris > > > > On Mar 21, 2014, at 12:18 PM, kimchy [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > I wonder why you are asking for this feature? If its because Java broke > > backward comp. on serialization of InetAddress that we use in our > > exceptions, then its a bug in Java serialization, hard for us to do > > something about it. > > > > You will loose a lot by trying to serialize exceptions using JSON, and we > > prefer not to introduce dependency on ObjectMapper in Jackson, or try and > > serialize exceptions using Jackson. > > > > I would be very careful in introducing this just because of a (one time > > bug) in Java. > > > > On Friday, March 21, 2014 5:18:38 PM UTC+1, Chris Berry wrote: > > > > > Greetings, > > > > > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It > > > is a beautiful tool. > > > > > > So, that said, it surprises me that Elasticsearch has such a pedestrian > > > flaw, and serializes it's Exceptions using Java Serialization. > > > In a big shop it is quite difficult (i.e. next to impossible) to keep > > > all the ES Clients on the same exact JVM as Elasticsearch, and thus, it is > > > not uncommon to get TransportSerializationExceptions instead of the > > > actual underlying problem. > > > I was really hoping this would be corrected in ES 1.0.X, but no such > > > luck. (As far as I can tell...) > > > > > > It seems that this is pretty easily fixed? > > > Just switch to a JSON representation of the basic Exception and > > > gracefully (forwards-compatibly) attempt to re-hydrate the actual Exception > > > class. > > > You'd just have to drop an additional "header" in the stream that tells > > > the code it is a JSON response and route to the right Handler it > > > accordingly. If the header is missing, then do things the old way with Java > > > Serialization?? > > > > > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > > > It may seem just an annoyance, but it is actually pretty bad, in that it > > > keeps Clients from seeing their real issues. Especially in shops where it > > > is difficult to see the Production logs of Elasticsearch itself. > > > > > > Thanks much, > > > -- Chris > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/) > > topic/elasticsearch/7bpam7mWjY8/unsubscribe. > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0% > > [40googlegroups.com](http://40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEJ%3D%2B2Uc7Kv3GBhC1Wfr3-\_aYMB02xy%3DSrCfvszBtA5yQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEJ%3D%2B2Uc7Kv3GBhC1Wfr3-_aYMB02xy%3DSrCfvszBtA5yQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Chris\_Berry\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_berry_2/32/1366_2.png) [@Chris\_Berry\_2](https://discuss.elastic.co/u/Chris_Berry_2)\ **Post date:** [March 21, 2014, 8:05pm UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/8 "2014-03-21T20:05:11Z")
We've built our own Elasticsearch Client that has niceties like; OAuth, the ability to swap Clusters for maintenance, Zookeeper integration, ease of configuration, retries, etc. Otherwise we'd have a lot of "wheel reinventing" going on. Plus, the Java Client is pretty nice, after all. 😉 Thanks for all the input. (And kimchy, thanks for the brilliant product) Cheers, -- Chris On Mar 21, 2014, at 2:47 PM, Matt Weber [matt.weber@gmail.com](mailto:matt.weber@gmail.com) wrote: > If this is a concern, why not have your client's use the REST api so they don't need to worry about matching their java version with the java version of the search cluster? > > Thanks, > Matt Weber > > On Fri, Mar 21, 2014 at 12:07 PM, kimchy [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Not trivializing the bug at all, god knows I spend close to a week tracing it down to a JVM backward incompatibility change, but this happened once over the almost 5 years Elasticsearch existed. To introduce a workaround to something that happened once, compared to potential bugs in the workaround (Jackson is great, but what would happen if there was a bug in it for example) is not a great solution. Obviously, if this happened more often, then this is something we need to address. > > On Friday, March 21, 2014 7:12:02 PM UTC+1, Chris Berry wrote: > If it happened once, then by definition it will happen again. History repeats itself. 😉 > > What exactly would you lose? > You are simply trading one rigid serialization scheme for another more lenient one. > Yes, you would have to introduce something like Jackson's Object Mapper, but that seems to be the defacto standard today and with your use of the Shade Plugin it wouldn't really be a burden on the Client anyway. > > With all due respect, you may be trivializing the impact of this one time bug. > It is difficult, at best, to inform all the Clients of your Cluster; "Hey, if you want to see what your Exceptions really are, then upgrade your JVM" > Especially in large SOA shops > > This just decouples the Client and Server deployments. > > Thanks much, > -- Chris > > On Mar 21, 2014, at 12:18 PM, kimchy [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > I wonder why you are asking for this feature? If its because Java broke backward comp. on serialization of InetAddress that we use in our exceptions, then its a bug in Java serialization, hard for us to do something about it. > > > > You will loose a lot by trying to serialize exceptions using JSON, and we prefer not to introduce dependency on ObjectMapper in Jackson, or try and serialize exceptions using Jackson. > > > > I would be very careful in introducing this just because of a (one time bug) in Java. > > > > On Friday, March 21, 2014 5:18:38 PM UTC+1, Chris Berry wrote: > > Greetings, > > > > Let me say up-front, I am a huge fan and proponent of Elasticsearch. It is a beautiful tool. > > > > So, that said, it surprises me that Elasticsearch has such a pedestrian flaw, and serializes it's Exceptions using Java Serialization. > > In a big shop it is quite difficult (i.e. next to impossible) to keep all the ES Clients on the same exact JVM as Elasticsearch, and thus, it is not uncommon to get TransportSerializationExceptions instead of the actual underlying problem. > > I was really hoping this would be corrected in ES 1.0.X, but no such luck. (As far as I can tell...) > > > > It seems that this is pretty easily fixed? > > Just switch to a JSON representation of the basic Exception and gracefully (forwards-compatibly) attempt to re-hydrate the actual Exception class. > > You'd just have to drop an additional "header" in the stream that tells the code it is a JSON response and route to the right Handler it accordingly. If the header is missing, then do things the old way with Java Serialization?? > > > > Are there any plans to fix this? Or perhaps to entertain a Pull Request? > > It may seem just an annoyance, but it is actually pretty bad, in that it keeps Clients from seeing their real issues. Especially in shops where it is difficult to see the Production logs of Elasticsearch itself. > > > > Thanks much, > > -- Chris > > > > -- > > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6ae5f173-a2b4-435c-8e5d-a43d377e2fb0%40googlegroups.com). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/891337f7-230f-4ce2-a2b4-57749f095748%40googlegroups.com). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit [https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/7bpam7mWjY8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoDB%2BVqpKvc3BPbnX2COxpwMi35GmON1bBKD74u9APJ2HQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJ3KEoDB%2BVqpKvc3BPbnX2COxpwMi35GmON1bBKD74u9APJ2HQ%40mail.gmail.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/BBE77D8F-DCCD-491B-A610-B4C568F2224A%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/BBE77D8F-DCCD-491B-A610-B4C568F2224A%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:41am UTC](https://discuss.elastic.co/t/java-serialization-of-exceptions/16528/9 "2017-07-06T01:41:31Z")