What I try to read through the lines is that your intention is to implement a JSON binary compression mechanism.
Note that this is not possible by scripting. You'd have to dig deeper. Technically, Elasticsearch already uses JSON binary compression under the hood https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/common/xcontent/smile/SmileXContent.java The use of SMILE is internal and triggered automatically.
I understand that many HTTP client do not/can not make use of SMILE because it's a Java API feature. To add blindly another binary compression on top of SMILE just for docs with inline scripts adds significant load on Elasticsearch cluster nodes.
Instead, you would have to examine the XContent API and replace SMILE by your codec, so all server nodes and client nodes could speak the same serialization language. This is even more than an ordinary Elasticsearch plugin could do, you'd have to exchange the transport layer. I hardly can see any justification of such a replacement.
For a comparison of SMILE against MessagePack, see https://www.quora.com/How-does-MessagePack-compare-to-Jacksons-Smile-JSON-encoding/answer/Tatu-Saloranta
Note, in that context, that Elasticsearch did once support Thrift as an optional node protocol beside compressed JSON https://github.com/elastic/elasticsearch-transport-thrift but that project had been stopped because benefit is too low.