# Intercept transport client/http client actions **URL:** \ **Category:** Elasticsearch\ **Created:** [March 16, 2011, 7:57am UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108 "2011-03-16T07:57:07Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 16, 2011, 7:57am UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/1 "2011-03-16T07:57:07Z")
Hi, Is it possible to intercept transport client actions? Basically we want to intercept action such as index to make some modifications on data before inserted. There will be many client around ES and we don't want to take care of this in every client. This will be beneficial for some other reasons too. -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 16, 2011, 7:39pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/2 "2011-03-16T19:39:49Z")
This is certainly possible, though at a cost. Th proper place to do that is actually on the primary shard using some sort of callback (so you know it only happens once). The cost associated is the fact that the document source will need to be parsed and represented in memory, then allow to callback to munge it, and then serialized again. This is because the way elasticsearch parses the document is using pull parsing. Open an issue for that. Also, what kind of changes are you going to do? Will you need to RPC call other systems to do it? -shay.banon On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > Hi, > Is it possible to intercept transport client actions? Basically we want to intercept action such as index to make some modifications on data before inserted. There will be many client around ES and we don't want to take care of this in every client. This will be beneficial for some other reasons too. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 16, 2011, 9:11pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/3 "2011-03-16T21:11:25Z")
No we will not use RPC. We need this to enrich the data. Also we want to block write operations for a certain time. This interception mechanism will be useful for this purpose too. I understood the cost of deserializing. Maybe we need to think again for enrichment. But it will be a very good improvement for other needs. On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > This is certainly possible, though at a cost. Th proper place to do that > is actually on the primary shard using some sort of callback (so you know it > only happens once). > > The cost associated is the fact that the document source will need to be > parsed and represented in memory, then allow to callback to munge it, and > then serialized again. This is because the way elasticsearch parses the > document is using pull parsing. > > Open an issue for that. Also, what kind of changes are you going to do? > Will you need to RPC call other systems to do it? > > -shay.banon > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > Hi, > Is it possible to intercept transport client actions? Basically we want to > intercept action such as index to make some modifications on data before > inserted. There will be many client around ES and we don't want to take care > of this in every client. This will be beneficial for some other reasons too. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 16, 2011, 9:13pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/4 "2011-03-16T21:13:00Z")
Got you. Actually, for blocking, there is already an internally used mechanism to block operations on indices (READ, WRITE, and META), it just needs to be exposed as a public API. On Wednesday, March 16, 2011 at 11:11 PM, Mustafa Sener wrote: > No we will not use RPC. We need this to enrich the data. Also we want to block write operations for a certain time. This interception mechanism will be useful for this purpose too. I understood the cost of deserializing. Maybe we need to think again for enrichment. But it will be a very good improvement for other needs. > > On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > This is certainly possible, though at a cost. Th proper place to do that is actually on the primary shard using some sort of callback (so you know it only happens once). > > > > The cost associated is the fact that the document source will need to be parsed and represented in memory, then allow to callback to munge it, and then serialized again. This is because the way elasticsearch parses the document is using pull parsing. > > > > Open an issue for that. Also, what kind of changes are you going to do? Will you need to RPC call other systems to do it? > > > > -shay.banon > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > > > > Hi, > > > Is it possible to intercept transport client actions? Basically we want to intercept action such as index to make some modifications on data before inserted. There will be many client around ES and we don't want to take care of this in every client. This will be beneficial for some other reasons too. > > > > > > -- > > > Mustafa Sener > > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 16, 2011, 9:18pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/5 "2011-03-16T21:18:20Z")
I created the issue > > > Discussed in > http://elasticsearch-users.115913.n3.nabble.com/Intercept-transpor…t-client-http-client-actions-tp2685929p2685929.html can you explain about internal mechanism? How can we use it? On Wed, Mar 16, 2011 at 11:13 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Got you. Actually, for blocking, there is already an internally used > mechanism to block operations on indices (READ, WRITE, and META), it just > needs to be exposed as a public API. > > On Wednesday, March 16, 2011 at 11:11 PM, Mustafa Sener wrote: > > No we will not use RPC. We need this to enrich the data. Also we want to > block write operations for a certain time. This interception mechanism will > be useful for this purpose too. I understood the cost of deserializing. > Maybe we need to think again for enrichment. But it will be a very good > improvement for other needs. > > On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > This is certainly possible, though at a cost. Th proper place to do that > is actually on the primary shard using some sort of callback (so you know it > only happens once). > > The cost associated is the fact that the document source will need to be > parsed and represented in memory, then allow to callback to munge it, and > then serialized again. This is because the way elasticsearch parses the > document is using pull parsing. > > Open an issue for that. Also, what kind of changes are you going to do? > Will you need to RPC call other systems to do it? > > -shay.banon > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > Hi, > Is it possible to intercept transport client actions? Basically we want to > intercept action such as index to make some modifications on data before > inserted. There will be many client around ES and we don't want to take care > of this in every client. This will be beneficial for some other reasons too. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 16, 2011, 9:20pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/6 "2011-03-16T21:20:33Z")
Its not exposed as an API (yet?), its used internally in elasticsearch (to block operations till recovery has happened for example). But, by design, it can easily be exposed as a public API for custom "blocks" to be defined per index. So, for example, you would have an API to add a block to index and block writes. On Wednesday, March 16, 2011 at 11:18 PM, Mustafa Sener wrote: > I created the issue > [A mechanism to intercept transport client/http client actions · Issue #785 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/785) > can you explain about internal mechanism? How can we use it? > > On Wed, Mar 16, 2011 at 11:13 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > Got you. Actually, for blocking, there is already an internally used mechanism to block operations on indices (READ, WRITE, and META), it just needs to be exposed as a public API. > > On Wednesday, March 16, 2011 at 11:11 PM, Mustafa Sener wrote: > > > > > No we will not use RPC. We need this to enrich the data. Also we want to block write operations for a certain time. This interception mechanism will be useful for this purpose too. I understood the cost of deserializing. Maybe we need to think again for enrichment. But it will be a very good improvement for other needs. > > > > > > On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > This is certainly possible, though at a cost. Th proper place to do that is actually on the primary shard using some sort of callback (so you know it only happens once). > > > > > > > > The cost associated is the fact that the document source will need to be parsed and represented in memory, then allow to callback to munge it, and then serialized again. This is because the way elasticsearch parses the document is using pull parsing. > > > > > > > > Open an issue for that. Also, what kind of changes are you going to do? Will you need to RPC call other systems to do it? > > > > > > > > -shay.banon > > > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > > > > > > > > Hi, > > > > > Is it possible to intercept transport client actions? Basically we want to intercept action such as index to make some modifications on data before inserted. There will be many client around ES and we don't want to take care of this in every client. This will be beneficial for some other reasons too. > > > > > > > > > > -- > > > > > Mustafa Sener > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > -- > > > Mustafa Sener > > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 16, 2011, 9:21pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/7 "2011-03-16T21:21:34Z")
It will be a very good feature On Wed, Mar 16, 2011 at 11:20 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Its not exposed as an API (yet?), its used internally in elasticsearch > (to block operations till recovery has happened for example). But, by > design, it can easily be exposed as a public API for custom "blocks" to be > defined per index. So, for example, you would have an API to add a block to > index and block writes. > > On Wednesday, March 16, 2011 at 11:18 PM, Mustafa Sener wrote: > > I created the issue > [A mechanism to intercept transport client/http client actions · Issue #785 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/785) > can you explain about internal mechanism? How can we use it? > > On Wed, Mar 16, 2011 at 11:13 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > wrote: > > Got you. Actually, for blocking, there is already an internally used > mechanism to block operations on indices (READ, WRITE, and META), it just > needs to be exposed as a public API. > > On Wednesday, March 16, 2011 at 11:11 PM, Mustafa Sener wrote: > > No we will not use RPC. We need this to enrich the data. Also we want to > block write operations for a certain time. This interception mechanism will > be useful for this purpose too. I understood the cost of deserializing. > Maybe we need to think again for enrichment. But it will be a very good > improvement for other needs. > > On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > This is certainly possible, though at a cost. Th proper place to do that > is actually on the primary shard using some sort of callback (so you know it > only happens once). > > The cost associated is the fact that the document source will need to be > parsed and represented in memory, then allow to callback to munge it, and > then serialized again. This is because the way elasticsearch parses the > document is using pull parsing. > > Open an issue for that. Also, what kind of changes are you going to do? > Will you need to RPC call other systems to do it? > > -shay.banon > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > Hi, > Is it possible to intercept transport client actions? Basically we want to > intercept action such as index to make some modifications on data before > inserted. There will be many client around ES and we don't want to take care > of this in every client. This will be beneficial for some other reasons too. > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 16, 2011, 9:31pm UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/8 "2011-03-16T21:31:14Z")
Open an issue for this as well, so we won't forget. On Wednesday, March 16, 2011 at 11:21 PM, Mustafa Sener wrote: > It will be a very good feature > > On Wed, Mar 16, 2011 at 11:20 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > Its not exposed as an API (yet?), its used internally in elasticsearch (to block operations till recovery has happened for example). But, by design, it can easily be exposed as a public API for custom "blocks" to be defined per index. So, for example, you would have an API to add a block to index and block writes. > > On Wednesday, March 16, 2011 at 11:18 PM, Mustafa Sener wrote: > > > > > I created the issue > > > [A mechanism to intercept transport client/http client actions · Issue #785 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/785) > > > can you explain about internal mechanism? How can we use it? > > > > > > On Wed, Mar 16, 2011 at 11:13 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > Got you. Actually, for blocking, there is already an internally used mechanism to block operations on indices (READ, WRITE, and META), it just needs to be exposed as a public API. > > > > On Wednesday, March 16, 2011 at 11:11 PM, Mustafa Sener wrote: > > > > > > > > > No we will not use RPC. We need this to enrich the data. Also we want to block write operations for a certain time. This interception mechanism will be useful for this purpose too. I understood the cost of deserializing. Maybe we need to think again for enrichment. But it will be a very good improvement for other needs. > > > > > > > > > > On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > > > > > This is certainly possible, though at a cost. Th proper place to do that is actually on the primary shard using some sort of callback (so you know it only happens once). > > > > > > > > > > > > The cost associated is the fact that the document source will need to be parsed and represented in memory, then allow to callback to munge it, and then serialized again. This is because the way elasticsearch parses the document is using pull parsing. > > > > > > > > > > > > Open an issue for that. Also, what kind of changes are you going to do? Will you need to RPC call other systems to do it? > > > > > > > > > > > > -shay.banon > > > > > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > > > > > > > > > > > > Hi, > > > > > > > Is it possible to intercept transport client actions? Basically we want to intercept action such as index to make some modifications on data before inserted. There will be many client around ES and we don't want to take care of this in every client. This will be beneficial for some other reasons too. > > > > > > > > > > > > > > -- > > > > > > > Mustafa Sener > > > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > > > > > -- > > > > > Mustafa Sener > > > > > [www.ifountain.com](http://www.ifountain.com) > > > > > > -- > > > Mustafa Sener > > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Mustafa\_Sener](https://avatars.discourse-cdn.com/v4/letter/m/ed8c4c/32.png) [@Mustafa\_Sener](https://discuss.elastic.co/u/Mustafa_Sener)\ **Post date:** [March 17, 2011, 7:50am UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/9 "2011-03-17T07:50:44Z")
I created the issue > > > Discussed in > http://elasticsearch-users.115913.n3.nabble.com/Intercept-transpor…t-client-http-client-actions-tp2685929p2685929.html On Wed, Mar 16, 2011 at 11:31 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Open an issue for this as well, so we won't forget. > > On Wednesday, March 16, 2011 at 11:21 PM, Mustafa Sener wrote: > > It will be a very good feature > > On Wed, Mar 16, 2011 at 11:20 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > wrote: > > > Its not exposed as an API (yet?), its used internally in elasticsearch > > (to block operations till recovery has happened for example). But, by > > design, it can easily be exposed as a public API for custom "blocks" to be > > defined per index. So, for example, you would have an API to add a block to > > index and block writes. > > > > On Wednesday, March 16, 2011 at 11:18 PM, Mustafa Sener wrote: > > > > I created the issue > > [A mechanism to intercept transport client/http client actions · Issue #785 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/785) > > can you explain about internal mechanism? How can we use it? > > > > On Wed, Mar 16, 2011 at 11:13 PM, Shay Banon \< > > [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)\> wrote: > > > > Got you. Actually, for blocking, there is already an internally used > > mechanism to block operations on indices (READ, WRITE, and META), it just > > needs to be exposed as a public API. > > > > On Wednesday, March 16, 2011 at 11:11 PM, Mustafa Sener wrote: > > > > No we will not use RPC. We need this to enrich the data. Also we want to > > block write operations for a certain time. This interception mechanism will > > be useful for this purpose too. I understood the cost of deserializing. > > Maybe we need to think again for enrichment. But it will be a very good > > improvement for other needs. > > > > On Wed, Mar 16, 2011 at 9:39 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > > > wrote: > > > > This is certainly possible, though at a cost. Th proper place to do > > that is actually on the primary shard using some sort of callback (so you > > know it only happens once). > > > > The cost associated is the fact that the document source will need to be > > parsed and represented in memory, then allow to callback to munge it, and > > then serialized again. This is because the way elasticsearch parses the > > document is using pull parsing. > > > > Open an issue for that. Also, what kind of changes are you going to do? > > Will you need to RPC call other systems to do it? > > > > -shay.banon > > > > On Wednesday, March 16, 2011 at 9:57 AM, Mustafa Sener wrote: > > > > Hi, > > Is it possible to intercept transport client actions? Basically we want to > > intercept action such as index to make some modifications on data before > > inserted. There will be many client around ES and we don't want to take care > > of this in every client. This will be beneficial for some other reasons too. > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > > > -- > > Mustafa Sener > > [www.ifountain.com](http://www.ifountain.com) > > -- > Mustafa Sener > [www.ifountain.com](http://www.ifountain.com) -- Mustafa Sener [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:09am UTC](https://discuss.elastic.co/t/intercept-transport-client-http-client-actions/4108/10 "2017-07-06T04:09:50Z")