# Is it possible to send gzip compressed requests to ES? **URL:** \ **Category:** Elasticsearch\ **Created:** [February 23, 2013, 12:35pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859 "2013-02-23T12:35:35Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![C\_Keser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/c_keser/32/2474_2.png) [@C\_Keser](https://discuss.elastic.co/u/C_Keser)\ **Post date:** [February 23, 2013, 12:35pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/1 "2013-02-23T12:35:35Z")
Hi, I have a visual query generator in my application from which user can generate complex and large ES queries. As these queries may be quite large (a few megabytes), I want to compress the search request I'm sending to ES. Trying this with curl has proved unsuccessful so far, using the command: curl -v -i --compressed -H "Content-encoding: gzip" -X POST --data @req.txt.gz [http://localhost:9200/test1/\_search?pretty=true](http://localhost:9200/test1/_search?pretty=true) ES just replies with a generic search response (the one you get when you send an empty post request to search url). If I remove the content encoding header ES replies with a SearchParseException (as expected because it can't parse compressed bytes as JSON). So, am I missing something here or ES just does not support compressed requests? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 23, 2013, 1:49pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/2 "2013-02-23T13:49:45Z")
On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > Hi, > > I have a visual query generator in my application from which user can > generate complex and large ES queries. As these queries may be quite > large (a few megabytes), I want to compress the search request I'm > sending to ES. Trying this with curl has proved unsuccessful so far, > using the command: You need to enable http compression on each node, by adding this to your config file: ``` http.compression: true ``` See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/http.html) clint > curl -v -i --compressed -H "Content-encoding: gzip" -X POST --data > @req.txt.gz [http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) > > ES just replies with a generic search response (the one you get when > you send an empty post request to search url). If I remove the content > encoding header ES replies with a SearchParseException (as expected > because it can't parse compressed bytes as JSON). > > So, am I missing something here or ES just does not support compressed > requests? > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![C\_Keser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/c_keser/32/2474_2.png) [@C\_Keser](https://discuss.elastic.co/u/C_Keser)\ **Post date:** [February 23, 2013, 1:56pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/3 "2013-02-23T13:56:26Z")
Already did that. Also doesn't setting http.compression affect the responses (from ES) only? I want ES to accept compressed requests. 23 Şubat 2013 Cumartesi 15:49:45 UTC+2 tarihinde Clinton Gormley yazdı: > On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > > > Hi, > > > > I have a visual query generator in my application from which user can > > generate complex and large ES queries. As these queries may be quite > > large (a few megabytes), I want to compress the search request I'm > > sending to ES. Trying this with curl has proved unsuccessful so far, > > using the command: > > You need to enable http compression on each node, by adding this to your > config file: > > ``` > http.compression: true > > ``` > > See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/http.html) > > clint > > > curl -v -i --compressed -H "Content-encoding: gzip" -X POST --data > > @req.txt.gz [http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) > > > > ES just replies with a generic search response (the one you get when > > you send an empty post request to search url). If I remove the content > > encoding header ES replies with a SearchParseException (as expected > > because it can't parse compressed bytes as JSON). > > > > So, am I missing something here or ES just does not support compressed > > requests? > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [February 23, 2013, 2:14pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/4 "2013-02-23T14:14:30Z")
On a different note , can i compress the response from ES ? Thanks Vineeth On Sat, Feb 23, 2013 at 7:26 PM, C. Keser [ckeser@gmail.com](mailto:ckeser@gmail.com) wrote: > Already did that. Also doesn't setting http.compression affect the > responses (from ES) only? I want ES to accept compressed requests. > > 23 Şubat 2013 Cumartesi 15:49:45 UTC+2 tarihinde Clinton Gormley yazdı: > > > On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > > > > > Hi, > > > > > > I have a visual query generator in my application from which user can > > > generate complex and large ES queries. As these queries may be quite > > > large (a few megabytes), I want to compress the search request I'm > > > sending to ES. Trying this with curl has proved unsuccessful so far, > > > using the command: > > > > You need to enable http compression on each node, by adding this to your > > config file: > > > > ``` > > http.compression: true > > > > ``` > > > > See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/ **guide/reference/modules/http.** html)[http://www.elasticsearch.org/guide/reference/modules/http.html](http://www.elasticsearch.org/guide/reference/modules/http.html) > > > > clint > > > > > curl -v -i --compressed -H "Content-encoding: gzip" -X POST --data > > > @req.txt.gz [http://localhost:9200/test1/\_\*\*search\?pretty\=true](http://localhost:9200/test1/_**search%5C?pretty%5C=true)[http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) > > > > > > ES just replies with a generic search response (the one you get when > > > you send an empty post request to search url). If I remove the content > > > encoding header ES replies with a SearchParseException (as expected > > > because it can't parse compressed bytes as JSON). > > > > > > So, am I missing something here or ES just does not support compressed > > > requests? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com). > > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 23, 2013, 2:16pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/5 "2013-02-23T14:16:53Z")
On Sat, 2013-02-23 at 05:56 -0800, C. Keser wrote: > Already did that. Also doesn't setting http.compression affect the > responses (from ES) only? I want ES to accept compressed requests. Oh right - hmmm... interesting No idea I'm afraid > 23 Şubat 2013 Cumartesi 15:49:45 UTC+2 tarihinde Clinton Gormley > yazdı: > On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > \> Hi, > \> > \> > \> I have a visual query generator in my application from which > user can > \> generate complex and large ES queries. As these queries may > be quite > \> large (a few megabytes), I want to compress the search > request I'm > \> sending to ES. Trying this with curl has proved unsuccessful > so far, > \> using the command: > > ``` > You need to enable http compression on each node, by adding > this to your > config file: > > http.compression: true > > See > http://www.elasticsearch.org/guide/reference/modules/http.html > > clint > > > > > > > > curl -v -i --compressed -H "Content-encoding: gzip" -X POST > --data > > @req.txt.gz http://localhost:9200/test1/_search\?pretty > \=true > > > > > > > > > > > > ES just replies with a generic search response (the one you > get when > > you send an empty post request to search url). If I remove > the content > > encoding header ES replies with a SearchParseException (as > expected > > because it can't parse compressed bytes as JSON). > > > > > > So, am I missing something here or ES just does not support > compressed > > requests? > > > > -- > > You received this message because you are subscribed to the > Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails > from it, send > > an email to elasticsearc...@googlegroups.com. > > For more options, visit > https://groups.google.com/groups/opt_out. > > > > > > ``` > > -- > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [February 23, 2013, 2:18pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/6 "2013-02-23T14:18:41Z")
On Sat, 2013-02-23 at 19:44 +0530, Vineeth Mohan wrote: > On a different note , can i compress the response from ES ? yes - set http.compression: true, and include this header in your request: Accept-Encoding: gzip, deflate clint > Thanks > Vineeth > > On Sat, Feb 23, 2013 at 7:26 PM, C. Keser [ckeser@gmail.com](mailto:ckeser@gmail.com) wrote: > Already did that. Also doesn't setting http.compression affect > the responses (from ES) only? I want ES to accept compressed > requests. > > ``` > 23 Şubat 2013 Cumartesi 15:49:45 UTC+2 tarihinde Clinton > Gormley yazdı: > On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > > Hi, > > > > > > I have a visual query generator in my application > from which user can > > generate complex and large ES queries. As these > queries may be quite > > large (a few megabytes), I want to compress the > search request I'm > > sending to ES. Trying this with curl has proved > unsuccessful so far, > > using the command: > > You need to enable http compression on each node, by > adding this to your > config file: > > http.compression: true > > See > http://www.elasticsearch.org/guide/reference/modules/http.html > > clint > > > > > > > > curl -v -i --compressed -H "Content-encoding: gzip" > -X POST --data > > @req.txt.gz http://localhost:9200/test1/_search > \?pretty\=true > > > > > > > > > > > > ES just replies with a generic search response (the > one you get when > > you send an empty post request to search url). If I > remove the content > > encoding header ES replies with a > SearchParseException (as expected > > because it can't parse compressed bytes as JSON). > > > > > > So, am I missing something here or ES just does not > support compressed > > requests? > > > > -- > > You received this message because you are subscribed > to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving > emails from it, send > > > an email to elasticsearc...@googlegroups.com. > > For more options, visit > https://groups.google.com/groups/opt_out. > > > > > > ``` -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [February 23, 2013, 8:57pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/7 "2013-02-23T20:57:38Z")
On Saturday, February 23, 2013 2:56:26 PM UTC+1, C. Keser wrote: > Already did that. Also doesn't setting http.compression affect the > responses (from ES) only? I want ES to accept compressed requests. this setting should trigger ES to accept gzip too. I think you need to write "Content-Encoding" with Uppercase first letters can you try this: curl -v -i --compressed -H "Content-Encoding: gzip" -X POST --data @req.txt.gz [http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) simon > 23 Şubat 2013 Cumartesi 15:49:45 UTC+2 tarihinde Clinton Gormley yazdı: > > > On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > > > > > Hi, > > > > > > I have a visual query generator in my application from which user can > > > generate complex and large ES queries. As these queries may be quite > > > large (a few megabytes), I want to compress the search request I'm > > > sending to ES. Trying this with curl has proved unsuccessful so far, > > > using the command: > > > > You need to enable http compression on each node, by adding this to your > > config file: > > > > ``` > > http.compression: true > > > > ``` > > > > See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/http.html) > > > > clint > > > > > curl -v -i --compressed -H "Content-encoding: gzip" -X POST --data > > > @req.txt.gz [http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) > > > > > > ES just replies with a generic search response (the one you get when > > > you send an empty post request to search url). If I remove the content > > > encoding header ES replies with a SearchParseException (as expected > > > because it can't parse compressed bytes as JSON). > > > > > > So, am I missing something here or ES just does not support compressed > > > requests? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![C\_Keser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/c_keser/32/2474_2.png) [@C\_Keser](https://discuss.elastic.co/u/C_Keser)\ **Post date:** [February 23, 2013, 9:37pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/8 "2013-02-23T21:37:43Z")
HTTP headers are case insensitive according to specs but i gave it a try anyway; the response is still a generic search response... I wonder if anyone else is able to reproduce this issue...? 23 Şubat 2013 Cumartesi 22:57:38 UTC+2 tarihinde simonw yazdı: > On Saturday, February 23, 2013 2:56:26 PM UTC+1, C. Keser wrote: > > > Already did that. Also doesn't setting http.compression affect the > > responses (from ES) only? I want ES to accept compressed requests. > > this setting should trigger ES to accept gzip too. I think you need to > write "Content-Encoding" with Uppercase first letters can you try this: > > curl -v -i --compressed -H "Content-Encoding: gzip" -X POST --data @req. > txt.gz [http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) > > simon > > > 23 Şubat 2013 Cumartesi 15:49:45 UTC+2 tarihinde Clinton Gormley yazdı: > > > > > On Sat, 2013-02-23 at 04:35 -0800, C. Keser wrote: > > > > > > > Hi, > > > > > > > > I have a visual query generator in my application from which user can > > > > generate complex and large ES queries. As these queries may be quite > > > > large (a few megabytes), I want to compress the search request I'm > > > > sending to ES. Trying this with curl has proved unsuccessful so far, > > > > using the command: > > > > > > You need to enable http compression on each node, by adding this to your > > > config file: > > > > > > ``` > > > http.compression: true > > > > > > ``` > > > > > > See [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/http.html) > > > > > > clint > > > > > > > curl -v -i --compressed -H "Content-encoding: gzip" -X POST --data > > > > @req.txt.gz [http://localhost:9200/test1/\_search\?pretty\=true](http://localhost:9200/test1/_search%5C?pretty%5C=true) > > > > > > > > ES just replies with a generic search response (the one you get when > > > > you send an empty post request to search url). If I remove the content > > > > encoding header ES replies with a SearchParseException (as expected > > > > because it can't parse compressed bytes as JSON). > > > > > > > > So, am I missing something here or ES just does not support compressed > > > > requests? > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [February 24, 2013, 8:32am UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/9 "2013-02-24T08:32:44Z")
Try something like this curl -XPOST -H "Accept-Encoding: gzip" -H "Content-Encoding: gzip" --data-binary @query.json.gz --compressed localhost:9200/\_search Jörg Am 23.02.13 22:37, schrieb C. Keser: > HTTP headers are case insensitive according to specs but i gave it a > try anyway; the response is still a generic search response... > > I wonder if anyone else is able to reproduce this issue...? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![C\_Keser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/c_keser/32/2474_2.png) [@C\_Keser](https://discuss.elastic.co/u/C_Keser)\ **Post date:** [February 24, 2013, 12:20pm UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/10 "2013-02-24T12:20:08Z")
Thanks Jörg, "--data-binary" flag solved the problem! 24 Şubat 2013 Pazar 10:32:44 UTC+2 tarihinde Jörg Prante yazdı: > Try something like this > > curl -XPOST -H "Accept-Encoding: gzip" -H "Content-Encoding: gzip" > --data-binary @query.json.gz --compressed localhost:9200/\_search > > Jörg > > Am 23.02.13 22:37, schrieb C. Keser: > > > HTTP headers are case insensitive according to specs but i gave it a > > try anyway; the response is still a generic search response... > > > > I wonder if anyone else is able to reproduce this issue...? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:49am UTC](https://discuss.elastic.co/t/is-it-possible-to-send-gzip-compressed-requests-to-es/10859/11 "2017-07-06T02:49:52Z")