# Enterprise Search - GZip compression for App Engine API response

**URL:** <https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515>\
**Category:** Elastic Search\
**Tags:** elastic-app-search\
**Created:** [January 6, 2024, 5:42pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515 "2024-01-06T17:42:25Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 6, 2024, 5:42pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/1 "2024-01-06T17:42:25Z")

</div>

We are using Enterprise Search to build our e-commerce use cases. One of our challenges is the API response time and payload size. I am wondering if there is a way we can enable GZip compression for the Enterprise Search App Engine API. Today our payload size with uncompressed API is 4 MB and trying to reduce the size by enabling GZip compression.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 6, 2024, 6:50pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/2 "2024-01-06T18:50:01Z")

</div>

Looks like I got the answer for this question in this documentation: [API Reference | App Search documentation [8.11] | Elastic](https://www.elastic.co/guide/en/app-search/current/api-reference.html#overview-api-references-compression)

Please let me know if there is any other way.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 6, 2024, 7:19pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/3 "2024-01-06T19:19:49Z")

</div>

The `Accept-Encoding` header has no impact on the response payload from App Engine ☹

Have tried with `gzip` and `application/gzip` values for the `Accept-Encoding` header.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 9, 2024, 9:39pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/4 "2024-01-09T21:39:15Z")

</div>

@Carlos_D @Sean_Story - Do you guys have any insight into this and any suggestions?

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [January 10, 2024, 5:48pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/5 "2024-01-10T17:48:09Z")

</div>

Hi @Pradeep_Renukaiah ,

App Search does not support zipped payloads.

Can you clarify - are you trying to ingest a 4MB chunk of data? Or your search results are 4MB?

Either way, that seems like an unreasonably large payload. What is your use case? Have you considered if there are any fields you're indexing that don't need to be indexed? How many documents is that 4MB payload split between? Could you decrease your batch/page size?

For context, War and Peace, the novel by Tolstoy, is about 3MB of plain text. Shipping a novel's worth of JSON data is not something we really want to optimize for.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 10, 2024, 6:07pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/6 "2024-01-10T18:07:31Z")

</div>

@Sean_Story - I am using multi\_search api to pull the documents and the cumulative payload size is 4 MB for 360 documents and aggregations across. However, I am trying to understand how to enable gzip compression for the App Search API responses. I tried using Accept-Encoding header as per the documentation [API Reference | App Search documentation [8.11] | Elastic](https://www.elastic.co/guide/en/app-search/current/api-reference.html#overview-api-references-compression) but not able to enable compression.

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [January 10, 2024, 7:45pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/7 "2024-01-10T19:45:58Z")

</div>

Can you leverage the [result\_fields](https://www.elastic.co/guide/en/app-search/current/result-fields-highlights.html) for the individual queries in your multi\_search to reduce your response payload? Or change the page size for each of your queries in the multi\_search? I expect that you are not wanting to display the full contents of those 360 results to a user on a single page. Try to structure your request so that you only fetch the results you need at a given moment.

> I tried using Accept-Encoding header as per the documentation

Thanks, I honestly didn't realize this was a feature we'd added. TIL  
Testing locally, I am not having issues with `Accept-Encoding` working. Are you getting an error, or is it just returning un-compressed JSON?

Can you share a minimal curl command that reproduces the issue you're having?

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 10, 2024, 8:54pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/8 "2024-01-10T20:54:56Z")

</div>

There is no error but just not compressing the JSON. I tried the same payload with gzip compression enabled using a standalone server (not app engine but just a spring boot app) and returning the 4 MB payload. After compression the size became 40KB and the response time was subsecond. Without compression, the download time of the content is a few seconds.

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [January 10, 2024, 9:08pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/9 "2024-01-10T21:08:34Z")

</div>

@Pradeep_Renukaiah can you please share a minimal curl command that uses `Accept-Encoding` but does not result in compressed results? I am not having any issue getting compressed results with a request like:

```auto
curl -X POST '<enterprise search host>/api/as/v1/engines/<engine_name>/multi_search' \
-H 'Content-Type: application/json' \
-H 'Accept-Encoding: gzip' \
-H 'Authorization: Bearer <search key>' \
-d '{
  "queries": [
    {"query": ""}
  ]
}' | less

```

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 10, 2024, 9:38pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/10 "2024-01-10T21:38:04Z")

</div>

@Sean_Story - I am testing with postman and here is the curl command, however `Accept-Encoding` is not taking effect. Postman can show the package size with and without encoding:

```auto
curl --location --request GET '<enterprise search host>/api/as/v1/engines/test-engine/multi_search' \
--header 'Accept-Encoding: gzip' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <search key>' \
--data '{
    "queries": [
        {
            "query": "test",
            "filters": {
                "all": [
                    {
                        "languageId": [
                            -1
                        ],
                        "type": [
                            "article"
                        ]
                    }
                ]
            }
        },
        {
            "query": "test",
            "filters": {
                "all": [
                    {
                        "languageId": [
                            -1
                        ],
                        "type": [
                            "posts"
                        ]
                    }
                ]
            }
        },
        {
            "query": "test",
            "filters": {
                "all": [
                    {
                        "languageId": [
                            -1
                        ],
                        "type": [
                            "book"
                        ]
                    }
                ]
            }
        },
        {
            "query": "test",
            "filters": {
                "all": [
                    {
                        "languageId": [
                            -1
                        ],
                        "type": [
                            "tweet"
                        ]
                    }
                ]
            }
        }
    ]
}'

```

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [January 10, 2024, 9:57pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/11 "2024-01-10T21:57:31Z")

</div>

I'm still struggling to reproduce. Do you get JSON or gzipped content if you run my curl example?

What version of Enterprise Search are you running?

Please try executing as curl, without postman. That will help me to isolate where your issue is coming from.

---

<div class="post-metadata">

**Author:** ![Pradeep\_Renukaiah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pradeep_renukaiah/32/126499_2.png) [@Pradeep\_Renukaiah](https://discuss.elastic.co/u/Pradeep_Renukaiah)\
**Post date:** [January 10, 2024, 10:14pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/12 "2024-01-10T22:14:26Z")

</div>

This is interesting, when I tested using cURL with Accept-Encoding, the output file size is 332KB. However, when I try the same with postman or spring boot framework, the response is not compressed.

I am using Enterprise search version 8.10.2

---

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [January 10, 2024, 10:20pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/13 "2024-01-10T22:20:55Z")

</div>

Ok, so sound like the issue is with postman. I don't think I can really help you there.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 7, 2024, 10:21pm UTC](https://discuss.elastic.co/t/enterprise-search-gzip-compression-for-app-engine-api-response/350515/14 "2024-02-07T22:21:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.