# IOException from apm server

**URL:** <https://discuss.elastic.co/t/ioexception-from-apm-server/241756>\
**Category:** APM\
**Tags:** java, server\
**Created:** [July 18, 2020, 5:59pm UTC](https://discuss.elastic.co/t/ioexception-from-apm-server/241756 "2020-07-18T17:59:25Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mvaradan](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@mvaradan](https://discuss.elastic.co/u/mvaradan)\
**Post date:** [July 18, 2020, 5:59pm UTC](https://discuss.elastic.co/t/ioexception-from-apm-server/241756/1 "2020-07-18T17:59:25Z")

</div>

**Kibana version:** 7.8.0  
**Elasticsearch version:** 7.8.0  
**APM Server version:** 7.8.0  
**APM Agent language and version:** Java, 1.17.0  
**Fresh install or upgraded from other version?:** Fresh

I am trying elastic APM on my dev set up with docker containers using docker-for-windows. I was able to quickly setup the elasticsearch, kibana and apm-server using the docker compose from [https://www.elastic.co/guide/en/apm/get-started/current/quick-start-overview.html](https://www.elastic.co/guide/en/apm/get-started/current/quick-start-overview.html). Modified the default apm-server.yml with these values (read\_timeout: 15s, write\_timeout: 15s and max\_event\_size: 1024000 (bytes). The docker containers came up fine. I’m trying this on a Java application (that uses jdk8.0.252 linux\_x64), which also runs in a docker container. I also downloaded the apm-agent and have it mounted in the app container folder, “elastic-apm-agent-1.17.0.jar” (from maven). I started the application using these settings (-Delastic.apm.config\_file=//elasticapm.properties)

```auto
    recording=true
    instrument=true
    service_name=ma
    hostname=ma
    environment=docker-dev
    transaction_max_spans=1000
    sanitize_field_names=<headers-to-exclude>
    ignore_urls=<urls-to-skip-like-healthy>
    server_urls=http://host.docker.internal:8200
    server_timeout=15s
    max_queue_size=2000
    api_request_time=15s
    api_request_size=1mb
    metrics_interval=15s
    application_packages=<app-java-packages>
    stack_trace_limit=100
    log_level=DEBUG

```

However, I frequently see the IOException from the server (below). I read [https://www.elastic.co/guide/en/apm/server/current/common-problems.html#io-timeout](https://www.elastic.co/guide/en/apm/server/current/common-problems.html#io-timeout) . Can the timeouts be same or do they have to be incremental? or is this a different issue?

```auto
    2020-07-18 16:44:28,492 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Sending payload to APM server failed
    java.io.IOException: Server returned HTTP response code: 400 for URL: http://host.docker.internal:8200/intake/v2/events
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1900) ~[?:1.8.0_252]
    	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_252]
    	at co.elastic.apm.agent.report.AbstractIntakeApiHandler.endRequest(AbstractIntakeApiHandler.java:129) [?:?]
    	at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.endRequest(IntakeV2ReportingEventHandler.java:162) [?:?]
    	at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.handleEvent(IntakeV2ReportingEventHandler.java:85) [?:?]
    	at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.onEvent(IntakeV2ReportingEventHandler.java:73) [?:?]
    	at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.onEvent(IntakeV2ReportingEventHandler.java:44) [?:?]
    	at co.elastic.apm.agent.shaded.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:168) [?:?]
    	at co.elastic.apm.agent.shaded.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) [?:?]
    	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
    2020-07-18 16:44:28,495 [elastic-apm-server-reporter] WARN co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - {
      "accepted": 0,
      "errors": [
        {
          "message": "read tcp 172.20.0.4:8200-\u003e172.20.0.1:41842: i/o timeout
        }
      ]
    }

```

---

<div class="post-metadata">

**Author:** ![mvaradan](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@mvaradan](https://discuss.elastic.co/u/mvaradan)\
**Post date:** [July 19, 2020, 11:41pm UTC](https://discuss.elastic.co/t/ioexception-from-apm-server/241756/2 "2020-07-19T23:41:14Z")

</div>

Consistently see the above exception when server\_timeout=30s and api\_request\_time=30s

---

<div class="post-metadata">

**Author:** ![axw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/axw/32/28197_2.png) [@axw](https://discuss.elastic.co/u/axw)\
**Post date:** [July 20, 2020, 1:00am UTC](https://discuss.elastic.co/t/ioexception-from-apm-server/241756/3 "2020-07-20T01:00:55Z")

</div>

If you change `api_request_time` in the agent, you need to ensure that the server's `read_timeout` and `write_timeout` are both greater by a significant amount. By default `api_request_time` is 10 seconds, and `read_timeout` and `write_timeout` are both 30 seconds.

The `read_timeout` and `write_timeout` config control how long the server will allow for reading the request body, and writing the response. On the other hand, `api_request_time` controls how long the request body is kept active for streaming events. If the agent keeps the request body open for longer than the server allows, it will lead to this error.

Why are you setting these config values? Typically you shouldn't need to change them, so unless you do I would recommend leaving them unset so that they use the default values.

---

<div class="post-metadata">

**Author:** ![mvaradan](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@mvaradan](https://discuss.elastic.co/u/mvaradan)\
**Post date:** [July 20, 2020, 4:35am UTC](https://discuss.elastic.co/t/ioexception-from-apm-server/241756/4 "2020-07-20T04:35:28Z")

</div>

Thanks for the response, and good to know. Since the document about these properties are kind of one liner ([https://www.elastic.co/guide/en/apm/server/current/configuration-process.html](https://www.elastic.co/guide/en/apm/server/current/configuration-process.html)), wanted to understand better the correlation b/w these properties and their significance. Specifically, for certain services / apps (legacy) where the # of tracing spans could be \> 2k with quite a few remote service invocations and DB hits, and also when the request is long running for bulk / batch processing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 10, 2020, 12:35am UTC](https://discuss.elastic.co/t/ioexception-from-apm-server/241756/5 "2020-08-10T00:35:30Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
