# EOF while reading

**URL:** <https://discuss.elastic.co/t/eof-while-reading/319538>\
**Category:** APM\
**Tags:** go\
**Created:** [November 22, 2022, 10:58am UTC](https://discuss.elastic.co/t/eof-while-reading/319538 "2022-11-22T10:58:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![leschev](https://avatars.discourse-cdn.com/v4/letter/l/a9a28c/32.png) [@leschev](https://discuss.elastic.co/u/leschev)\
**Post date:** [November 22, 2022, 10:58am UTC](https://discuss.elastic.co/t/eof-while-reading/319538/1 "2022-11-22T10:58:40Z")

</div>

Hello!

I have a problem with debugging apm-server. I take example request body from [here](https://www.elastic.co/guide/en/apm/guide/current/api-event-example.html).

```auto
curl --request POST http://*.*.*.*:*/intake/v2/events -H "Content-Type: application/x-ndjson" -d @payload.ndjson

```

I get response:

```auto
{
  "accepted": 0,
  "errors": [
    {
      "message": "EOF while reading metadata",
      "document": "..."
    }
}

```

In code I see that exception - [link](https://github.com/elastic/apm-server/blob/2a1c5d50fff5eb2e68e589716f389682acfa5d89/internal/processor/stream/processor.go#L112). Golang documentation says:

> When Read encounters an error or end-of-file condition after successfully reading n \> 0 bytes, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call.

APM-server is installed by the installing the .deb package, not by Fleet-server. Elasticsearch and APM-Server vesrions is 8.4.1.

What an error could caused by? What about this error actually? How can I decide it?

UPD: also I can't enable APM integration in the Kibana :c

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/2/e2d3ea31b439c437c05e36c08dff80201cf68af9.png)

---

<div class="post-metadata">

**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\
**Post date:** [November 22, 2022, 12:51pm UTC](https://discuss.elastic.co/t/eof-while-reading/319538/2 "2022-11-22T12:51:12Z")

</div>

Hi @leschev ,

try using `--data-binary` instead of `-d`:  
`curl --request POST http://*.*.*.*:*/intake/v2/events -H "Content-Type: application/x-ndjson" --data-binary @payload.ndjson`

---

<div class="post-metadata">

**Author:** ![leschev](https://avatars.discourse-cdn.com/v4/letter/l/a9a28c/32.png) [@leschev](https://discuss.elastic.co/u/leschev)\
**Post date:** [November 22, 2022, 12:58pm UTC](https://discuss.elastic.co/t/eof-while-reading/319538/3 "2022-11-22T12:58:54Z")

</div>

It's done, thanks a lot!

What should I do with problem I updated in the topic? I'm talking about "No Apm Server detected".

---

<div class="post-metadata">

**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\
**Post date:** [November 25, 2022, 8:20am UTC](https://discuss.elastic.co/t/eof-while-reading/319538/4 "2022-11-25T08:20:49Z")

</div>

Happy to hear we solved the first issue.

Have you installed the APM Integration? You can follow the [guide for installing the APM Integration](https://www.elastic.co/guide/en/apm/guide/current/apm-server-configuration.html#_install_the_apm_integration). This is required for setting up index templates, ingest pipelines etc and the APM Server won't start ingesting data to Elasticsearch until it can find these assets.

---

<div class="post-metadata">

**Author:** ![leschev](https://avatars.discourse-cdn.com/v4/letter/l/a9a28c/32.png) [@leschev](https://discuss.elastic.co/u/leschev)\
**Post date:** [November 28, 2022, 11:04am UTC](https://discuss.elastic.co/t/eof-while-reading/319538/5 "2022-11-28T11:04:59Z")

</div>

I tried to install APM Integration, but when I click on "Check APM Server status" I see this error:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/3/133c4019e082ee4090bf07f680d1cc8ba8b8e125.png)

APM Server is installed as .deb package.

---

<div class="post-metadata">

**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\
**Post date:** [November 28, 2022, 5:46pm UTC](https://discuss.elastic.co/t/eof-while-reading/319538/6 "2022-11-28T17:46:23Z")

</div>

You described earlier that you managed to successfully send sample data to the APM Server. Could you verify if these data are actually indexed into Elasticsearch or if there is any error logged in the APM Server logs?

---

<div class="post-metadata">

**Author:** ![leschev](https://avatars.discourse-cdn.com/v4/letter/l/a9a28c/32.png) [@leschev](https://discuss.elastic.co/u/leschev)\
**Post date:** [November 29, 2022, 10:45am UTC](https://discuss.elastic.co/t/eof-while-reading/319538/7 "2022-11-29T10:45:29Z")

</div>

Oh, I deeped down to the problem and created and configured roles, users and index templates for apm-server:

- role `apm-server` with `all` (for test) privilleges for `metrics-apm*`, `logs-apm*` and `traces-apm*` and with `monitor`, `cluster:admin/xpack/monitoring/bulk` cluster privilleges
- user `apm-server` with role `apm-server`
- index templates `logs-apm`, `metrics-apm` and `traces-apm` configured as data streams with patterns `logs-apm*`, `metrics-apm*` and `traces-apm*`

It is absolutely true that my test requests are processing cause sample data are indexing. I see indices in the UI and in responses using Search API.

After several apm-server restarts I get a few errors.

IDK why I see this error. I have index template with neccessary index pattern:

```auto
Nov 29 10:22:26 fleet-server apm-server[11648]: {"log.level":"error","@timestamp":"2022-11-29T10:22:26.852Z","log.logger":"beater","log.origin":{
"file.name":"beater/waitready.go","file.line":62},"message":"precondition 'apm integration installed' failed: error querying Elasticsearch for in
tegration index templates: unexpected HTTP status: 404 Not Found ({\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason
\":\"index template matching [logs-apm.error] not found\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [logs-
apm.error] not found\"},\"status\":404}): to remediate, please install the apm integration: https://ela.st/apm-integration-quickstart","service.n
ame":"apm-server","ecs.version":"1.6.0"}

```

And the second error:

```auto
Nov 29 10:22:26 fleet-server apm-server[11648]: {"log.level":"error","@timestamp":"2022-11-29T10:22:26.858Z","log.logger":"kibana","log.origin":{
"file.name":"kibana/connecting_client.go","file.line":79},"message":"failed to obtain connection to Kibana: fail to get the Kibana version: fail
to unmarshal the response from GET https://{KIBANA}:443/api/status. Response: <!DOCTYPE html> <html class=\"login-pf\"> <head>
     <meta charset=\"utf-8\"> <meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" /> <meta name=\"robots\" content=\"n
oindex, nofollow\"> <link rel=\"icon\" type=\"image/x-icon\" href=\".... (truncated). Kibana status api returns: invalid character '<' lookin
g for beginning of value","service.name":"apm-server","ecs.version":"1.6.0"}

```

This happened after I configured `apm-server.kibana` section in apm-server.yml. I haven't this section before so it possible that it is cause with apm-server integration installing issue. But it is not work. I think cause is enabled SSO on Kibana side. Are there ways to configure `apm-server.kibana` to work with SSO?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 20, 2022, 6:45am UTC](https://discuss.elastic.co/t/eof-while-reading/319538/8 "2022-12-20T06:45:37Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.