# Is it possible to use APM server without TLS to receive OpenTelemetry data?

**URL:** <https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844>\
**Category:** APM\
**Tags:** open-telemetry\
**Created:** [July 26, 2024, 9:00am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844 "2024-07-26T09:00:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![seikyo-cho-lvgs](https://avatars.discourse-cdn.com/v4/letter/s/90ced4/32.png) [@seikyo-cho-lvgs](https://discuss.elastic.co/u/seikyo-cho-lvgs)\
**Post date:** [July 26, 2024, 9:00am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844/1 "2024-07-26T09:00:57Z")

</div>

Install OpenTelemetry Operator

```auto
helm install opentelemetry-operator open-telemetry/opentelemetry-operator \
--set "manager.collectorImage.repository=otel/opentelemetry-collector-k8s"

```

Create OpenTelemetry Collector

```auto
cat <<EOF | kubectl apply -f -
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: my-collector
spec:
  mode: deployment
  config: |
    receivers:
      otlp:
        protocols:
          http:
            endpoint: ":4318"
          grpc:
            endpoint: ":4317"

    processors:
      memory_limiter:
        check_interval: 1s
        limit_mib: 2000

    exporters:
      otlp/elastic:
        endpoint: apm-server-quickstart-apm-http.default.svc.cluster.local:8200
        headers:
          Authorization: "Bearer elastic"

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [memory_limiter]
          exporters: [otlp/elastic]
EOF

```

Install APM server  
(Elasticsearch and Kibana have been installed with the same version in ECK CRD)

```auto
cat <<EOF | kubectl apply -f -
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
  name: apm-server-quickstart
  namespace: default
spec:
  version: 8.13.4
  count: 1
  elasticsearchRef:
    name: quickstart
  kibanaRef: 
    name: quickstart
EOF

```

Create APM Agent for OpenTelemetry in Kibana

 ![agent](https://us1.discourse-cdn.com/elastic/original/3X/5/1/512eb10c3891a8aab0fdae9b61bf5fd78eb89490.png)

Port forward

```auto
kubectl port-forward service/apm-server-quickstart-apm-http 8200
kubectl port-forward service/my-collector-collector 4317:4317
kubectl port-forward service/my-collector-collector 4318:4318

```

## Method 1

Set OTEL environment variables and start an application  
(Isn't there a variable for trace like `OTEL_TRACES_EXPORTER`?)

```auto
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp
export OTEL_RESOURCE_ATTRIBUTES=service.name=testService,service.version=1.1,deployment.environment=production
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:8200
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer elastic"
node app.js

```

Access the application.  
There isn't any log sent to APM server.

## Method 2

Set OTEL environment variables and start an application

```auto
export OTEL_TRACES_EXPORTER="otlp"
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
export OTEL_NODE_RESOURCE_DETECTORS="env,host,os"
export OTEL_SERVICE_NAME="test-service"
export NODE_OPTIONS="--require @opentelemetry/auto-instrumentations-node/register"
node app.js

```

Access the application.  
Got these logs in OpenTelemetry Collecotor's `my-collector-collector` pod:

```auto
2024-07-26T08:33:49.989Z	warn	zapgrpc/zapgrpc.go:193	[core] [Channel #1 SubChannel #6]grpc: addrConn.createTransport failed to connect to {Addr: "10.43.99.11:8200", ServerName: "apm-server-quickstart-apm-http.default.svc.cluster.local:8200", }. Err: connection error: desc = "transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate is valid for apm-server-quickstart-apm-http.default.apm.local, apm-server-quickstart-apm-http, apm-server-quickstart-apm-http.default.svc, apm-server-quickstart-apm-http.default, not apm-server-quickstart-apm-http.default.svc.cluster.local"	{"grpc_log": true}
2024-07-26T08:33:49.989Z	info	exporterhelper/retry_sender.go:118	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "traces", "name": "otlp/elastic", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate is valid for apm-server-quickstart-apm-http.default.apm.local, apm-server-quickstart-apm-http, apm-server-quickstart-apm-http.default.svc, apm-server-quickstart-apm-http.default, not apm-server-quickstart-apm-http.default.svc.cluster.local\"", "interval": "4.620562738s"}

```

If use this in OpenTelemetryCollector:

```auto
    exporters:
      otlp/elastic:
        endpoint: apm-server-quickstart-apm-http:8200
        headers:
          Authorization: "Bearer elastic"

```

Got

```auto
2024-07-26T08:48:52.383Z	warn	zapgrpc/zapgrpc.go:193	[core] [Channel #2 SubChannel #7]grpc: addrConn.createTransport failed to connect to {Addr: "10.43.99.11:8200", ServerName: "apm-server-quickstart-apm-http:8200", }. Err: connection error: desc = "transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority"	{"grpc_log": true}
2024-07-26T08:48:52.384Z	info	exporterhelper/retry_sender.go:118	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "traces", "name": "otlp/elastic", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"", "interval": "4.247949438s"}

```

If use this in OpenTelemetryCollector:

```auto
    exporters:
      otlp/elastic:
        endpoint: apm-server-quickstart-apm-http:8200
        headers:
          Authorization: "Bearer elastic"
        tls:
          insecure: true

```

Got

```auto
2024-07-26T08:52:32.436Z	warn	zapgrpc/zapgrpc.go:193	[core] [Channel #2 SubChannel #7]grpc: addrConn.createTransport failed to connect to {Addr: "10.43.99.11:8200", ServerName: "apm-server-quickstart-apm-http:8200", }. Err: connection error: desc = "error reading server preface: EOF"	{"grpc_log": true}
2024-07-26T08:52:32.436Z	info	exporterhelper/retry_sender.go:118	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "traces", "name": "otlp/elastic", "error": "rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: EOF\"", "interval": "5.575712041s"}

```

So if don't use TLS, is it possible to access APM server?  
If necessary, how to do with cert-manager?

---

<div class="post-metadata">

**Author:** ![Martin\_Slanina](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martin_slanina/32/136341_2.png) [@Martin\_Slanina](https://discuss.elastic.co/u/Martin_Slanina)\
**Post date:** [July 27, 2024, 11:25am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844/2 "2024-07-27T11:25:38Z")

</div>

Hi,

I had same issue. You can use 'insecure\_skip\_verify: true' to resolve this problem.

OTEL Collector contrib example:

```yaml
      otlp/elastic:
        # Elastic APM server https endpoint without the "https://" prefix
        endpoint: "apm-server-apm-http.elastic-system:8200"
        tls:
          insecure: false
          insecure_skip_verify: true
        headers:
            ......auth stuff.....

```

Also you need to add to Kibana manifest xpack fleet app package:  
[Documentation Link](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-apm-eck-managed-es.html)

```yaml
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana
  namespace: elastic-system
spec:
  version: 8.14.3
  count: 1
  config:
    xpack.fleet.packages:
      - name: apm
        version: latest

```

---

<div class="post-metadata">

**Author:** ![seikyo-cho-lvgs](https://avatars.discourse-cdn.com/v4/letter/s/90ced4/32.png) [@seikyo-cho-lvgs](https://discuss.elastic.co/u/seikyo-cho-lvgs)\
**Post date:** [July 29, 2024, 2:59am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844/3 "2024-07-29T02:59:40Z")

</div>

@Martin_Slanina  
Thank you. I tried that, but in my environment I got this error in `my-collector-collecotr` pod:

```auto
{"kind": "exporter", "data_type": "traces", "name": "otlp/elastic", "error": "not retryable error: Permanent error: rpc error: code = Unauthenticated desc = authentication failed", "dropped_items": 16}

```

---

<div class="post-metadata">

**Author:** ![Martin\_Slanina](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/martin_slanina/32/136341_2.png) [@Martin\_Slanina](https://discuss.elastic.co/u/Martin_Slanina)\
**Post date:** [July 29, 2024, 10:20am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844/4 "2024-07-29T10:20:18Z")

</div>

Did you take Kubernetes secret `apm-server-apm-token` -\> secret-token and put it into Authorization header in exporter config?:

```yaml
....
 endpoint: apm-server-quickstart-apm-http:8200
   headers:
     Authorization: "Bearer xxxxxxx" <------ secret token
...

```

---

<div class="post-metadata">

**Author:** ![seikyo-cho-lvgs](https://avatars.discourse-cdn.com/v4/letter/s/90ced4/32.png) [@seikyo-cho-lvgs](https://discuss.elastic.co/u/seikyo-cho-lvgs)\
**Post date:** [July 30, 2024, 12:17am UTC](https://discuss.elastic.co/t/is-it-possible-to-use-apm-server-without-tls-to-receive-opentelemetry-data/363844/5 "2024-07-30T00:17:02Z")

</div>

@Martin_Slanina  
Yes, as I wrote in the question, I set the secret to operator exporter directly:  
[![](https://us1.discourse-cdn.com/elastic/optimized/3X/5/1/512eb10c3891a8aab0fdae9b61bf5fd78eb89490_2_1248x1000.png) ](https://us1.discourse-cdn.com/elastic/optimized/3X/5/1/512eb10c3891a8aab0fdae9b61bf5fd78eb89490_2_1248x1000.png)

```auto
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: my-collector
spec:
  mode: deployment
  config: |
    receivers:
      otlp:
        protocols:
          http:
            endpoint: ":4318"
          grpc:
            endpoint: ":4317"

    processors:
      memory_limiter:
        check_interval: 1s
        limit_mib: 2000

    exporters:
      otlp/elastic:
        endpoint: apm-server-quickstart-apm-http.default.svc.cluster.local:8200
        headers:
          Authorization: "Bearer elastic"
        tls:
          insecure: false
          insecure_skip_verify: true

    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [memory_limiter]
          exporters: [otlp/elastic]

```