# Beats, ECK, and Openshift 3.11

**URL:** <https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627>\
**Category:** Elastic Cloud on Kubernetes (ECK)\
**Created:** [August 3, 2020, 9:15pm UTC](https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627 "2020-08-03T21:15:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![SeanPlacchetti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanplacchetti/32/51652_2.png) [@SeanPlacchetti](https://discuss.elastic.co/u/SeanPlacchetti)\
**Post date:** [August 3, 2020, 9:15pm UTC](https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627/1 "2020-08-03T21:15:36Z")

</div>

Hi,

I started looking into deploying out the various beats based on the examples in the documentation to our OpenShift 3.11 cluster and have run into some issues. I wonder if there are quickstart examples for OpenShift given the block is generally a no-go:

```auto
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0

```

I'm setting up heartbeat to get started, it seemed the simplest option and have not been able to get the pod rolling:

```auto
6m 3h 51 heartbeat-beat-heartbeat-667744cc.1627d1f5b16c4a1d ReplicaSet Warning FailedCreate replicaset-controller Error creating: pods "heartbeat-beat-heartbeat-667744cc-" is forbidden: unable to validate against any security context constraint: [spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 0: must be in the ranges: [1000360000, 1000369999]]
4m 4m 8 heartbeat.1627dbf513377bbf Beat Warning AssociationError beat-controller Association backend for elasticsearch is not configured
4m 4m 1 heartbeat.1627dbf524a8af1b Beat Normal AssociationStatusChange beat-kibana-association-controller Association status changed from [] to [Established]
4m 4m 1 heartbeat.1627dbf539331c72 Beat Normal AssociationStatusChange beat-es-association-controller Association status changed from [] to [Established]
4m 4m 1 heartbeat-beat-heartbeat.1627dbf53d7f0ba7 Deployment Normal ScalingReplicaSet deployment-controller Scaled up replica set heartbeat-beat-heartbeat-8964c6c44 to 1
2m 4m 15 heartbeat-beat-heartbeat-8964c6c44.1627dbf53e810d28 ReplicaSet Warning FailedCreate replicaset-controller Error creating: pods "heartbeat-beat-heartbeat-8964c6c44-" is forbidden: unable to validate against any security context constraint: [spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used]

```

When I saw these event errors I ran:

```auto
oc adm policy add-scc-to-user hostaccess heartbeat
oc adm policy add-scc-to-user privileged heartbeat

```

Which lead to the logs:

```auto
4s 4s 1 heartbeat-beat-heartbeat.1627dc3081d5a1f7 Deployment Normal ScalingReplicaSet deployment-controller Scaled up replica set heartbeat-beat-heartbeat-7cfd79499d to 1
4s 4s 8 heartbeat.1627dc30531d642d Beat Warning AssociationError beat-controller Association backend for elasticsearch is not configured
4s 4s 1 heartbeat.1627dc3063753bd1 Beat Normal AssociationStatusChange beat-kibana-association-controller Association status changed from [] to [Established]
4s 4s 1 heartbeat.1627dc3079a61db7 Beat Normal AssociationStatusChange beat-es-association-controller Association status changed from [] to [Established]
1s 4s 10 heartbeat-beat-heartbeat-7cfd79499d.1627dc3082bff32d ReplicaSet Warning FailedCreate replicaset-controller Error creating: pods "heartbeat-beat-heartbeat-7cfd79499d-" is forbidden: unable to validate against any security context constraint: [spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 0: must be in the ranges: [1000360000, 1000369999]]

```

Not sure if I'm missing something. I didn't see anything in the github issues or the documentation for the beats about provisions for running on OpenShift, so any help would be great.

---

<div class="post-metadata">

**Author:** ![michael.morello](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael.morello/32/47448_2.png) [@michael.morello](https://discuss.elastic.co/u/michael.morello)\
**Post date:** [August 4, 2020, 11:43am UTC](https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627/2 "2020-08-04T11:43:42Z")

</div>

> [@SeanPlacchetti](#):
>
> `oc adm policy add-scc-to-user privileged heartbeat`

If `heartbeat` is a service account, and not a regular user, then you should use the `-z` flags:

```auto
oc adm policy add-scc-to-user privileged -z heartbeat

```

It also assumes that the service account has been set in the `podTemplate`:

```auto
    podTemplate:
      spec:
        serviceAccountName: heartbeat
        securityContext:
          runAsUser: 0

```

---

<div class="post-metadata">

**Author:** ![SeanPlacchetti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanplacchetti/32/51652_2.png) [@SeanPlacchetti](https://discuss.elastic.co/u/SeanPlacchetti)\
**Post date:** [August 4, 2020, 1:43pm UTC](https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627/3 "2020-08-04T13:43:34Z")

</div>

@michael.morello Thank you for the reply!

Sorry, I did run the full servicename path in my command, just took it out for obscuring the namespace name, should've referenced that better. What I didn't do and didn't think to even check on, was whether the servicename was being made from the yaml (I added a similar section to the metricbeat yaml example), the addition to the podTemplate showed me the error that the servicename didn't exist - not sure why that didn't work. After creating the serviceaccount like the APM documentation for OpenShift, a pod started - unfortunately, it's in crashbackoffloop with the following error:

```auto
2020-08-04T13:26:48.357Z	INFO	instance/beat.go:647	Home path: [/usr/share/heartbeat] Config path: [/usr/share/heartbeat] Data path: [/usr/share/heartbeat/data] Logs path: [/usr/share/heartbeat/logs]
2020-08-04T13:26:48.357Z	INFO	instance/beat.go:390	heartbeat stopped.
2020-08-04T13:26:48.357Z	ERROR	instance/beat.go:958	Exiting: Failed to create Beat meta file: open /usr/share/heartbeat/data/meta.json.new: permission denied
Exiting: Failed to create Beat meta file: open /usr/share/heartbeat/data/meta.json.new: permission denied

```

Here's our full yaml:

```auto
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: heartbeat
spec:
  type: heartbeat
  version: 7.8.0
  elasticsearchRef:
    name: observability-elasticsearch
  kibanaRef:
    name: observability-kibana
  config:
    heartbeat.monitors:
    - type: tcp
      schedule: '@every 5s'
      hosts: ["elasticsearch-observability-es-http.observability-namespace.svc:9200"]
    - type: tcp
      schedule: '@every 5s'
      hosts: ["observability-kibana-kb-http.observability-namespace.svc:5601"]
    - type: tcp
      schedule: '@every 5s'
      hosts: ["vendor-elasticsearch-es-http.app-namespace.svc:9200"]
    - type: tcp
      schedule: '@every 5s'
      hosts: ["vendor-kibana-kb-http.app-namespace.svc:5601"]
    - type: tcp
      schedule: '@every 5s'
      hosts: ["vendor-elasticsearch-staging-es-http.ingest-namespace.svc:9200"]
    - type: tcp
      schedule: '@every 5s'
      hosts: ["vendor-kibana-staging-kb-http.ingest-namespace.svc:5601"]
  deployment:
    replicas: 1
    podTemplate:
      spec:
        serviceAccount: heartbeat
        serviceAccountName: heartbeat
        securityContext:
          runAsUser: 0
...

```

I ran these commands to create the serviceaccount:

```auto
 1134 oc create serviceaccount heartbeat -n observability-namespace
 1135 oc adm policy add-scc-to-user anyuid -z heartbeat -n observability-namespace
 1138 oc adm policy add-scc-to-user privileged -z heartbeat

```

Any thoughts?

Thanks for your time.

---

<div class="post-metadata">

**Author:** ![SeanPlacchetti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/seanplacchetti/32/51652_2.png) [@SeanPlacchetti](https://discuss.elastic.co/u/SeanPlacchetti)\
**Post date:** [August 4, 2020, 3:55pm UTC](https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627/4 "2020-08-04T15:55:05Z")

</div>

@michael.morello Thanks again for your time, I was able to get an answer to get heartbeat running on OpenShift 3.11 with ECK through the support portal that mapped to much of what I did plus an extra step of setting a SELinux label on the App Nodes. I'm going to post it below for people looking for answers and finding this thread.

_In this example `elastic` is the name of the `Namespace` and `beats` the name of the `ServiceAccount` , replace those values by the desired ones_

Beware that if SELinux is enabled, you may hit another issue, the Pod will fail with the following message:

```auto
> kubectl logs heartbeat-beat-heartbeat-5cdf958d86-n77f2 -n elastic -f
2020-08-04T06:14:41.519Z INFO instance/beat.go:647 Home path: [/usr/share/heartbeat] Config path: [/usr/share/heartbeat] Data path: [/usr/share/heartbeat/data] Logs path: [/usr/share/heartbeat/logs]
2020-08-04T06:14:41.519Z INFO instance/beat.go:390 heartbeat stopped.
2020-08-04T06:14:41.519Z ERROR instance/beat.go:958 Exiting: Failed to create Beat meta file: open /usr/share/heartbeat/data/meta.json.new: permission denied
Exiting: Failed to create Beat meta file: open /usr/share/heartbeat/data/meta.json.new: permission denied

```

In order to fix this error the label `svirt_sandbox_file_t` must be applied to the directory `/var/lib/elastic/heartbeat/heartbeat-data/` :

```auto
chcon -Rt svirt_sandbox_file_t /var/lib/elastic/heartbeat/heartbeat-data/

```

This should be done on all the hosts where the heartbeat Pod can be deployed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 8:08am UTC](https://discuss.elastic.co/t/beats-eck-and-openshift-3-11/243627/5 "2022-11-04T08:08:43Z")

</div>


