Enable xpack.security but not password authentication

This is a complicated side effect of the fact that licensing for security changed in Elasticsearch 7.1 combined with the fact the license for a cluster is stored in cluster state, and dynamically changeable.

The consequence of those circumstances means that when you start a node in ES 7.x, it doesn't know what license it has, or whether security should default to on or off.
So, while we documented that security is "off" by default in ES 7.x on a basic license, that's a simplification - by default it's in a half-on state, where it supports TLS (so it can join other nodes in the cluster) and then disables the rest of security (authentication, RBAC) when it determines that the cluster has a basic license. And security can be automatically re-activated if you install a paid license into the cluster.

That behaviour does not exist in ES 8.x

Is it possible to enable xpack.security but not enforce the authentication?

Yes. You can enable security, and then turn on anonymous access. That will have roughly the same effect.
You can decide what role your anonymous users have. For example, if you have a lot of search clients, you could make the anonymous only have read access and then configure any ingest/management clients with a username/password.

As described above, implicitly disabled isn't actually the same as explicitly disabled. The default behaviour for basic licensed nodes is to have security enabled, but inactive.