User with `superuser` permissions doesn't have rights to apply ILM policies

Hello.

We introduced authentication in our cluster last week and now ILM does not work anymore.
When I look at the errors, I can see

security_exception: action [indices:monitor/stats] is unauthorized for user [read_only_kibana]

But that user has superuser permissions:

{
  "read_only_kibana" : {
    "username" : "read_only_kibana",
    "roles" : [
      "superuser"
    ],
    "full_name" : null,
    "email" : null,
    "metadata" : { },
    "enabled" : true
  }
}

Is there anything obvious we're missing? Shouldn't superuser be able to run ILM related operations?

Now I tried to delete the user and use another one, but I still see the message
security_exception: action [indices:monitor/stats] is unauthorized for user [read_only_kibana]
How is this possible? That user doesn't exist anymore:

$ curl -XGET "http://localhost:9200/_security/user/read_only_kibana"
{}

How/where can I verify with which user is ILM running?