Memlock settings not taking effect?

I still see this log when I start ES (v2.3.5) even though I have these settings in /etc/security/limits.conf, and have rebooted the machine. I also have bootstrap.mlockall: true in /etc/elasticsearch.yml.

[2016-10-04 18:51:22,933][WARN ][bootstrap                ] These can be adjusted by modifying /etc/security/limits.conf, for example:
        # allow user 'elasticsearch' mlockall
        elasticsearch soft memlock unlimited
        elasticsearch hard memlock unlimited

What OS?

CentOS 7.2. I upgraded to ES v2.4.1 now, and still see it.

Note that the setting was renamed in 2.4

bootstrap.memory_lock: true

Yes, of course he elasticsearch.yml file was updated when i installed v2.4.1