I just completed a similar set of steps to change elk.ico in Kibana 4.6.1 on Ubuntu (installed from .deb package);
I edited the icon at /opt/kibana/src/ui/public/images/elk.ico
(if you're on a different platform or just un-tarred the tar.gz your path may be different)
I saved my edit on top of that same elk.ico file.
I copied that file to /opt/kibana/optimize/bundles/src/ui/public/images/elk.ico
I restarted Kibana service (not sure if this was necessary).
I opened Chromium browser and cleared all my cache and history.
I opened Kibana in that browser, and it shows the modified icon on the tab and on bookmarks bar where I just saved it to.
Now I refreshed Kibana in Firefox without closing the tab or clearing history. I see the old tab icon.
I closed that Firefox tab and reopened Kibana and I see the modified icon.
Lee