@xavierfacq Nop, would that make any difference?
What I found was this. According to this I changed my aggs and it's working once I recreated the index using a PUT and then tried pushing data to that index.
Once I query, it's working now. But then without recreating the whole thing is there any way I could insert the "fielddata": true for the existing index I've got at the moment? 