Hello Sebastien,
We already had a problem with this so I can safely say that - yes, this can be a problem! Elasticsearch balances the shards on all nodes based on the shard count which makes sense as the best practices say 20 shards per 1GB of RAM.
We have a cluster 3 equally large nodes(cpu,RAM, storage) but our shard sizes were very different: some were 25GB per shard and some were only 1GB per shard(I know...)
In the end, one node created an alert because the storage was nearly full(>80%) while the other 2 nodes had still >40% free.
The best way to solve this is to make all shards more or less the same size. Otherwise - and that was our temporary workaround then - you can take a look at the index shard allocation. We used the allocation settings to pin the different large indices to specific nodes within the cluster to better distribute the load.
Best regards
Wolfram