I found out the root issue. The quick raising of direct memory buffers on master is caused by kibana. When I turn off kibana then the buffer pools stops raising that fast. I cannot find what exactly in kibana causes it.
In tcp dump on elasticsearch server I see thousands of "empty" packets from kibana
22:55:31.651839 IP essearch1-staging.42057 > ip-192-168-31-223.eu-west-1.compute.internal.9300: Flags [.], ack 10443091, win 12315, options [nop,nop,TS val 635171125 ecr 1515861], length 0
E..4.X@.@.....!...!..I$T....P.m...0........
%..5..!U
There is thousands of them. When I turn off kibana then these messages stops appearing.