Well, I was using these two versions together (Elasticsearch 5.0.0-alpha5 and Kibana 5.0.0-alpha5 for linux) and tried with these two as well (Elasticsearch 5.1.1 and Kibana 5.1.1 for windows) to see if problem still persists.
Using lower version of kibana, for example (kibana 4.6.3) helped in resolving the issue.
I hope someone finds this useful.