I am strugeling with this for quite som time. And guess i've found the problem now! \o/
I tried to build the gem for logstash-filter-geocoder and everything seems to work fine, it just was not available to logstash although it said "Installation successful". However in one time it worked and i could not remember what i did differently. But the working gem had a bigger filesize.. For the solution see here: Files in gemspec.
So creating a gem from a directory not controlled by git does create some kind of "empty" gem. That is way installation seems to be successful.