It was pointed out 3 months ago here and nothing has changed - 66 PRs and a lot of bugs not being fixed.
Latest commit was made 5+ months ago.
elasticsearch-rails will be incompatible with an upcoming Rails 5.2 and it looks like nobody cares. How can we rely on a product with such bad support?
1 Like
Would you be willing to take on some of the maintainer load to assist?
5 Likes
Of course not. Can't you hire 1 more developer?
Wow! Come on man, you’re getting this stuff for free. Don’t be a
freeloader, if you see problems share the load! That’s the joy and burden
of open source 
5 Likes
Are you sure I'm using it for free? Elasticsearch is a commercial product with $104m of funding and a lot of paid subscriptions, why the hell would I work for them for free?
Btw, you're walking on the streets and breathing air for free - do you clean them up and plant new trees from time to time (also for free)? Nope? Why not?
I get your frustration, but unless you're one of the Elastic investors, or paying for Elastic support, or willing to help the rest of us do the work, this is the nature of most major open source projects. There's often a company that offers products/services behind the project. You can pay for support to influence the projects direction. You also have a fantastic option with open source you don't get with proprietary software: roll up your sleeves and create the projects direction with the rest of us!
I think a better analogy would be roads and infrastructure we use for "free" but pay for with taxes. Elasticsearch was built with other people's labor, and you get to use it for free. As with a great deal of open source, your options boil down to:
- Do the work you need yourself and/or hire someone
- Be grateful for what work others have contributed for you for free
- Pay for a sufficiently high tier of Elastic support that you can influence the direction of the project
You can get mad, but this is the nature of the world we live in. You're free to use Elasticsearch for free or move on to other projects. Solr has sunspot, for example.
4 Likes
50+ people already spent their time to create and send 66 PRs. For now their time and effort is wasted, so would be mine. As I can see from you answers - nobody gives a shit about it, the only maintainer is doing nothing and it's ours problem. I'm not going to invest my time and pay for a product with such an attitude.
It's sad to know that elasticsearch-rails/ruby won't have any future, including support for Rails 5.2, unless some idiot will work for free and without any rights, so ES can sell more subscriptions and earn more money. Why don't you put it on the front page - "free labor wanted"?
1 Like
@kshnurov I am following up internally to see what we can do to improve the current state and prevent similar things from happening in the future.
I do ask for patience while we sort this out but I will provide an update as soon as I can.
5 Likes
I don't think that quite cuts it. Elastic puts out their software in an open source reasons to provide a service and grow their business. elasticsearch-rails is "elastic/elasticsearch-rails" for a reason. It shows official support. Indeed, for any project running under "elastic", you need to assign full rights to Elastic though a CLA.
Free or not, you can fail providing service. At minimum, that means informing users that you don't find supporting this project viable anymore and need to deprecate it. That's fair. Telling people that are frustrated by the lack of support to get changes merged to just take over maintainership on their own? No. That's a terrible move.
Second, you actually cannot influence Elastics direction by becoming a customer. At least not in a straight-forward fashion. Sure, you can fast-track a bugfix where your infrastructure is affected, or you can hope for 10 other customers wishing for the same feature. But by and large, Elastic is in product control and will build features that help them grow their business. I've tried. It doesn't work. It's reasonable.
Indeed, I had a customer unable to upgrade Elasticsearch because they were relying Elastics Puppet support, which was laying bare for a while. Happens. There's always someone falling through the cracks.
I'm fine with that, Elastic doesn't pretend to offer that service. What they offer is support and help with getting and keeping your product running, plus their X-Pack extensions. That's a package very much worth the price they ask for, the support people are A+ trained, take initiative and the additions are incredibly flexible and dead cheap compared to building them on their own!
Elastic support very much worth it, but let's not pretend they are something they aren't.
In my opinion, Elastic primarily does Open Source, but no FOSS. And that's also okay, it's very much in the (potential) customers interest. Software and its evolution is hard to judge and evaluate, open development and free access is a crucial component of fair dealing between vendors and customers in the software world. Also, easy contribution of bug fixes and tiny features also allows customers to exercise some steering.
For the F, it's missing the governance structure.
I think this analogy is badly thought out. Elastic is not comparable to public infrastructure, as much as many companies like for example recently Shopify would like to make us believe. Payment for service to Elastic is not a tax, it is - as I layed out above - a fee in exchange for a service. This service is not building Elasticsearch and adjacent products. Elasticsearch has no governance structure beyond Elastic.
I'm confused that you bring up this point in defense of Elastic here, as I'd absolutely agree with this point if we were talking about Solr - the Apache Foundation does indeed build FOSS infrastructure and has a ton of policy built around that, especially around projects falling out of support.
Finally, I'd like to add that Elastic doesn't have a good track record of caring about people providing free and enthusiast projects around their product except for those that get bought. My personal experience: I did for a 3 years run the largest Elasticsearch User Group in Europe (the foundation predated the foundation of the company), which literally wouldn't be featured by Elastic because I'm not an Elastic partner. I finally stopped it and renamed it to "Search Usergroup" because of additionally growing demands from Elastic community management. The icing on the cake was a proposed partner contract that explicitly included the permission to run meetups.
I literally got told by a Elastic employees "What do you expect? We're a business and do what makes income?".
I'm also fine with that, there's a reason why I now don't do community work for commercial vendors anymore.
tl;dr: Elastic offers great services and a great product. They are a (mostly) nice company with many great people working there. Let's not pretend they are not a commercial entity with a very sharp (and reasonable!) focus on profit though.
3 Likes
I can, if you need help. Got some references both in Ruby & Elasticsearch and got spare time at least for a few weeks to clean things up.
1 Like
Hello,
this thread escalated faster than I had a chance to react, unfortunately.
I would like to return to the original question, and focus us back on the very important topic: the development and maintenance of the Ruby client and the Rails integrations for Elasticsearch.
The first crucial aspect to address is that the Ruby client and the Rails integrations are two completely separate projects, which together contain eight different Rubygems (not counting the X-Pack integration, with a very different degree of problematic aspects. For any further discussion, let's keep this distintion in mind.
The Ruby client has 6 open pull requests right now, with [5 pull requests] waiting for reply from the contributor. There are also 10 issues with questions waiting for reply, and 26 issues with general discussion waiting for reply. The client and associated libraries, like elasticsearch-extensions are consistently updated, with a lot of activity from October 2017, actively tracking the changes in the Elasticsearch 6.x branch. Travis CI currently fails because of incorrect snapshot URL for the Elasticsearch 6.0 package, not with a load of actual test failures. That, to me, doesn't look like an "abandoned" project at all. If it does to our community, I'm very much in favour of a discussion about how can we improve here.
The Rails integration is a different story, and I understand the frustration expressed in the original post by @kshnurov. The project itself consists of three totally different Rubygems: the integration with ActiveRecord models, the persistence layer, and the Rails helpers and extensions. All three are pretty different when it comes to complexity, usage — and community contibutions. Just compare the download numbers between elasticsearch-model and elasticsearch-persistence: 3 million versus 600 thousands. But one of biggest sources of troubles, manifesting in the number of issues and pull requests submitted, is actually elasticsearch-persistence, which skews the perspective quite significantly.
With that in mind, let's consider the technical details. For a good example, I've been actively and publicly telling people that the ActiveRecord pattern for the elasticsearch-persistence is too challenging from the development and usage standpoint, and that it's going to be deprecated in favour of the Repository pattern. Now, you can hold it against the maintainer, in this case me, to not act on such a plan in a very long time. Doing this transition properly, which means updating the code with deprecation notices, updating the Rails example application so people are not left out in the cold with a "deprecation notice", writing a blog post about this transition and the motivation, would help the community very much, and I am personally frustrated that it didn't happen.
Another good example are the pull requests themselves. As is the case with any larger open source project, there's a tension between the stability of the library and the agility with updating the code. This is very important to keep in mind.
In optimal cases, the project strikes a good balance, and the Rubygems for the Rails integration current do not achieve this. The reasons for that are predictable, no matter how much frustrating they can be: merging a contribution is not as easy as clicking the button at Github. For many pull requests — and here I want to repeat what I've said on a similar thread couple of months ago — the first complication is that it's not evident what is actually the real problem, and if the solution is in a correct part of the source code. This is amplified by the pull request not having tests, for example. The burden is than on the maintainer to: understand the problem at hand, verify the correctnes of the solution, provide code-based evidence for the problem in a unit/integration test, rework the entire contribution, and push it to the repository. As you might imagine, you may easily spend an hour or two doing that, and that significantly impacts the agility aspect mentioned above.
In a situation like that, a maintainer — me, in this case — can make a conscious decision to focus more on the stability aspect, in order to not break people's applications. Granted, many people and contributors will be frustrated by the lack of said agility, but a much more greater number of people would be impacted by careless, "agile" approach to updating the codebase.
With all that said, we are still actively trying to find a good solution here, in order to be more agile with the development and maintenance of the Rubygems for Rails integration. Of course, you may wonder, what's so hard about that for a company with $104m of funding, that's just silly, no? But in reality, it is surprisingly hard.
Returning back to you original question: "are the Rubygems for Elasticsearch abandoned?", the short answer is "no, not at all".
The larger answer is that in a tight situation, we made a conscious decision to keep focus on the Ruby client, which is the foundation for any other Ruby integration and application, and to actively search for a solution for developing and maintaining the Rubygems for Rails integration without breaking existing applications.
In this thread, I have tried to provide an honest answer to the issue raised — I do realize it's quite long, hopefuly it's not overwhelming, and will at least a little bit assure you that what you see is not just mere negligence and that the situation is a bit more complex than it seems from a first sight.
Best,
Karel
9 Likes
Hi @fdevillamil, please ping me over email, or via Github issues, and we can definitely work on a plan!
@karmi you've said the same thing 3 months ago. You're telling that elasticsearch-ruby is actively developed, which is definitely not true. Coming once in half a year and merging 6+ months old PRs is not an active development. Neither is tons of unanswered issues. According to the links you've provided people are waiting for YOUR response for many months.
In elasticsearch-rails the situation is even worse - 100+ of unanswered issues. You're telling that elasticsearch-persistence is a problem, which is another lie. How about this PR, fixing elasticsearch-model compatibility with Rails 5.2 and laying there for 2 months? What about another 50, not related to persistence in any way?
Thanks to github it's pretty easy to see all the activity. For me it looks like doing nothing since May:
https://github.com/elastic/elasticsearch-rails/graphs/commit-activity
https://github.com/elastic/elasticsearch-ruby/graphs/commit-activity
I must admit I'm not sure what is exactly your point now, @kshnurov.
In my last reply, I've tried to demonstrate that there's a significant difference between the elasticsearch-ruby and the elasticsearch-rails projects. This difference applies, in my opinion, to the maintenance aspect as well. The Ruby client, especially, has it's ebbs and flows, because it's closely tied to changes in the Elasticsearch API. I'm afraid I don't have a good idea about how the "commit activity" should look like to be considered "healthy" or how it is, in general, connected to the maintenance aspect.
The libraries for Rails integrations are a different story, as I have said myself in the reply, and yes, we are still looking for a good solution, as I have said myself in my reply. We are actively trying to come up with a solution — it is just more complicated than we ourselves thought it would be.
Technical detail about the changed_attributes fix for Rails 5.2 pull request you mention: the elasticsearch-model Rubygem still keeps the compatibility with Rails 3, 4 and 5. It is of course questionable if we should support an ancient Rails version like 3, effectively obsoleted by Rails 5, which was released a year ago, but I don't want to make such an invasive change on a whim, just because a single pull request makes the decision. This is what sometimes makes a supposedly simple process like merging a pull request quite tortuous...
3 Likes
@karmi, my point is pretty simple: you're completely ignoring your job (it is a job, isn't it?) and producing only excuses: hard, complicated, difficult.
What's so difficult in adding one more instance_methods.include? for changed_attributes case? Is it too complicated for you to ask PR's author to do this? What's so hard in at least responding to PR's?
@kshnurov I'd really like that this conversion stays on a technical level. Please don't attack people like this. You have no idea what @karmi is working on in the company.
You raised valid points about your concerns regarding those repositories and I'd appreciate that you keep the conversation very professional.
Thanks for understanding.
FYI: https://www.elastic.co/community/codeofconduct
Be respectful. Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It's important to remember that a community where people feel uncomfortable or threatened is not a productive one. We expect members of the Elastic community to be respectful when communicating with other community members, as well as with people outside the Elastic community.
8 Likes
@dadoonet I'm sorry for that. Unfortunately on technical level there's absolutely no conversation since @karmi simply doesn't respond to any of PRs.
@kshnurov I understand your frustration, as I have also experienced the lack of maintenance for some Elastic components that we have been/are still using.
Unfortunately it doesn't seem like maintaining all of these open source components is necessarily someone's full time job - I think in a lot of cases the maintainer has moved on to other projects within the company and maintains the repository only as much as his spare time can allow.
Thankfully, since it is open source, you don't have to wait for Elastic to fix it - you can fork the repository and make the necessary changes yourself. We have done this for several components, and while it does force us to maintain our own fork and periodically "sync" it with Elastic's version, it also gives us the agility to fix bugs that would have otherwise blocked us from using the component at all.
We have done this with the Logstash ElasticSearch output and with the ElasticSearch Spark connector, and will probably have to do it with the logstash clone filter as well (as it has a major bug that is being ignored).
I think your time would be better spent if you went ahead and fixed whatever needs fixing, and adapted it to your needs.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.