Dove: We want to cheer... but have some mixed feelings

Our History

Our small team of eight has been using FeathersJs as Backend & API for about three years now. In the beginning, the first week we chose Feathers, was challenging. Feathers introduced new work and think patterns which took us some time to grasp. After that initial bump, we never looked back and started to recommend it to anybody. The community is very helpful and nice. It was great.

Stability

The software proved super stable and all just works. We never had even one serious issue in all these years. I have tremendous admiration for the team behind the tool. Especially David, @daffl is by far the nicest, most knowledgable, and most responsive founder of an open source project that I know of. I am just very grateful that such people give their energy to the community.

Nice to juniors

What is also great, is that junior developers could maintain our API. And this is a good thing when you realize that maintaining APIs is usually not the hobby of senior programmers. Of course, all juniors have this first tedious week where they need to get their heads around “hooks” and stuff before we can use Feathers, but it is worth it.

Some minor wishes

During the time, we only had a few desires:

1. It would have been nice to have Typescript support. We moved to Vue 3 on the Front-End, and there we are happy to use TS
2. We would like to get rid of the myservice.class.js files from the feathers generate service. You see, we had decided already about twenty(oops) years ago that software with abundant use of classes is not the easiest to maintain over time. It all sounds super savvy at first, but if the developers are either junior or even seniors when time passes, we better do without. I realize other people might feel different. But for us, when scaffolding new services, we just told the junior to pretend these files did not exist.
3. Documentation. We did not reach a level where we felt confident enough that we could contribute to the project with user manuals ourselves, but we knew we had to one day. Do not get me wrong, the current doc is great and we are thankful, but juniors almost always get a bit lost. They need to start Googling. Coincidentally, then there the best information again comes from David))

Now Dove

So we were super happy to learn that in September 2022 we could start a new Greenfield project, and we decided to go for Feathers 5.0, aka Dove. However, truth be told, that has been a week of mixed feelings.

The good

Typescript is omnipresent everywhere. You can not go wrong with that. Also, the project shows lots of love and care and feels robust and trustworthy.

The bad

We found upgrading from previous Feathers to be less pleasant. Even though there is a long list of migration tips, we still get a lot of errors. For example, yesterday we noticed that no longer is working

// in users.service.js
app.use('/version', { async find() { return { v: await versionInfo() } })

I know, these are small things. And while it will take us some weeks, I suppose it might make our code a bit even better.

The Ugly

In all this, the main issue with Dove is that we feel we lost the beauty of the simplicity of Feathers.

As an example, there are no longer myService.hooks.js files. This is sad, as it was a dream: simple files with the before, after and error hooks that we all understood. Perfect. Instead, Dove hides hooks in myService.class.js. And there it’s confusing, as Dove introduces around hooks. I read the documentation on that twice, but I can not get the logic. I am sure it is super savvy though.

Then the schemas. That’s the worst about the new version. Previously a new myService.model.js was about ten lines of code, without field definitions. Now in the new myService.schema.ts, we have more than fifty lines! Of course, there all kinds of magic happen and no doubt the code is more robust for query and patch and whatnot. But so much boilerplate? For each new service again? It feels scary to me and to juniors at first glance.

The list is long, but that would be too much for this post.

The future

I recall I felt the same long ago with Modx and later also with a full stack framework called Meteor. Both were amazing products created by, very smart developers. But after their initial success, new versions came that were too smart for the majority of the community. Both lost momentum and never really recovered.

Maybe with Dove, it is just the CLI and scaffolding that tries to scare us. Maybe the elegant simplicity of the previous versions can be reinstated? What can we do to help?

For now, it seems the best we can do is try to downgrade back to version 4.5, after the weekend though. It’s Friday today))

About the long term, we do not know though. And hopefully, others can weigh in. What is wise? Should we accept that the target audience for Feathers has changed to a more sophisticated, higher level of developers? Should we stay on version 4? What about long-term maintenance?

[marshallswain]

I’m adding my experience to the conversation. I get where people are coming from. I started out pushing hard against pretty much everything in this release for the same reasons.

Things I was skeptical about

I was a major skeptic on around hooks back when they were still called async hooks. I could see the value of giving a single hook access to the before and after parts of a request, but I LOVE our simple hooks API, and I typically get around the problem by storing something in params and retrieving it in the after hook. Some endpoints end up storing a lot, but I was happy with the workflow because it worked and was familiar.

I didn’t like Feathers Schemas because I loved my Joi tooling, as well. For all of the same reasons: it’s easy to learn, easy to read, and easy to write. Tough to beat.

I didn’t like Feathers Resolvers because (1) I thought they were tied to the schemas (they’re not), and (2) I was happy to use hooks like the hammer for all nails. (“When the only tool you have is a hammer, everything begins to look like a nail.”) and (3) I looked at them as a completely separate and new API, not realizing the similarities (because I had no experience with them, yet).

Things I learned

About Feathers Resolvers

The Feathers Resolvers are the most important part of this release. We just need to show simpler ways of using them. The majority of the usage examples are based around generating a new app. What’s missing is the documentation showing how to implement them in an existing app. I’ll admit that this is mostly my fault. The solution is pretty straightforward: I’ll find places to document how to use them with existing APIs. We need a resolvers guide. The resolvers feel ambiguous, but once you start using them, there are very clear patterns of whether something works better in a resolver or in a hook. A guide written for existing users will clear up most of the unknowns.

The dispatch resolvers are finally the solid solution to data leakage. Using the protect hook, you had to be very careful when you populated data. It has been too easy to forget that populating one more level would potentially expose sensitive data. You had to be really careful to implement supportive patterns. We needed a solution that was impossible to get wrong, and I feel like we’ve achieved that with the resolveDispatch hook.

One more note about resolvers. They kind of ARE a separate entity because we’re going to end up with reusable resolver functions. These functions will be far more reusable than hooks, with much less code. The learning curve is actually VERY small. The bigger frustration will come from wanting to upgrade our existing codebases and having to juggle a hook library with a resolver library. I think I can help with this in the docs, as well. For huge apps, you might want to just carry on with what you have already been doing.

About Feathers Schemas

Feathers Schemas shine in three places for general users:

1. A standards-compliant, officially-supported solution. It comes baked into the generator, now, so you don’t have to go searching for solutions. If we would have had this years ago, I wouldn’t have created all of my tooling around Joi. Back when JSON Schema was missing conditional schema support, it was too limited for what we needed. Now it can support the huge majority of service APIs. Having a standard around which we can build community tooling is something we’ve needed for a while.

2. Offering a high degree of portability. It’s one thing to publish a package with all of your Joi validations and install it in multiple places. It’s far more portable to be able to pass schemas around in network requests. It’s not a benefit that most apps will need, but having a standards-compliant schema that can be easily passed from worker to worker is pretty amazing for developing data-safe, dynamic apps.

3. Existing as a machine-readable format. JSON Schemas can be used in many tools for generating API docs. I haven’t done much experimentation in this area, because I tend to write a lot of documentation by hand. But that’s exactly why I like the idea.

If your app doesn’t benefit from the above list, it’s less likely to be worth the switch. Feathers Schemas is 100% optional for most apps.

Benefits for the people maintaining Feathers.

We want to help people with their APIs, but there’s still some fragmentation due to ORM lock-in. Having an official solution replaces the scattered schemas/validations in most ORMs. We can have conversations about a single validation solution, instead of having several sub-communities: one for each ORM.

The rest of the benefits are generally “nice to haves”.

They’re more important for some than others. I know I appreciate them, but they have less sway when it comes to tech decisions.

• Automated types from your schemas. I love not having to create separate types.
• No more tight coupling with ORM validation. The ORM solutions are pretty clunky compared to a standalone validation solution. This benefit applies equally well to any standalone validation library.

About around hooks

I wasn’t a fan of around hooks until I converted most of my before and after hooks to resolver utilities. When you’re using regular hooks for everything, you need the granularity of before and after as opposed to one hook with access to both. When you start using the resolvers, you don’t need the granularity nearly as much. Hooks become useful only for side effects outside of manipulating a record or populating data. Things like caching and logging are great candidates to remain as hooks.

I admit that it took me over a year to like around hooks. @daffl and I had a lot of conversations about them and it took us a long time to figure out a way to integrate them in the most supportive way possible.

Things I’m realizing

Now that I’m actually a fanboi of all of the new features, I see the problems falling into the following categories:

1. We like the simplicity of what we’re accustomed to doing.

   It’s fair to be concerned about new APIs, especially for somebody who is in charge of teaching those APIs to a team of developers. I can see we need some more guide-centric material in the docs to help with this problem. This is also part of the reason that I chose to make the docs for Feathers-Pinia 100% guide-focused, with each guide containing a section for the relevant API docs. We won’t take that route with Feathers, but I now have some ideas for documentation improvements.

   Action Item: The solution to this issue can easily be to keep doing what you’ve been doing. It’s most likely that whatever you’re already doing in your apps will continue to work, which is an often-overlooked, huge benefit that comes from this next point:

2. There’s not a lot of room for improvement in the existing APIs. This is a testament to how good the Feathers core is.

   The hooks API is so simple. Once you get your head unstuck from the MVC patterns that dominate backend thinking, you fall in love with the cleanest implementation of aspect-oriented programming in API development (Accurate, at least, in any of the three languages I’ve used over the years: Ruby, PHP, Node). Our hook object, with its declarative arrays of hooks, makes it awesome to work with AOP: instead of having a bunch of middleware scattered around the application, it’s all in one place.

   I feel like @daffl and I found a good balance with the around hooks implementation. They “play nice” with the existing hooks, which was priority number 1 for me. So I say that there’s not a lot of improvement in the existing APIs. The hooks API is solid. The addition of around hooks gives us an optional, cleaner way of handling before/after scenarios. But the resolvers do a great job of building ON TOP OF the existing API.

   Action Item: As with the previous item, the action step here is that we can keep using what we’re already using. There are optional tools available on top of the existing API. Hooks are still the foundation.

3. TypeScript can make things fugly.

   I don’t think anybody in this current thread has this issue, but there are still a few others who do. It’s not an issue for those with TS experience, but for those with limited TS experience it can be daunting.

   With this release, we’ve gone all-in on TypeScript, and it adds a lot of extra code. @daffl has expertly crafted the internal types. He’s skilled with types in a way that strikes a balance. He made the internal Feathers types in a way that not many user-land types have to be added. A TypeScript app is always going to require more code. Generate a JS app with the new generator and look at how terse and clean it is. I miss those days, but the benefits of TypeScript are very apparent (as long as you don’t let it slow down productivity, which takes practice).

   Anyway, my opinion is that the TS code is definitely uglier, but better editor tooling with TS is a major win for junior devs. And they can usually stick to using basic types when implementing hooks, etc.

   Action Item: There’s not really an action item here. We’ll continue to do the best we can to keep types simple, focus on code, and only allow TS to improve our DX.

4. The generated app is a bit too verbose

   Building on the previous point, there’s too much code in the current pre-release of the generated app. With every new release of the generator, it takes some time to strike a balance between self-documenting and terse. The current one leans too much towards advanced use cases.

   Action item: @daffl and I figured out some great ways to drastically simplify it. It’s much more Feathers-y, now It’s less intimidating, easier to understand, and offers a nice path to more advanced use cases. We really appreciate the feedback that has led to the newest changes.

5. We’re failing to show the incremental upgrade path

   This failing lands mostly on me for not noticing earlier. I’m not really doing a good job of showing how to add resolvers to our existing apps. The current docs and generator make it look like you need to use around hooks to use resolvers, but that’s not accurate.

   Also, I’m not going a good job of relating to what we already know. We are all accustomed to using utility hooks. Resolvers are just another type of hook, but we talk about them like they’re a completely standalone entity. Resolvers are just a super-useful utility hook. I can see that this is more of a communication problem than anything else. Resolvers are powerful, and even junior-dev friendly.

   Action Items:

   • Docs for adding resolvers to an existing app.
   • A Resolver-specific migration guide to better illustrate what can be improved by a resolver.

[daffl]

Thank you everybody for the input! This discussion lead to some revisiting current v5 app patterns and solid refactoring and we tried to address many of the concerns mentioned here in https://github.com/feathersjs/feathers/pull/2772. More information can be found in the PR description but specifically working with @marshallswain and @fratzinger it includes the following:

• Reduced boilerplate for class and schema initialisation
• Hooks moved to the <name>.ts file where they are registered in a way that also makes them service type safe
• Resolver independent schema definitions
• Additional support for schema definitions using TypeBox which is more concise and declarative and similar to e.g. a Mongoose model definition
• Ability to generate a service without schemas

We are working on a detailed CLI guide that describes the patterns from the generator and how to do common things (like relationships, code formatting etc.).