[FEATURE] We need better testing + communication

Is your feature request related to a problem? Please describe. Over the last months, I’ve experienced regressions in Framer Motion on an almost version by version basis. Minor versions frequently break existing implementations, while long standing issues remain unfixed. Since we decided to use Motion as a basis for our application, it has now become a major dependency to the point where if there’s a blocking issue, we cannot release an update to the app, while we are powerless to improve the situation. As you can probably imagine, this has created quite some anxiety on our part.

I don’t believe this aligns with your goals either, as Framer seems to want to position itself as a tool to kickstart both design and development of professional applications; I’m guessing this has also been the original motivation to release framer-motion as open source. This is why I think it is important to address this issue.

Describe the solution you’d like I would like to kickstart a discussion of what we can do to improve this situation. One way might be to further improve the test coverage/depth, but I’m not sure, you might have a better eye on it than I do from the outside.

I want to help out any way that is welcome, I’m doing my best right now to document issues but am not getting much response on that, if at all. So if this is not the way, please let me know if there’s a different way you’d like me/the community to contribute. I’d take the time to contribute to the tests/code, but obviously only if that is wanted and the work will not be tossed out. The lack of response to most of the issues I’ve posted so far (which often take a lot of effort to reproduce and document) makes me hesitant in that regard.

Lastly I want to emphasize that I love the library, I think it’s the best web animation framework out there. That is why I am invested enough in it to write this, I hope this yields a constructive dialog.

[mattgperry]

Sure, yeah I would ignore the second branch for now. Anything that is urgent you can carry on as you were like with fix/jump, I’ve probably missed some PRs but can take a look and publish a beta for you.

The projection system is largely similar in feature/perfect-projection but the crucial change is it’s been extracted as (essentially) a stand-alone, imperative library that can be tested in plain HTML files without React or animations. The situation as it was, was untenable, as you’ve probably experienced it’s hard to replicate issues and genuinely in terms of my sanity I couldn’t develop any further with it. So we’ve started afresh putting testing at the forefront.

Currently this the second week of a 6-8 week rewrite so it’s early now but there’s already some major improvements especially around page and element scroll, that are fully supported by tests.

[mattgperry]

Hi Rijk

I agree with the overall gist. I would also like better testing and communication. Both of these take time and I currently don’t have enough of that. That’s the underlying problem.

There’s essentially two products here, Framer and Framer Motion. Currently, no one is paying for Framer Motion. So, when you file a bug report, and I’m responsible for another report within Framer, yours is at the back of the queue, where it stays until there’s no more queue. I can assure you I’d love nothing more than to tend to my little garden and close out all the bugs but ultimately this isn’t the product that is paying my bills.

What you’ve been experiencing over the past couple months, is we’ve been pushing where and how we use layout animations within Framer. As a result there’s been movement within this code and where we’ve been fixing things in Framer we might end up breaking them in your code.

The good news is that despite appearances, testing has never been better. Because of the complexities involved in testing it, until a couple of months ago layout animations have been the one missing spot in our test coverage. Right now I just ran the test suite and it took five minutes to run all the new tests that have been added recently. So although I might have broken something in your implementation, we’re increasing this testing suite and as we address more bugs and edge cases we should naturally create a more stable implementation. But we don’t have a test for every single thing right now so it’ll take time and more code movement.

I don’t agree that you are powerless in this situation. If you are getting as much value from this as you say then you could go in and fix these problems yourself, adding a test to ensure they don’t happen again. It might be that I don’t have time to come in and investigate every issue right now but bug fix PRs get merged quite quickly.