Roadmap to a v3.0.0 of flow-typed

Roadmap to a v3.0.0 of flow-typed

I recently took the time to read through the opened issues and as others have already mentioned I also do think that there is room for improvement in how flow-typed works. A few opened issues already address various topics which, in my eyes, would lead to breaking changes and therefore to a new major release (following the semantic release versioning) and currently hinder the flow community as a whole to progress to a matured state (in my eyes).

As far as I can see there are no specific plans, at least on GitHub issues, which are dedicated to a release plan or how flow-typed will progress in the longer run. This is why I created this issue.

I would like and could contribute a few weeks of my freelance time to this project in the upcoming three months, but before starting and risking to throw away time and code, it would make sense to agree on a certain set of problems that we want to solve and maybe even to agree on a schedule for this (major) release.

Serving type definitions via npm

While taking a look through the opened issues, I faced that most of them are in regards or at least related to the current implementation of having all definitions inside the repository of flow-typed itself, e.g.

1. Publish libraries as npm packages? #990
2. Committing libdef files to source control #413
3. [Discussion] What can we learn from DefinitelyTyped? #395
4. … and in-directly Dependencies between library definitions? #16

Both issues stopped at one certain problem, the version pinning between …

• … the type definition itself since it could contain bugfixes.
• … the library of the type definition.
• … the verion of flow that the type definition supports.

To solve this, I would propose to adapt the proposal of @EdwardDrapkin in his comment on #990.

How do we handle the versioning of the typing and the associated library?

First and foremost we should use existing functionalities as much as possible, e.g. the peerDependencies feature of npm/yarn. This way we can specify supported version(-range)s with almost no effort! Secondly I would propose that type definitions will follow a strict semver versioning out of the box.

Let’s use an imaginary typing package, e.g. @flowtyped/axios, by the time of writing this the axios library is currently published as version 0.17.0. The first two numbers, namely the major and minor version, should always match the version of the targeted library version that the typing supports…

What about bug fixes to the typing?

… the patch version whatsoever can be used independently to release bugfixes of the typing itself, so a case such as the following will be completely valid since the API of axios is still the same and only a bugfix of the typing was published.

	MAJOR	MINOR	PATCH
Version of the library	0	17	1
Version of the typing	0	17	12

Another beneficial result of this would be that users would get automatic updates/fixes in case they prepend the caret (^) to their typing versions within the package.json.

What about the supported flow version of the typing?

Again I would propose to use the peerDependency feature to specify a supported semver range of flow-bin.

But of course this would not allow serving types for multiple versions of flow in the same typing release. To allow this, I would follow @thejameskyle propose in #990 to introduce a mapping within the package.json to do this with a few adaptions, e.g.

{
    "name": "@flowtyped/axios",
    "flow-typed": {
        "types": {
            "byFlowVersion": {
                "v0.28.x-v0.30.x": "flow_v0.28.x-v0.30.x.js",
                "v0.30.x-v0.38.x": "flow_v0.30.x-v0.38.x.js",
                "latest": "flow_latest.js"
            }
        }
    }
}

You see that the property itself is named flow-typed and not flow, I think this is a bit safer since otherwise it could conflict in the future if flow itself would be configured in the package.json. I would also add a more generic latest key which should always be present, more on this later on.

What about packages that do not follow strict semver versioning?

Introducing a boolean property in the package.json configuration of the typing could solve this, e.g.

{
    "name": "@flowtyped/axios",
    "flow-typed": {
        "types": {
            "isIndependentlyVersioned": true,
            "byFlowVersion": {
                "v0.28.x-v0.30.x": "flow_v0.28.x-v0.30.x.js",
                "v0.30.x-v0.38.x": "flow_v0.30.x-v0.38.x.js",
                "latest": "flow_latest.js"
            }
        }
    }
}

---

Changes to the CLI

Installing new typings via flow-typed install

Since typings would be served via npm, the flow-typed install command would need to be reworked from the ground up. Step-by-step it would …

• … parse the projects package.json for dependencies as it does currently.
• … filter out dependencies for which types where already installed.
• … query the NPM registry via the HTTP API for matching types packages.
• … prompt the developer if he would like to install the package into the devDependencies.
• … execute a npm install or yarn add in the background. (This is also something that we would need to make configurable)

But how should flow resolve the types within node_modules automatically? I think there are two options, either each typing package has it’s own postinstalland preuninstall script which would copy the files into the ${process.cwd()}/flow-typed/npm folder or we solve this issue in a central place with a new CLI command…

Symlinking/copying the installed types into flow-typed/npm with flow-typed create-symlinks

This command would …

• … do a grep through all package.json files in the node_modules folder of the project for package.json files with a “flow-typed” section defined.
• … read the pkg["flow-typed"].types configuration.
• … resolves the typing that matches the projects flow-bin version.
• … if no match was found, it prompts the user to use the pkg["flow-typed"].types.latest version with a warning.
• … if the user wants to use the latest, a flow-typed.lock file will be created with information for the next run (especially to avoid prompts on CI).
• … and symlinks the file into the projects flow-typed/npm folder.

This way, the following issues would also be solved since the create-symlinks will also respect flow-typed configurations in library packages and not only typings in the @flowtype scope:

1. [Ecosystem] Vendoring in libraries? #286
2. What is the preferred way for a module to provide its own libdef? #1451

So you see the flow-typed install command would only be executed upon installing new packages and the flow-typed create-symlinks command would be run upon setting up a project or during CI / CD.

---

Migrating the old type definitions into separate repositories

Of course this would totally conflict with the current setup, hence a new major release would need to be published, but I think it’s not a huge task to create a migration script which will automatically transform the existing type definitions into their own repositories while keeping their git histories using the git filter-branch command. Not huge because the existing definitions follow a strict convention in regards to their structure and tests.

The only question I have is, if the definitions should be moved into separate git repositories under the flow-typed organization or if we create a new mono-repository namely flow-typed-definitions in which all definitions could be placed.

I wouldn’t like to keep the current structure where the CLI is in the same repository as the definitions since this causes way more noise and it makes separating CLI and definition pull requests harder, so I guess a separate mono-repository or even single repositories for each definition would make more sense.

Moving all definitions into a mono-repository

• PRO: Easier to contribute new definitions to the @flowtyped npm scope and GitHub organization.
• PRO: Easier to keep the tooling in sync

Moving definitions into single responsibility repositories

• PRO: Easier to integrate automatic release management via semantic-release and commitizen
• PRO: Easier to manage issues
• PRO: Tests / CI would run extremely fast (Running def tests is unnecessarily slow #414)
• PRO: Definition authors could be given PR merge rights on their definition repositories which would make review processes a bit faster
• PRO: Additional tooling / tests could be added more easily [tests] Idea: Use a runtime test library for tests #100

IMO moving them into single responsibility repositories would be more beneficial, but I really don’t know how new definitions would be handled, initially authors could just create their own definition repositories on their own and transfer them into the @flowtype after a review process or something along the lines, but that is definitely out of my scope.

The tooling won’t be a problem as far as I can see since it could be abstracted into it’s separate CLI, so up to our next point! 😃

---

Creating a flow-typed-definition CLI

So in the process of creating the migration script for definitions, most of the logic could be easily abstracted into a flow-typed-definition CLI, similar to create-react-app it could be a central point to create new definition repositories with ease and therefore solve [CONTRIB] Make it easier to work on libdefs #146.

This CLI could also hold commands for definition repository maintainers, e.g. the test script that we currently hold within flow-typed itself (run_def_tests.sh) and more…

---

Fin.

All of these proposals on how things could work should be seen as a basis for discussions, not as carved in stone and perfectly thought out.

I did not intend to undermine the already existing issues, but to have a central place for discussions and to take a look at them from a greater perspective. While I know that this is a pretty big proposal / change, I do think that the flow community would really appreciate it if we would solve these bigger pain-points/issues sooner than later, which of course should not mean that we should push everything out in a quick-shot move.

Let’s take the time to discuss everything and try to find a format for a good flow-typed configuration within the project / within type packages that will be extendable in the longer run! 😃 ❤️

/cc @DullReferenceException @TrySound @marudor @jeffmo @thejameskyle @EdwardDrapkin @bengotow @gtrufitt @lydell @samwgoldman @ryyppy @zertosh @callumlocke @GAntoine @doberkofler

[EdwardDrapkin]

If it’s not too late, I want to make sure that being able for libdefs to declare dependencies on other libdefs isn’t lost in the v3 work; we’ve got a bunch of express packages’ libdefs that I can’t PR because they require definitions like express$Request to be useful, and there’s no way for me to make the tests pass as-is. I’m working on getting as many of our Flow libdefs merged as possible, but I fear that if I PR a bunch of express middleware that’s bailing on callback types, they’ll be less than useful, let alone how useless a more tightly coupled ecosystem (feathers, specifically in my case) would be if expressed as Function and Object.

[mruzekw]

I like the direction v3 is headed.

+1 for a monorepo with lerna or yarn workspaces

I’m concerned though about keeping up the quality of libdefs (especially published via a possibly private npm package). Could we still run tests against different versions of flow?