Add option to read/output to memory from rw generate (instead of writing files directly to disk)

• Current behavior: When runningrw generate..., the CLI reads and writes directly to disk
• This makes it hard to integrate the generators with IDE environments. For example, if the user has open/dirty/unsaved files, having the CLI writing directly to disk can lead to conflicts (the dreaded: “this file was modified on disk, do you want to overwrite?” scenario).
• Proposed behavior: Add an option (ex: --json or --dry-run) that tells the generator to output a JSON map representing the files that were modified/generated, instead of applying those changes directly.
• This would enable seamless integration with IDEs (taking into account unsaved files, undo/redo, etc) and it can open the door for post-processing generated files, for example

Example

$ yarn rw g page About /about --json

will print:

{
  "/foo/web/src/pages/About/About.js": "import ...",
  "/foo/web/src/Routes.js": "..."
}

JSON Format

The JSON format could be very simple:

• Added files are represented by entries with content
• Modified files are represented by entries with the new content (if IDEs/tools want to be smart about calculating diffs, it is up to them)
• Deleted files are represented by entries with “null” value.

{
  "/foo/new-file.js": "content.",
  "/foo/modified-file.js": "new content",
  "/foo/deleted-file.js": null,
}

Passing File Overrides / Reading from Memory

• In order to fully eliminate the “this file was modified on disk, do you want to overwrite?” error, a second feature must be implemented: Generators must accept a set of file overrides.
• This is necessary since some generators operate on existing files (they read a file from disk, run a regular expression or some transformation, and then write the new file). If Routes.js is open in an IDE and unsaved (with changes), for example, running rw g page will result in a conflict.
• The solution is to enable passing “file overrides” when running generators.

Example

$ rw g page About /about --json --override '{"foo/web/src/Routes.js": "..."}'

[aldonline]

Just wanted to link this forum topic reply from Tom where he said he wasn’t a fan of json on the command line https://community.redwoodjs.com/t/scaffold-textarea-s/800/11?u=tobbe

The hard part is intercepting the filesystem calls. The rest is just the API to surface this feature. I’m just trying to provide a starting point that lets us get this done quickly. A lower level API to generators is even better, but it will take longer.

In the end, the point where the IDE will call this is roughly here. So it can be via importing an API, or an exec call.

[aldonline]

Clearly you’re the IDE expert here, and I have no idea how the integration would work, but wouldn’t it be easier/better if the generators also had an API, and were importable as modules?

Yes absolutely. Having a programmatic API for generators would open even more doors. But this one is necessary, and should be easy to implement.

Modified files are represented by entries with the new content

How would you indicate where in the file this new content should go?

I was suggesting the easiest way: Adding the “complete” content to the map. Always the complete content. No need to worry about diffs, insertion points, edits, etc. A new file and an edited file look the same in the map. The only way to know which is which is to compare against the existing working tree.

The solution is to enable passing “file overrides” when running generators.

What would this do exactly? Revert all unsaved changes, and then apply the changes from the generator?

The simplest use case:

• You open Routes.js
• Make a small edit (for example, add a comment) and DON’T save
• Run rw g page...
• Today, this will result in a conflict. The generated file will have a line added by the generator, but it won’t have your unsaved change. This is because the generator logic did a readFileSync(“…Routes.js”) somewhere. So it doesn’t know about your change.
• With the “overrides” option, the IDE would be able to pass the “unsaved” content of Routes.js (and any other open files)