database: Migrate JSON fields to ArrayField or other native types.

This issue requires expertise with Django and postgres and the ability to make some executive decisions without too much guidance. It’s not a great issue for beginners, although first-time contributors with strong Django backgrounds may do well here.

We have some things in our database that are serialized as JSON to a database field, rather than introducing a whole new database table. We are generally fine with this approach, but we can use more modern Django features like ArrayField to support certain types of data. Here is one example of JSON in our code:

1117     # Contains serialized JSON of the form:
1118     #    [("step 1", true), ("step 2", false)]
1119     # where the second element of each tuple is if the step has been
1120     # completed.
1121     onboarding_steps: str = models.TextField(default='[]')

If you search for JSON in zerver/models.py you will find candidates for fields that we possibly want to migrate to ArrayField. Obviously, part of the work here will be to create backfill migrations. And then, of course, we want to make sure that we take advantage of the Django features and ensure that tests still pass.

See https://docs.djangoproject.com/en/3.1/ref/contrib/postgres/fields/ for more info.

It’s possible that certain fields won’t map nicely to ArrayField (including possibly the example above), and we’ll want to keep them in JSON. As part of your audit, you can update this issue with your findings. In some cases you can also do small commits in a PR to clarify comments.

Also, see https://chat.zulip.org/#narrow/stream/2-general/topic/Mute.20user.20.2316915/near/1113151

[ligmitz]

@edith007 Not now. You can go ahead and work on this if you want.

[asah]

PostgreSQL guy here (but zulip newb): is this for performance? or some other reason? I’m asking because the world is migrating to more use of JSON fields and therefore better support for them…

(y, I can maybe help…)