Handle where clauses correctly for records where the field is `null` or `not defined` - MongoDB

Problem

All ID’s removed for abrreviation

model Test {
  /// mandatory
  id String @id @default(auto()) @map("_id") @db.ObjectId
  /// just to differentiate
  name String @unique 

  nullable Boolean? // <--- This
}

When you declare a model with a nullable field, let’s say Test (as described above), we can end up with 3 possible documents in our MongoDB databases.

// 1: `nullable` is optional but has a null value
PRISMA.test.create({ data: { name: 'with null', nullable: null } });
databaseRecord = { name: 'with null', nullable: null }

// 2: `nullable` is optional but specified
PRISMA.test.create({ data: { name: 'with true', nullable: true } });
databaseRecord = { name: 'with true', nullable: true }

// 3: `nullable` is optional and ignored
PRISMA.test.create({ data: { name: 'empty' } });
databaseRecord = { name: 'empty' }

And that should be the desired behaviour, as we can save bytes with these null optional fields.

But the problem occurs when we try to filter these documents. Simply by using where: { nullable: { not: null } } or where: { nullable: null }, the 3rd option gets filtered out.

console.log({
  nulls: await PRISMA.test.findMany({ where: { nullable: null }, }),
  notNulls: await PRISMA.test.findMany({ where: { nullable: { not: null } }, }),
  all: await PRISMA.test.findMany(),
});

result = {
  nulls: [{ name: 'with null', nullable: null }],
  notNulls: [{ name: 'with true', nullable: true }],
  all: [
    { name: 'with null', nullable: null },
    { name: 'with true', nullable: true },
    // The nullable field is present here, instead of just a { name: 'empty' } object,
    // which would cause obj.nullable to return undefined.
    { name: 'empty', nullable: null }, 
  ],
};

Suggested solution

As said before, I don’t think that use null values instead of not setting fields is a good way. Because we can save up a lot of bytes if we take in comparison a multi million record collection.

This issue maybe should have been created in the rust side of this project, but I’m not familiar enought with prisma arquitecture.

Anyways, I think that transforming the query to handle both options (not present & present and null) like this stackoverflow post does for the native mongodb javascript driver is good enough.

Alternatives

I don’t mind any other alternative, as long it works as intended 😃

Additional context

• MongoDB 5.0
• Prisma Client ^3.15.2
• Prisma ^3.15.2

MongoDB Compass Test collection screenshot:

[arthurfiorette]

@janpio https://github.com/arthurfiorette/prisma-issue-13949

Outputs:

Created entities:
{
  withNull: { id: '62b3762171bf26e40b14616a', name: 'with null', nullable: null },
  withTrue: { id: '62b3762171bf26e40b14616b', name: 'with true', nullable: true },
  empty: { id: '62b3762171bf26e40b14616c', name: 'empty', nullable: null }
}

Filtered records
{
  nulls: [
    {
      id: '62b3762171bf26e40b14616a',
      name: 'with null',
      nullable: null
    }
  ],
  notNulls: [
    {
      id: '62b3762171bf26e40b14616b',
      name: 'with true',
      nullable: true
    }
  ],
  all: [
    {
      id: '62b3762171bf26e40b14616a',
      name: 'with null',
      nullable: null
    },
    {
      id: '62b3762171bf26e40b14616b',
      name: 'with true',
      nullable: true
    },
    { id: '62b3762171bf26e40b14616c', name: 'empty', nullable: null }
  ]
}

[dpetrick]

This is intended behavior.

You can filter these fields with isSet: true/false: https://www.prisma.io/docs/reference/api-reference/prisma-client-reference#isset

The null/undefined result discussion can be tracked here: https://github.com/prisma/prisma/issues/12555