Automatic conversion from Int to ID is problematic

Graphene-SQLAlchemy automatically converts columns of type SmallInteger or Integer to ID! fields if they are primary keys, but does not convert such columns to ID fields if they are foreign keys.

Take for example this schema:

class Department(Base):
    __tablename__ = 'department'
    id = Column(Integer, primary_key=True)
    name = Column(String)

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    name = Column(String)
    department_id = Column(Integer, ForeignKey('department.id'))
    department = relationship(Department)

class DepartmentType(SQLAlchemyObjectType):
    class Meta:
        model = Department

class UserType(SQLAlchemyObjectType):
    class Meta:
        model = User

class Query(ObjectType):

    departments = List(DepartmentType)
    users = List(UserType)

    def resolve_departments(self, info):
        return DepartmentType.get_query(info)

    def resolve_users(self, info):
        return UserType.get_query(info)

You can run the following query:

query {
  users {
    id
    name
    departmentId
    department {
      id
    }
  }
}

As a result, you get something like:

{
  "data": {
    "users": [
      {
        "id": "1",
        "firstName": "Fred",
        "departmentId": 1,
        "department": {
          "id": "1"
        }
      },
      {
        "id": "2",
        "firstName": "Barnie",
        "departmentId": 2,
        "department": {
          "id": "2"
        }
      }
    ]
  }
}

As you see, department.id is a string (because IDs are returned as strings), while departmentId is a number. This turned out to be a huge problem and source of error in practice. Working with this inconsistent, fault-prone interface has bitten me many times. When storing ids in objects on the frontend, or using ids as filters, I never know whether I should use numbers or strings. Currently I have conversions from number to string and vice versa everywhere in my frontend code, and if I don’t do it correctly, things stop working in hard to debug ways because you often don’t recognize such type mismatches. On the server side, do I take ids used as filter parameters as IDs or Ints? If I do the former, I must then convert them to integer when using them as filter arguments for SQLAlchemy. So, really, this is no fun to work with and doesn’t work in practice, because you always have this mental burden of thinking about whether your ids should be represented as strings or numbers and whether you need to convert them when passing them around.

I suggest the conversions should be consistent. Either convert all keys, including foreign keys, to IDs, or do not make a special case conversion for primary keys. Actually I’d prefer the latter, since then I never need to think about the type and since storing numbers on the frontend uses less memory.

Now of course I know that there is the relay specification which assumes there is an id field with a type of ID. So when using the relay interface, things are different. In this case, I suggest converting to IDs everywhere (including foreign keys) - but here we need conversion of the values to global ids anyway, they are not just the row ids converted to strings.

[taffit]

I’m sharing the opinion of @Cito and @ddrouin: The columns from the database should be left untouched. I named my primary keys id in the database and was completely irritated, that all of a sudden they are some encoded strings. Also, when I get back an entity with id='aas23af32asd', and I want to look that one up in the database, how can I get the real, original id of the entry?
I find the approach cited by @Cito much more appealing: If you want/need to introduce another property/field of type ID!, do it with your own name, without overwriting existing columns.

Just my two cents. As a beginner with graphene, I was lost initially.

[jnak]

Hi everyone,

I’m interesting in pushing this over the finish line. We all agree here that the conversion should be consistent. The only thing left to decide is wether we want to convert all keys to IDs or completely opt out of this behavior.

I’m personally in favor of converting all keys to IDs because it is more correct from a type perspective. The ‘ID’ type means that clients are not supposed to operate on in any way. For example, the generated flow or typescript types should rightfully prevent users to concatenate two IDs, even though they are stored as strings in the DB.

Additionally, I don’t think that we should not use IDs because they potentially save some memory on the frontend. This is a micro-optimization that I doubt will have any meaningful impact unless you fetch many tens of thousands of GraphQL objects. If you do happen to be in situation like this, you will probably run into bigger performance bottlenecks than converting strings to ids.

Regarding the nodeId of graphile, I’m open to the idea but we should definitely make that configurable since that it does not follow the Relay spec.

@Nabellaleen Any thoughts on that? @Cito Do you guys feel strongly the other way?

Cheers, J