TypeScript plugin or runtime tracing to allow query composition?

https://github.com/xialvjun/ts-sql-plugin is an interesting project that uses a TS plugin to typecheck sql query strings. https://github.com/mmkal/slonik-tools/tree/master/packages/typegen#readme is also an interesting project that uses runtime tracing to detect query types on first run.

Both should allow you to do something roughly like this:

const notDeleted = (asOf = null) => sql`
  deleted_at is null or deleted_at > ${asOf || sql`now()`}
`

const myQuery = (bar) => sql`
  select * from mytable
  where ${notDeleted()}
  and foo = ${bar}
`

// result is fully typed
const result = await db.query(myQuery(bar))

Is pgTyped interested in experimenting with such techniques to enable query composition, for convenient sharing of common conditions etc?

[maxpain]

@adelsz what do you think about this concept?

We could generate one pgtyped.d.ts for the entire project with the generic function query and mappings, where all our queries are typescript literals and mapped to corresponding params and results.

Example:

declare module '@pgtyped/query' {
  export type Mappings = {
    '\n			SELECT cluster_id\n			FROM cluster_subnets\n			WHERE $clientIP <<= subnet\n		': {
      params: {
        clientIP: string | null | void
      }
      result: {
        cluster_id: number
      }
    }
  }

  export function query<T extends keyof Mappings>(
    ...args: Mappings[T]['params'] extends void
      ? [T]
      : [T, Mappings[T]['params']]
  ): Promise<Mappings[T]['result'][]>
}

And use like this:

import query from '@pgtyped/query' 

// both result and params are strictly typed
const [subnet] = await query(`
  SELECT cluster_id
  FROM cluster_subnets
  WHERE $clientIP <<= subnet
`, { clientIP })

query function implementation:

import pg from 'postgres' // for example
import { parseTSQuery } from './loader/typescript'
import { processTSQueryAST } from './preprocessor-ts'

export default function query(...args) {
    const [query, params] = args

    // Actually, we should cache queryAst.
    const { query: queryAst } = parseTSQuery(query)

    const { query: processedQuery, bindings } = processTSQueryAST(
        queryAst,
        params
    )

    return pg.unsafe(processedQuery, bindings)
}

We also would make a type-safe generic es6 template tag (even with strictly typed params), but there is a problem in typescript itself

[maxpain]

In an ideal world, we would use TypeScript Type Providers, but this issue is unresolved since 2015 and we need something useful right now.

At the moment using pgtyped on big projects is pretty painful.

For example, at my projects, I have tons of imports at the beginning of the files.

Moreover, we have to place all our queries at the beginning of the files to speed-up performance

Yes, we tried to use SQL files and use imports like this:

import * as queries from './src/books/queries';

await queries.findBookById.run(
    {
      bookId: 42,
    },
    client,
  );

But it is very uncomfortable to scroll a file (or open a separate .sql file) every time you create a new query. Moreover, you have to think about unique names of new SQL queries and when using separate SQL files you have to write all the annotations stuff:

/*
  @name selectSomeUsers
  @param ages -> (...)
  @param names -> (...)
*/
SELECT FROM users WHERE age in :ages or name in :names;

Someone may get benefits of reusing the same SQL queries, but in practice in different places you use different columns, joins, WHERE conditions, etc.

Also, we like to save context and write SQL queries in the same place we call/use them.