Allow running arbitrary async code in a transaction?

Hi! Thanks for creating this library! It has been very nice to use.

At work we have this use case:

1. Start transaction
2. UPDATE scheduler.schedule SET sent_date = @now WHERE id = @id
3. Publish a message to RabbitMQ (quick async operation that can fail)
4. Commit transaction, or rollback if publishing to RabbitMQ failed

We couldn’t find a way to do this, so we rolled our own. Since we’re not super experienced F#/dotnet developers, we simply copied executeTransactionAsync and modified it slightly to do what we needed. Here’s the result:

type QueryOrAsync =
    | SqlQuery of string * List<List<string * SqlValue>>
    | CustomAsync of Async<Result<unit, string>>

// Based on: https://github.com/Zaid-Ajaj/Npgsql.FSharp/blob/dad861b4f2e5a27300747854052bd033a2438bf3/src/Sql.fs#L554-L589
// Changed `(props: SqlProps)` to `(connectionString: string)` as a workaround for `SqlProps` being kinda private.
let executeTransactionAsyncWithCustomAsyncs (queries: List<QueryOrAsync>) (connectionString: string) =
    async {
        try
            let! token = Async.CancellationToken
            // Changed `props.CancellationToken` to `CancellationToken.None` because private.
            use mergedTokenSource = CancellationTokenSource.CreateLinkedTokenSource(token, CancellationToken.None)
            let mergedToken = mergedTokenSource.Token
            if List.isEmpty queries then
                return Ok []
            else
                // Changed from `getConnection props` because private.
                let connection = new NpgsqlConnection(connectionString)
                try
                    // Removed `if x then connection.Open()` because we got "Connection already open" errors otherwise.
                    // Doesn’t make sense to first open synchronously and then asynchronously on the next line?
                    do! Async.AwaitTask(connection.OpenAsync mergedToken)
                    use transaction = connection.BeginTransaction()
                    let affectedRowsByQuery = ResizeArray<int>()
                    for queryOrAsync in queries do
                        match queryOrAsync with
                        | SqlQuery(query, parameterSets) ->
                            if List.isEmpty parameterSets then
                                use command = new NpgsqlCommand(query, connection, transaction)
                                let! affectedRows = Async.AwaitTask(command.ExecuteNonQueryAsync mergedToken)
                                affectedRowsByQuery.Add affectedRows
                            else
                                for parameterSet in parameterSets do
                                    use command = new NpgsqlCommand(query, connection, transaction)
                                    // We had to copy the entire `populateRow` function into our code
                                    // since `poulateRow` is private...
                                    populateRow command parameterSet
                                    let! affectedRows = Async.AwaitTask(command.ExecuteNonQueryAsync mergedToken)
                                    affectedRowsByQuery.Add affectedRows
                        | CustomAsync asyncResult ->
                            // Please excuse our ugly async code.
                            let theAsync =
                                async.Bind
                                    (asyncResult,
                                     function
                                     | Ok() -> async.Return()
                                     | Error message -> raise (System.Exception(message)))
                            do! theAsync
                    do! Async.AwaitTask(transaction.CommitAsync mergedToken)
                    return Ok(List.ofSeq affectedRowsByQuery)
                finally
                    connection.Dispose()
        with error -> return Error error
    }

The above allows us to do the following:

sendEvent now publishScheduledEvent connectionString eventToTrigger =
    connectionString
    |> executeTransactionAsyncWithCustomAsyncs
        [ SqlQuery
            ("UPDATE scheduler.schedule SET sent_date = @now WHERE id = @id",
                [ [ ("now", Sql.timestamptz now)
                    ("id", Sql.int64 eventToTrigger.id) ] ])
            CustomAsync(publishScheduledEvent eventToTrigger.payload) ]

We’ve tried it and it works: If publishScheduledEvent fails, the scheduler.schedule table isn’t updated.

Would you be interested in adding something like this to your library? You might be able to come up with a much nicer API than we did (we didn’t spend much time on it). As long as we can fulfil our use case we’d be happy!

[Zaid-Ajaj]

Alright, so it took awhile for me to come up with a serious API to tackle this problem that I am actually happy with. Although I liked the continuation style by @jukkhop, it meant that the library has to have specific transaction command and non-transaction commands.

Instead, I came up with the following API as of v3.12 (please see new docs) which allows to run the SQL commands or any arbitrary code in the context of a transaction. You basically create the transaction yourself and re-use it across commands as follows:

open Ngpsql
open Npgsql.FSharp

let connectionString = " . . . "

// 1) Create the connection
use connection = new NpgsqlConnection(connectionString)
connection.Open()

// 2) Create the transaction from the connection
use transaction = connection.BeginTransaction()

// 3) run SQL commands against the transaction
for number in [1 .. 10] do
    let result =
        Sql.transaction transaction
        |> Sql.query "INSERT INTO table (columnName) VALUES (@value)"
        |> Sql.parameters [ "@value", Sql.int 42 ]
        |> Sql.executeNonQuery

    printfn "%A" result

// 4) commit the transaction, rollback or do whatever you want
transaction.Commit()

The magic happens when you provide the transaction via the Sql.transaction function.

If you don’t like creating the connection yourself because you want to use the builder API, you can instead let the library create the connection as follows:

use connection =
    connectionString
    |> Sql.connect
    |> Sql.createConnection

connection.Open()

Please give this a try and let me know if you encounter any problem

[lydell]

This is on my list of stuff to try out – I’ll let you know once I get to it! 👍