[BUG] Memory Leak when querying SQL Anywhere 17 with node 64-bit

Describe your system

• odbc Package Version: 2.3.5
• ODBC Driver: SQL Anywhere 17
• Database Name: SQL Anywhere Network Server Version
• Database Version: 17.0.9.4913
• Node.js Version: 12.16.3 (64-bit)
• Node.js OS: Windows 10 Pro

Describe the bug When querying our SQL Anywhere 17 database with node and node-odbc 64-bit we experience an extremely high memory consumption, whereas the same constelation, but with node and node-odbc 32-bit causes a lot of encoding issues. The query takes a long time while memory usage goes up to 99% (60 GB and more on our system with 64 GB memory). We are able to reproduce this issue on different machines and with various database tables with the following code snippet:

const odbc = require('odbc');

async function test() {
  try {
    const connectionConfig = {
      connectionString: 'Driver=SQL Anywhere 17;Server=TEST;Host=localhost:1234;UID=abc;PWD=def',
      connectionTimeout: 10,
      loginTimeout: 10,
    };
    const connection = await odbc.connect(connectionConfig);
    const sql =`SELECT * FROM dummy`;

    const result = await connection.query(sql);

    console.log(`result: ${result}`);
  } catch (error) {
    console.error(error);
  }
}

test();

Here is the debug output:

ODBCConnection::Init
ODBCStatement::Init
[SQLHENV: 000001D47DD37F40] ODBC::Connect()
[SQLHENV: 000001D47DD37F40] ODBC::ConnectAsyncWorker::Execute()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBC::ConnectAsyncWorker::Execute(): Calling SQLGetInfo(ConnectionHandle = 000001D47DD38BA0, InfoType = 30 (SQL_MAX_COLUMN_NAME_LEN), InfoValuePtr = 000001D47F854770, BufferLength = 2, StringLengthPtr = 0)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBC::ConnectAsyncWorker::Execute(): SQLGetInfo succeeded: SQLRETURN = 0, InfoValue = 128
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBC::ConnectAsyncWorker::Execute(): Calling SQLGetInfo(ConnectionHandle = 000001D47DD37F40, InfoType = 72 (SQL_TXN_ISOLATION_OPTION), InfoValuePtr = 000001D47F854774, BufferLength = 4, StringLengthPtr = 0)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBC::ConnectAsyncWorker::Execute(): SQLGetInfo succeeded: SQLRETURN = 0, InfoValue = 15
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBC::ConnectAsyncWorker::OnOk()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBCConnection::Query()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBCConnection::QueryAsyncWorker::Execute(): Running SQL 'SELECT * FROM dummy;'
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBCConnection::QueryAsyncWorker::Execute(): Running SQLAllocHandle(HandleType = SQL_HANDLE_STMT, InputHandle = 000001D47DD38BA0, OutputHandlePtr = 000001D47F85AED0)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::QueryAsyncWorker::Execute(): Calling SQLExecDirect(Statment Handle = 000001D47DD3AD70, StatementText = SELECT * FROM dummy;, TextLength = SQL_NTS) 
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::QueryAsyncWorker::Execute(): SQLExecDirect passed with SQLRETURN = 0
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::RetrieveResultSet()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::RetrieveResultSet(): Running SQLRowCount(StatementHandle = 000001D47DD3AD70, RowCount = 0)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::RetrieveResultSet(): SQLRowCount passed: SQLRETURN = 0, RowCount = 10
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns(): Running SQLNumResultCols(StatementHandle = 000001D47DD3AD70, ColumnCount = 0
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns(): SQLNumResultCols passed: ColumnCount = 1
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns(): Running SQLDescribeCol(StatementHandle = 000001D47DD3AD70, ColumnNumber = 1, ColumnName = , BufferLength = 128, NameLength = 0, DataType = 0, ColumnSize = 0, DecimalDigits = 0, Nullable = 0)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns(): SQLDescribeCol passed: ColumnName = dummy, NameLength = 5, DataType = -10, ColumnSize = 2147483647, DecimalDigits = 0, Nullable = 0
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns(): Running SQLBindCol(StatementHandle = 000001D47DD3AD70, ColumnNumber = 1, TargetType = -8, TargetValuePtr = 000001D52880B040, BufferLength = 0, StrLen_or_Ind = 0
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::BindColumns(): SQLBindCol succeeded: StrLeng_or_IndPtr = 0
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::FetchAll()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::FetchAll(): Running SQLFetch(StatementHandle = 000001D47DD3AD70) (Running multiple times)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::FetchAll(): SQLFetch succeeded: Stored 10 rows of data, each with 1 columns
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::FetchAll(): Running SQLCloseCursor(StatementHandle = 000001D47DD3AD70) (Running multiple times)
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0][SQLHSTMT: 000001D47DD3AD70] ODBCConnection::FetchAll(): SQLCloseCursor succeeded
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0]ODBCConnection::QueryAsyncWorker::OnOk()
[SQLHENV: 000001D47DD37F40][SQLHDBC: 000001D47DD38BA0] ODBCConnection::~ODBCConnection

I was able to narrow it down to queries of fields with data type nvarchar (with different sizes). This issue might be related to this one: https://github.com/markdirish/node-odbc/issues/29

[dabo248]

Looks like column’s buffer size is wrongly calculated in odbc_connection.cpp, causing the high memory usage.

      case SQL_WCHAR:
      case SQL_WVARCHAR:
      case SQL_WLONGVARCHAR:
        column->buffer_size = (column->ColumnSize + 1) * sizeof(SQLWCHAR);
        column->bind_type = SQL_C_WCHAR;
        data->boundRow[i] = new SQLWCHAR[column->buffer_size]();
        break;

ColumnSize is 2147483647, causing the allocation to fail. We have implemented a workaround with a fixed buffer size of 1024, which is fine for us (https://github.com/BusyBytes/node-odbc/commit/d8d86aafef740519cc214ec93578ad40c220b0f4).

[markdirish]

2.4.0 landed a while ago, and there were lots of memory leak fixes in that version. I have also published 2.4.1, which had additional memory leak fixes. Going to close this issue, but feel free to open a new issue with any additional memory leaks!