Skip to content

Commit 081f3e2

Browse files
committed
OPTIMIZATION #1: Direct PyUnicode_DecodeUTF16 for NVARCHAR conversion (Linux/macOS)
Problem: - Linux/macOS performed double conversion for NVARCHAR columns - SQLWCHAR → std::wstring (via SQLWCHARToWString) → Python unicode - Created unnecessary intermediate std::wstring allocation Solution: - Use PyUnicode_DecodeUTF16() to convert UTF-16 directly to Python unicode - Single-step conversion eliminates intermediate allocation - Platform-specific optimization (Linux/macOS only) Impact: - Reduces memory allocations for wide-character string columns - Eliminates one full conversion step per NVARCHAR cell - Regular VARCHAR/CHAR columns unchanged (already optimal)
1 parent 8ed531e commit 081f3e2

File tree

1 file changed

+13
-2
lines changed

1 file changed

+13
-2
lines changed

mssql_python/pybind/ddbc_bindings.cpp

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3294,8 +3294,19 @@ SQLRETURN FetchBatchData(SQLHSTMT hStmt, ColumnBuffers& buffers, py::list& colum
32943294
if (!isLob && numCharsInData < fetchBufferSize) {
32953295
#if defined(__APPLE__) || defined(__linux__)
32963296
SQLWCHAR* wcharData = &buffers.wcharBuffers[col - 1][i * fetchBufferSize];
3297-
std::wstring wstr = SQLWCHARToWString(wcharData, numCharsInData);
3298-
row[col - 1] = wstr;
3297+
// OPTIMIZATION #1: Direct UTF-16 decode - eliminates intermediate std::wstring
3298+
PyObject* pyStr = PyUnicode_DecodeUTF16(
3299+
reinterpret_cast<const char*>(wcharData),
3300+
numCharsInData * sizeof(SQLWCHAR),
3301+
NULL, // errors - use default handling
3302+
NULL // byteorder - auto-detect
3303+
);
3304+
if (pyStr) {
3305+
row[col - 1] = py::reinterpret_steal<py::object>(pyStr);
3306+
} else {
3307+
PyErr_Clear();
3308+
row[col - 1] = std::wstring(L"");
3309+
}
32993310
#else
33003311
row[col - 1] = std::wstring(
33013312
reinterpret_cast<wchar_t*>(&buffers.wcharBuffers[col - 1][i * fetchBufferSize]),

0 commit comments

Comments
 (0)