Skip to content

Commit 7159d81

Browse files
committed
docs: Update OPTIMIZATION_PR_SUMMARY with OPT #2 details
1 parent 94b8a69 commit 7159d81

File tree

1 file changed

+58
-2
lines changed

1 file changed

+58
-2
lines changed

OPTIMIZATION_PR_SUMMARY.md

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,64 @@ if (pyStr) {
4949

5050
---
5151

52-
## 🔜 OPTIMIZATION #2: Direct Python C API for Numeric Types
53-
*Coming next...*
52+
## ✅ OPTIMIZATION #2: Direct Python C API for Numeric Types
53+
54+
**Commit:** 94b8a69
55+
56+
### Problem
57+
All numeric type conversions went through pybind11 wrappers, which add unnecessary overhead:
58+
```cpp
59+
row[col - 1] = buffers.intBuffers[col - 1][i]; // pybind11 does:
60+
// 1. Type detection (is this an int?)
61+
// 2. Create py::int_ wrapper
62+
// 3. Convert to PyObject*
63+
// 4. Bounds-check list assignment
64+
// 5. Reference count management
65+
```
66+
67+
This wrapper overhead costs ~20-40 CPU cycles per cell for simple operations.
68+
69+
### Solution
70+
Use Python C API directly to bypass pybind11 for simple numeric types:
71+
- **Integers**: `PyLong_FromLong()` / `PyLong_FromLongLong()`
72+
- **Floats**: `PyFloat_FromDouble()`
73+
- **Booleans**: `PyBool_FromLong()`
74+
- **Assignment**: `PyList_SET_ITEM()` macro (no bounds checking - list pre-allocated with correct size)
75+
76+
### Code Changes
77+
```cpp
78+
// BEFORE (pybind11 wrapper)
79+
row[col - 1] = buffers.intBuffers[col - 1][i];
80+
81+
// AFTER (direct Python C API)
82+
if (buffers.indicators[col - 1][i] == SQL_NULL_DATA) {
83+
Py_INCREF(Py_None);
84+
PyList_SET_ITEM(row.ptr(), col - 1, Py_None);
85+
} else {
86+
PyObject* pyInt = PyLong_FromLong(buffers.intBuffers[col - 1][i]);
87+
PyList_SET_ITEM(row.ptr(), col - 1, pyInt);
88+
}
89+
```
90+
91+
### Impact
92+
- ✅ Eliminates pybind11 wrapper overhead (20-40 CPU cycles per cell)
93+
- ✅ Direct array access via `PyList_SET_ITEM` macro (expands to `list->ob_item[i] = value`)
94+
- ✅ No bounds checking (we pre-allocated the list with correct size)
95+
- ✅ Explicit NULL handling for each numeric type
96+
97+
### Affected Data Types
98+
**Optimized (7 types):**
99+
- `SQL_INTEGER``PyLong_FromLong()`
100+
- `SQL_SMALLINT``PyLong_FromLong()`
101+
- `SQL_BIGINT``PyLong_FromLongLong()`
102+
- `SQL_TINYINT``PyLong_FromLong()`
103+
- `SQL_BIT``PyBool_FromLong()`
104+
- `SQL_REAL``PyFloat_FromDouble()`
105+
- `SQL_DOUBLE`, `SQL_FLOAT``PyFloat_FromDouble()`
106+
107+
**Not Changed:**
108+
- Complex types like `DECIMAL`, `DATETIME`, `GUID` (still use pybind11 for type conversion logic)
109+
- String types (already optimized or use specific paths)
54110

55111
---
56112

0 commit comments

Comments
 (0)