Commit 7f2d022
authored
GCD bug fix
The standard-library documentation for `trailingZeroBitCount` states:
> If the value is zero, then `trailingZeroBitCount` is equal to `bitWidth`.
Thus, if `T` is not a fixed-width integer type, and `x == 0`, and `y` has more trailing zeros than the representation of `x`, then the gcd calculation will be incorrect.
In particular, the left-shift by `min(xtz, ytz) == xtz` will not “undo” the earlier right-shift by `ytz`, so the final result will be incorrectly right-shifted by `ytz - xtz`.
To fix this I have added an early exit when `x == 0`. I am not sure if this is the optimal solution, and I’m open to alternatives.
***
This change ensures that `x != 0` at the top of the loop, so I have also converted it from a `while` to a `repeat-while` loop, in order to eliminate a redundant extra comparison on the first iteration.
I don’t know if this actually affects performance, and I’m happy to change it back if that’s preferred.1 parent 4ce74b4 commit 7f2d022
1 file changed
+6
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| |||
39 | 40 | | |
40 | 41 | | |
41 | 42 | | |
42 | | - | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
43 | 47 | | |
44 | 48 | | |
45 | 49 | | |
46 | | - | |
| 50 | + | |
47 | 51 | | |
48 | 52 | | |
49 | 53 | | |
0 commit comments