Skip to content

Commit 6e694d7

Browse files
ferrolhoferrolho
committed
Update benchmark results
1 parent 475312c commit 6e694d7

File tree

3 files changed

+97
-92
lines changed

3 files changed

+97
-92
lines changed

‎docs/src/benchmarks.md‎

Lines changed: 47 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,14 @@
11
# Benchmarks
22

3-
To attain maximal performance, it is recommended to pass `-O3`, `--check-bounds=no` as command line flags to `julia`. As of Julia 1.1, maximizing performance for the `dynamics!` algorithm requires either setting the number of BLAS threads to 1 (`using LinearAlgebra; BLAS.set_num_threads(1)`) if using OpenBLAS (the default), or compiling Julia with MKL. See [this issue](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/issues/500) for more information.
3+
To attain maximal performance, it is recommended to pass `-O3` and `--check-bounds=no` as command line flags to `julia`:
4+
5+
```bash
6+
cd RigidBodyDynamics.jl
7+
julia -O3 --check-bounds=no perf/runbenchmarks.jl
8+
```
9+
10+
> **Warning**
11+
> For Julia versions previous to `v1.8`, maximizing performance for the `dynamics!` algorithm requires either setting the number of BLAS threads to 1 (`using LinearAlgebra; BLAS.set_num_threads(1)`) if using OpenBLAS (the default), or compiling Julia with MKL. See [this issue](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/issues/500) for more information.
412
513
Run `perf/runbenchmarks.jl` to see benchmark results for the Atlas robot (v5). Results below are for the following scenarios:
614

@@ -9,69 +17,62 @@ Run `perf/runbenchmarks.jl` to see benchmark results for the Atlas robot (v5). R
917
3. Do inverse dynamics.
1018
4. Do forward dynamics.
1119

12-
Note that results on CI builds are **not at all** representative because of code coverage. Results on a reasonably fast laptop at commit [870bea6](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/commit/870bea668d5b11ce0555fa0552592d2c3cb15c54):
20+
> **Note**
21+
> Results on CI builds are **not at all** representative because of code coverage.
1322
14-
Output of `versioninfo()`:
23+
Below are the results for **RBD.jl 2.5.0** (commit [`93a5ea`](https://github.com/JuliaRobotics/RigidBodyDynamics.jl/commit/93a5eaf15a5f6714b1ec1ce621b053542dcb721d)) using **Julia 1.11.1** on an **Apple MacBook Air (M2, 2023)** (16GB RAM, 512GB SSD):
1524

25+
Output of `versioninfo()`:
1626
```
17-
Julia Version 1.5.3
18-
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
27+
Julia Version 1.11.1
28+
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
29+
Build Info:
30+
Official https://julialang.org/ release
1931
Platform Info:
20-
OS: macOS (x86_64-apple-darwin18.7.0)
21-
CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
32+
OS: macOS (arm64-apple-darwin22.4.0)
33+
CPU: 8 × Apple M2
2234
WORD_SIZE: 64
23-
LIBM: libopenlibm
24-
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
35+
LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
36+
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)
2537
```
2638

27-
Note that this is a different machine than the one that was used for earlier benchmarks.
28-
29-
Mass matrix:
39+
> **Note**
40+
> This is a different machine than the one that was used for earlier benchmarks.
3041
42+
Mass matrix ([`mass_matrix!`](@ref)):
3143
```
32-
memory estimate: 0 bytes
33-
allocs estimate: 0
34-
--------------
35-
minimum time: 4.415 μs (0.00% GC)
36-
median time: 4.579 μs (0.00% GC)
37-
mean time: 4.916 μs (0.00% GC)
38-
maximum time: 19.794 μs (0.00% GC)
44+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
45+
Range (min … max): 3.728 μs … 10.024 μs ┊ GC (min … max): 0.00% … 0.00%
46+
Time (median): 3.874 μs ┊ GC (median): 0.00%
47+
Time (mean ± σ): 3.903 μs ± 180.208 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
48+
Memory estimate: 0 bytes, allocs estimate: 0.
3949
```
4050

41-
Mass matrix and Jacobian from left hand to right foot:
42-
51+
Mass matrix ([`mass_matrix!`](@ref)) and Jacobian ([`geometric_jacobian!`](@ref)) from left hand to right foot:
4352
```
44-
memory estimate: 0 bytes
45-
allocs estimate: 0
46-
--------------
47-
minimum time: 4.860 μs (0.00% GC)
48-
median time: 4.982 μs (0.00% GC)
49-
mean time: 5.399 μs (0.00% GC)
50-
maximum time: 24.712 μs (0.00% GC)
53+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
54+
Range (min … max): 3.941 μs … 9.020 μs ┊ GC (min … max): 0.00% … 0.00%
55+
Time (median): 4.103 μs ┊ GC (median): 0.00%
56+
Time (mean ± σ): 4.135 μs ± 196.842 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
57+
Memory estimate: 0 bytes, allocs estimate: 0.
5158
```
5259

5360
Note the low additional cost of computing a Jacobian when the mass matrix is already computed. This is because RigidBodyDynamics.jl caches intermediate computation results.
5461

55-
Inverse dynamics:
56-
62+
Inverse dynamics ([`inverse_dynamics!`](@ref)):
5763
```
58-
memory estimate: 0 bytes
59-
allocs estimate: 0
60-
--------------
61-
minimum time: 4.256 μs (0.00% GC)
62-
median time: 4.541 μs (0.00% GC)
63-
mean time: 4.831 μs (0.00% GC)
64-
maximum time: 21.625 μs (0.00% GC)
64+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
65+
Range (min … max): 2.736 μs … 5.666 μs ┊ GC (min … max): 0.00% … 0.00%
66+
Time (median): 2.866 μs ┊ GC (median): 0.00%
67+
Time (mean ± σ): 2.882 μs ± 119.781 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
68+
Memory estimate: 0 bytes, allocs estimate: 0.
6569
```
6670

67-
Forward dynamics:
68-
71+
Forward dynamics ([`dynamics!`](@ref)):
6972
```
70-
memory estimate: 0 bytes
71-
allocs estimate: 0
72-
--------------
73-
minimum time: 13.600 μs (0.00% GC)
74-
median time: 14.419 μs (0.00% GC)
75-
mean time: 16.071 μs (0.00% GC)
76-
maximum time: 55.328 μs (0.00% GC)
73+
BenchmarkTools.Trial: 10000 samples with 10 evaluations.
74+
Range (min … max): 9.791 μs … 13.899 μs ┊ GC (min … max): 0.00% … 0.00%
75+
Time (median): 9.874 μs ┊ GC (median): 0.00%
76+
Time (mean ± σ): 9.942 μs ± 292.126 ns ┊ GC (mean ± σ): 0.00% ± 0.00%
77+
Memory estimate: 0 bytes, allocs estimate: 0.
7778
```

‎perf/Manifest.toml‎

Lines changed: 50 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# This file is machine-generated - editing it directly is not advised
22

3-
julia_version = "1.9.0"
3+
julia_version = "1.9.4"
44
manifest_format = "2.0"
55
project_hash = "65dd92ee98603779b07a7023ff0989262d569492"
66

@@ -16,20 +16,20 @@ uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
1616

1717
[[deps.BenchmarkTools]]
1818
deps = ["JSON", "Logging", "Printf", "Profile", "Statistics", "UUIDs"]
19-
git-tree-sha1 = "d9a9701b899b30332bbcb3e1679c41cce81fb0e8"
19+
git-tree-sha1 = "f1dff6729bc61f4d49e140da1af55dcd1ac97b2f"
2020
uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
21-
version = "1.3.2"
21+
version = "1.5.0"
2222

2323
[[deps.Calculus]]
2424
deps = ["LinearAlgebra"]
25-
git-tree-sha1 = "f641eb0a4f00c343bbc32346e1217b86f3ce9dad"
25+
git-tree-sha1 = "9cb23bbb1127eefb022b022481466c0f1127d430"
2626
uuid = "49dc2e85-a5d0-5ad3-a950-438e2897f1b9"
27-
version = "0.5.1"
27+
version = "0.5.2"
2828

2929
[[deps.CompilerSupportLibraries_jll]]
3030
deps = ["Artifacts", "Libdl"]
3131
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
32-
version = "1.0.2+0"
32+
version = "1.0.5+0"
3333

3434
[[deps.Dates]]
3535
deps = ["Printf"]
@@ -65,10 +65,10 @@ uuid = "92d709cd-6900-40b7-9082-c6be49f344b6"
6565
version = "0.2.2"
6666

6767
[[deps.JLLWrappers]]
68-
deps = ["Preferences"]
69-
git-tree-sha1 = "abc9885a7ca2052a736a600f7fa66209f96506e1"
68+
deps = ["Artifacts", "Preferences"]
69+
git-tree-sha1 = "be3dc50a92e5a386872a493a10050136d4703f9b"
7070
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
71-
version = "1.4.1"
71+
version = "1.6.1"
7272

7373
[[deps.JSON]]
7474
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
@@ -79,12 +79,12 @@ version = "0.21.4"
7979
[[deps.LibCURL]]
8080
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
8181
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
82-
version = "0.6.3"
82+
version = "0.6.4"
8383

8484
[[deps.LibCURL_jll]]
8585
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
8686
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
87-
version = "7.84.0+0"
87+
version = "8.4.0+0"
8888

8989
[[deps.LibGit2]]
9090
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
@@ -93,32 +93,32 @@ uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
9393
[[deps.LibSSH2_jll]]
9494
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
9595
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
96-
version = "1.10.2+0"
96+
version = "1.11.0+1"
9797

9898
[[deps.Libdl]]
9999
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
100100

101101
[[deps.Libiconv_jll]]
102-
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
103-
git-tree-sha1 = "c7cb1f5d892775ba13767a87c7ada0b980ea0a71"
102+
deps = ["Artifacts", "JLLWrappers", "Libdl"]
103+
git-tree-sha1 = "61dfdba58e585066d8bce214c5a51eaa0539f269"
104104
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
105-
version = "1.16.1+2"
105+
version = "1.17.0+1"
106106

107107
[[deps.LightXML]]
108108
deps = ["Libdl", "XML2_jll"]
109-
git-tree-sha1 = "e129d9391168c677cd4800f5c0abb1ed8cb3794f"
109+
git-tree-sha1 = "3a994404d3f6709610701c7dabfc03fed87a81f8"
110110
uuid = "9c8b4983-aa76-5018-a973-4c85ecc9e179"
111-
version = "0.9.0"
111+
version = "0.9.1"
112112

113113
[[deps.LinearAlgebra]]
114114
deps = ["Libdl", "OpenBLAS_jll", "libblastrampoline_jll"]
115115
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
116116

117117
[[deps.LogExpFunctions]]
118118
deps = ["DocStringExtensions", "IrrationalConstants", "LinearAlgebra"]
119-
git-tree-sha1 = "0a1b7c2863e44523180fdb3146534e265a91870b"
119+
git-tree-sha1 = "a2d09619db4e765091ee5c6ffe8872849de0feea"
120120
uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688"
121-
version = "0.3.23"
121+
version = "0.3.28"
122122

123123
[deps.LogExpFunctions.extensions]
124124
LogExpFunctionsChainRulesCoreExt = "ChainRulesCore"
@@ -183,26 +183,26 @@ version = "0.5.5+0"
183183

184184
[[deps.Parsers]]
185185
deps = ["Dates", "PrecompileTools", "UUIDs"]
186-
git-tree-sha1 = "7302075e5e06da7d000d9bfa055013e3e85578ca"
186+
git-tree-sha1 = "8489905bcdbcfac64d1daa51ca07c0d8f0283821"
187187
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
188-
version = "2.5.9"
188+
version = "2.8.1"
189189

190190
[[deps.Pkg]]
191191
deps = ["Artifacts", "Dates", "Downloads", "FileWatching", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
192192
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
193-
version = "1.9.0"
193+
version = "1.9.2"
194194

195195
[[deps.PrecompileTools]]
196196
deps = ["Preferences"]
197-
git-tree-sha1 = "259e206946c293698122f63e2b513a7c99a244e8"
197+
git-tree-sha1 = "5aa36f7049a63a1528fe8f7c3f2113413ffd4e1f"
198198
uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
199-
version = "1.1.1"
199+
version = "1.2.1"
200200

201201
[[deps.Preferences]]
202202
deps = ["TOML"]
203-
git-tree-sha1 = "7eb1686b4f04b82f96ed7a4ea5890a4f0c7a09f1"
203+
git-tree-sha1 = "9306f6085165d270f7e3db02af26a400d580f5c6"
204204
uuid = "21216c6a-2e73-6563-6e65-726566657250"
205-
version = "1.4.0"
205+
version = "1.4.3"
206206

207207
[[deps.Printf]]
208208
deps = ["Unicode"]
@@ -233,9 +233,9 @@ version = "1.2.2"
233233

234234
[[deps.RigidBodyDynamics]]
235235
deps = ["DocStringExtensions", "LightXML", "LinearAlgebra", "LoopThrottle", "Random", "Reexport", "Rotations", "SparseArrays", "StaticArrays", "TypeSortedCollections", "UnsafeArrays"]
236-
path = ".."
236+
git-tree-sha1 = "313ea256ced33aa6039987c9fef57e59aa229589"
237237
uuid = "366cf18f-59d5-5db9-a4de-86a9f6786172"
238-
version = "2.3.2"
238+
version = "2.4.0"
239239

240240
[[deps.Rotations]]
241241
deps = ["LinearAlgebra", "Quaternions", "Random", "StaticArrays", "Statistics"]
@@ -259,9 +259,9 @@ uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
259259

260260
[[deps.SpecialFunctions]]
261261
deps = ["IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"]
262-
git-tree-sha1 = "ef28127915f4229c971eb43f3fc075dd3fe91880"
262+
git-tree-sha1 = "2f5d4697f21388cbe1ff299430dd169ef97d7e14"
263263
uuid = "276daf66-3868-5448-9aa4-cd146d93841b"
264-
version = "2.2.0"
264+
version = "2.4.0"
265265

266266
[deps.SpecialFunctions.extensions]
267267
SpecialFunctionsChainRulesCoreExt = "ChainRulesCore"
@@ -270,15 +270,23 @@ version = "2.2.0"
270270
ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
271271

272272
[[deps.StaticArrays]]
273-
deps = ["LinearAlgebra", "Random", "StaticArraysCore", "Statistics"]
274-
git-tree-sha1 = "c262c8e978048c2b095be1672c9bee55b4619521"
273+
deps = ["LinearAlgebra", "PrecompileTools", "Random", "StaticArraysCore"]
274+
git-tree-sha1 = "777657803913ffc7e8cc20f0fd04b634f871af8f"
275275
uuid = "90137ffa-7385-5640-81b9-e52037218182"
276-
version = "1.5.24"
276+
version = "1.9.8"
277+
278+
[deps.StaticArrays.extensions]
279+
StaticArraysChainRulesCoreExt = "ChainRulesCore"
280+
StaticArraysStatisticsExt = "Statistics"
281+
282+
[deps.StaticArrays.weakdeps]
283+
ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
284+
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
277285

278286
[[deps.StaticArraysCore]]
279-
git-tree-sha1 = "6b7ba252635a5eff6a0b0664a41ee140a1c9e72a"
287+
git-tree-sha1 = "192954ef1208c7019899fbf8049e717f92959682"
280288
uuid = "1e83bf80-4336-4d27-bf5d-d5a4f845583c"
281-
version = "1.4.0"
289+
version = "1.4.3"
282290

283291
[[deps.Statistics]]
284292
deps = ["LinearAlgebra", "SparseArrays"]
@@ -317,15 +325,15 @@ uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
317325
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
318326

319327
[[deps.UnsafeArrays]]
320-
git-tree-sha1 = "3350f94f6caa02f324a23645bf524fc9334c7488"
328+
git-tree-sha1 = "da0c9ca60d3371a4bc86b4e65c45db17086fb3ac"
321329
uuid = "c4a57d5a-5b31-53a6-b365-19f8c011fbd6"
322-
version = "1.0.4"
330+
version = "1.0.6"
323331

324332
[[deps.XML2_jll]]
325-
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
326-
git-tree-sha1 = "93c41695bc1c08c46c5899f4fe06d6ead504bb73"
333+
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Zlib_jll"]
334+
git-tree-sha1 = "6a451c6f33a176150f315726eba8b92fbfdb9ae7"
327335
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
328-
version = "2.10.3+0"
336+
version = "2.13.4+0"
329337

330338
[[deps.Zlib_jll]]
331339
deps = ["Libdl"]
@@ -335,12 +343,12 @@ version = "1.2.13+0"
335343
[[deps.libblastrampoline_jll]]
336344
deps = ["Artifacts", "Libdl"]
337345
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
338-
version = "5.7.0+0"
346+
version = "5.8.0+0"
339347

340348
[[deps.nghttp2_jll]]
341349
deps = ["Artifacts", "Libdl"]
342350
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
343-
version = "1.48.0+0"
351+
version = "1.52.0+1"
344352

345353
[[deps.p7zip_jll]]
346354
deps = ["Artifacts", "Libdl"]

‎perf/runbenchmarks.jl‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,6 @@ using RigidBodyDynamics.OdeIntegrators
77
using Random
88
using Profile
99
using BenchmarkTools
10-
using LinearAlgebra
11-
12-
# Due to https://github.com/JuliaRobotics/RigidBodyDynamics.jl/issues/500:
13-
BLAS.set_num_threads(1)
1410

1511
const ScalarType = Float64
1612
# const ScalarType = Float32

0 commit comments

Comments
 (0)