1410 Accelerate add_edge for large graphs

[kilianvolmer]

Changes and Information

Please briefly list the changes (main added features, changed items, or corrected bugs) made:

• Added a GraphBuilder class that stores unorganized node and edge vectors and then builds a proper, organized graph.
• Added a new constructor to the graph that can be used by the GraphBuilder
• Turn original add_edge and add_node functions into void.

If need be, add additional information and what the reviewer should look out for in particular:

• This is the first time that I touched the python bindings, so please check that I did nothing silly.

Merge Request - Guideline Checklist

Please check our git workflow. Use the draft feature if the Pull Request is not yet ready to review.

Checks by code author

• Every addressed issue is linked (use the "Closes #ISSUE" keyword below)
• New code adheres to coding guidelines
• No large data files have been added (files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
• Tests are added for new functionality and a local test run was successful (with and without OpenMP)
• Appropriate documentation within the code (Doxygen) for new functionality has been added in the code
• Appropriate external documentation (ReadTheDocs) for new functionality has been added to the online documentation
• Proper attention to licenses, especially no new third-party software with conflicting license has been added
• (For ABM development) Checked benchmark results and ran and posted a local test above from before and after development to ensure performance is monitored.

Checks by code reviewer(s)

• Corresponding issue(s) is/are linked and addressed
• Code is clean of development artifacts (no deactivated or commented code lines, no debugging printouts, etc.)
• Appropriate unit tests have been added, CI passes, code coverage and performance is acceptable (did not decrease)
• No large data files added in the whole history of commits(files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
• On merge, add 2-5 lines with the changes (main added features, changed items, or corrected bugs) to the merge-commit-message. This can be taken from the briefly-list-the-changes above (best case) or the separate commit messages (worst case).

Closes #1410

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.28%. Comparing base (47f4820) to head (5f35220).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1411      +/-   ##
==========================================
- Coverage   97.29%   97.28%   -0.01%     
==========================================
  Files         180      180              
  Lines       15646    15681      +35     
==========================================
+ Hits        15223    15256      +33     
- Misses        423      425       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[HenrZu]

Thanks for the update! Im still not entirely sure about the overall usefulness of the feature.
Do you have some information about the overhead caused by the sorting and checking for dublicates?
if we want to add this feature, we should also extend the documentation of the current add edge function starting in line 169.

[HenrZu]

also mention that properties might get lost.
Additionally, if i add two edges with the same start/end combination. Which edges gets deleted? the one that was added latest?

[kilianvolmer]

Outdated comment, but we will now always keep the first edge if we use the GraphBuilder.

[HenrZu]

we do a copy here. Is this really neccessary?

[kilianvolmer]

As far as I know, yes.

[HenrZu]

Ok, now i see the motivation. Some questions.
Does that really make a noticeable difference? Have you ever quantified it?

Especially in simulations with many edges, we expect a relatively large runtime. Is the overhead caused by checking so much higher that we accept new sources of error (such as forgetting to call the sort function, etc.)?

[kilianvolmer]

Yes, it does make a difference of around four orders of magnitude in example simulations. For example, with 1.6 million edges I measure 4.5e+03 seconds with the standard function and 9.6e-01 seconds with the new functions.