Skip to content

Replace all sort functions with stable sort #2525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 23, 2024
Merged

Replace all sort functions with stable sort #2525

merged 16 commits into from
May 23, 2024

Conversation

amin1377
Copy link
Contributor

@amin1377 amin1377 commented Apr 2, 2024

In this pull request, all sorting functions have been replaced with stable sort to ensure consistency in results across platforms. This adjustment mirrors changes already implemented in the version of VPR used in SPEC benchmarks. The aim of this PR is to minimize discrepancies between SPEC's VPR and the upstream branch.

@github-actions github-actions bot added VPR VPR FPGA Placement & Routing Tool lang-cpp C/C++ code labels Apr 2, 2024
@amin1377
Copy link
Contributor Author

amin1377 commented Apr 3, 2024

QoR comparison on Titan ( I ran the benchmark twice to make sure that the results are consistent)
Screenshot from 2024-04-03 17-54-42
Screenshot from 2024-04-04 08-28-52

@vaughnbetz
Copy link
Contributor

Maybe the routing runtime reduction is real due to edge sorting somehow being faster? Please check the rr-graph build times.

I looked at the vtr_reg_strong failure and it is OK: one circuit gets a smaller minimum channel width (stereovision3 which is small) and then gets a longer wirelength. Please look at the other failures to confirm they're OK.

@vaughnbetz
Copy link
Contributor

Code looks fine. Maybe the stable sort leads to a more cache localized set of rr_graph edges (but that would only be possible if we actually had quite a few duplicate edges)?

@heshpdx
Copy link
Contributor

heshpdx commented Apr 11, 2024

In SPEC CPUv8 we started converting std::partition to std::stable_partition, and also investigating converting std::unordered_map to std::map and std::unordered_set to std::set. Even after all that there is still variation between different implementations of the C++ standard library (gcc vs llvm). Investigations ongoing...

@amin1377
Copy link
Contributor Author

I think that in VPR's placement and routing, we don't depend on the order of hashed objects. Currently, I'm working on reproducing the discrepancy between gcc-13 and llvm to localize the error. I'll then examine the code to see if such dependency exists. If it does, we also need to address it in the upstream VPR repository.

@vaughnbetz
Copy link
Contributor

Agreed. We definitely shouldn't have such a dependency on the order of hashed objects; if we do it is a bug. @soheilshahrouz found a (relatively rare) non-determinism in the master branch which may indicate someone is hashing on a pointer or some such. Adding him to the conversation in case he has any ideas or can help.

@vaughnbetz
Copy link
Contributor

We should fix the stable sort issue while investigating the other compiler-based output changed (maybe a non-determinism). @amin1377 : can you look at the CI failures and confirm they are benign (hopefully just small circuits with QoR changes)? Please put a report in of what you find so we can hopefully merge this.

@vaughnbetz
Copy link
Contributor

Looks at reg_nightly_1. Failures are 4 QoR failures on small arithmetic and multless const circuits (outside of bounds, but these are tiny designs) and one failure to complete (routing failure?) on a small multless const. Not sure why we had a routing failure; I think that one is worth looking at.

23:34:37 | vtr_reg_nightly_test1/arithmetic_tasks/multless_consts...[Fail]
23:34:37 | [Fail]
23:34:37 | fixed_k6_frac_ripple_N8_22nm.xml/mult_103.v/common vpr_status Task value 'success' does not match golden 'exited with return code 2'

@vaughnbetz
Copy link
Contributor

Updating branch to run CI again.
Amin, it would be good to put a note somewhere in the developer's guide about using stable_sort and stable_partition. Not sure where it should go -- maybe a new section on coding for reproducible results (spec)? I can help write it if you like, but I'd probably get you to put it in markup.

@vaughnbetz vaughnbetz merged commit f31fc95 into master May 23, 2024
100 checks passed
@vaughnbetz vaughnbetz deleted the stable_sort branch May 23, 2024 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lang-cpp C/C++ code VPR VPR FPGA Placement & Routing Tool
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants