The performance of the evaluation is poor

[JackeyMiao]

Hi,

It's a good job and so kind of you to share the code on the Github.

I have run and trained the model following the instruction, and choose the best performing model to evaluate on Setcover problem.

However, I found that the performance is so poor that it can hardly solve one of the big instance(all exceed the limit of time).

I wonder to know whether the situation is normal, or maybe something is wrong with the progress of my training?

The result can be attached as the following

learn2branch_CPU_20221023-214346.csv

hybrid_CPU_20221025-232556.csv

Thank you and have a good day.

[xiaofeishu141220]

Hello, I have configured scip-6.0.1 and pyscipopt-2.1.5 according to the installation idea provided by the author. However, when I run the code (02_generate_dataset.py), it does report this error.
"AttributeError: 'pyscipopt.scip.Model' object has no attribute 'getPseudoBranchCands'
Exception ignored in: 'pyscipopt.scip.PyBranchruleExeclp' "
I checked the following scip.pyx file and there is no function that the author need to call in their code, how can I solve this? Do I need to upgrade the solver version?

[pg2455]

Hello Xiao! Can you confirm if you have applied the patch as explained in the installation instructions here - https://github.com/ds4dm/learn2branch/blob/master/INSTALL.md.

It seems like your installation is missing the functions that are implemented in the patch. For some reason, your installation might not have the patch. Can you please check and confirm?

[pg2455]

Hello Jackey!

Setcover problems are hard in general. If you look at the Table 2.4 in the paper. You would see that the maximum number of instances solved are 27 out of 60. So it might happen that you don't see any solved instances. You might have to increase your time limit or the pool of instances to start seeing any results.

Also, please make sure that your evaluation machine is not overloaded with processes. The evaluation will highly depend on the hardware as well as the conditions of the hardware at the time evaluation. A good rule of thumb is to ensure that load average of your machine is always greater than 4 during evaluation.

I hope this helps! Please let me know if you have any other question.

[wangyuhu]

Hi,

It's a good job and so kind of you to share the code on the Github.

I have run and trained the model following the instruction, and choose the best performing model to evaluate on Setcover problem.

However, I found that the performance is so poor that it can hardly solve one of the big instance(all exceed the limit of time).

I wonder to know whether the situation is normal, or maybe something is wrong with the progress of my training?

The result can be attached as the following

learn2branch_CPU_20221023-214346.csv

hybrid_CPU_20221025-232556.csv

Thank you and have a good day.

Hello, buddy. I also encountered this issue while running this code. Have you solved it yet? Can you share your solution? Thanks very much! @pg2455