[MRG] DOC Add example: Effect of resampling on probability calibration #1151
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issue
Addres issue #1128.
What does this implement/fix? Explain your changes.
I added a new example to the examples folder showing with plots and metrics the effect of resampling methods on the calibration of predicted probabilities.
Context
CalibratedClassifierCVAny other comments?
This PR provides a manual workflow of an example so one can understand and see the issue. It does not implement an automated solution within the library's codebase.
A fix would likely require implementing a new meta estimator and this is beyond the scope of this contribution.
I hope with this example I can help someone understand the issue and get an idea of how to solve it.