Retry metadata fetch with longer timeout when no local file exists #3569
Conversation
| _are_symlinks_supported_in_dir: dict[str, bool] = {} | ||
|
|
||
| # Internal retry timeout for metadata fetch when no local file exists | ||
| _ETAG_RETRY_TIMEOUT = 60 |
There was a problem hiding this comment.
no need to make it user configurable imo. i guess if a user has very slow network, they can already set HF_HUB_ETAG_TIMEOUT to increase the initial timeout
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Wauplin
left a comment
Wauplin
left a comment
There was a problem hiding this comment.
It's funny how I thought this PR would be a lot messier 😄
| _are_symlinks_supported_in_dir: dict[str, bool] = {} | ||
|
|
||
| # Internal retry timeout for metadata fetch when no local file exists | ||
| _ETAG_RETRY_TIMEOUT = 60 |
|
Just before merging, @LysandreJik @julien-c do you happen to have an opinion on this update? It looks sensible to me but since it touches a core part of the lib', I'd be glad to have your opinion |
|
@Wauplin Zero opinion (nor objection) on my side |
|
Great, that's all I wanted to hear 😁 |
|
let's merge then! 😄 |
Discussed internally with @Wauplin. If the
HEADcall we make to fetch metadata when downloading files times out (10s), it falls back to locally cached files. if there's no cached file and the network is just slow, the download fails immediately, so this PR adds a retry mechanism with a longer timeout (60s) when no local file is found. The flow is like this: