Make safetensor shard filename parsing more flexible (#1817)

Context:

A lot of deepseek models have unconventional number of digits for the
total shard number in their filename, which leads to a filename parsing
fail. It breaks the file info navigation on the hub:



https://github.com/user-attachments/assets/75676c00-387c-4a92-808d-0d84caba7e51




[here](https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp?show_file_info=model-00001-of-000163.safetensors)
for instance, `model-00001-of-000163.safetensors` should be
`model-00001-of-00163.safetensors` (there's 1 extra "0" in
"model-00001-of-.safetensors")

Most of deepseek models with a lot of shards have this issue, since they
are important models, I suggest we make the regex more flexible to
accept a variable number of digits for shards in the filename.

Would that be acceptable?

Author: kefranabg

packages/hub/src/lib/parse-safetensors-metadata.spec.ts

@@ -143,6 +143,16 @@ describe("parseSafetensorsMetadata", () => {
 		assert.strictEqual(safetensorsShardFileInfo?.total, "00072");
 	});
 
+	it("should detect sharded safetensors filename with 6 digits", async () => {
+		const safetensorsFilename = "model-00001-of-000163.safetensors"; // https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Exp/blob/main/model-00001-of-000163.safetensors
+		const safetensorsShardFileInfo = parseSafetensorsShardFilename(safetensorsFilename);
+
+		assert.strictEqual(safetensorsShardFileInfo?.prefix, "model-");
+		assert.strictEqual(safetensorsShardFileInfo?.basePrefix, "model");
+		assert.strictEqual(safetensorsShardFileInfo?.shard, "00001");
+		assert.strictEqual(safetensorsShardFileInfo?.total, "000163");
+	});
+
 	it("should support sub-byte data types", async () => {
 		const newDataTypes: Array<"F4" | "F6_E2M3" | "F6_E3M2" | "E8M0"> = ["F4", "F6_E2M3", "F6_E3M2", "E8M0"];
 

packages/hub/src/lib/parse-safetensors-metadata.ts

@@ -14,7 +14,7 @@ export const SAFETENSORS_INDEX_FILE = "model.safetensors.index.json";
 export const RE_SAFETENSORS_FILE = /\.safetensors$/;
 export const RE_SAFETENSORS_INDEX_FILE = /\.safetensors\.index\.json$/;
 export const RE_SAFETENSORS_SHARD_FILE =
-	/^(?<prefix>(?<basePrefix>.*?)[_-])(?<shard>\d{5})-of-(?<total>\d{5})\.safetensors$/;
+	/^(?<prefix>(?<basePrefix>.*?)[_-])(?<shard>\d{5,6})-of-(?<total>\d{5,6})\.safetensors$/;
 export interface SafetensorsShardFileInfo {
 	prefix: string;
 	basePrefix: string;