Skip to content

Commit 1574d42

Browse files
committed
dataset_tool.py docs update, tf32 disable for calc_metrics.py
- Add a justification for using uncompressed zip and uncompressed png (#22) - Clarify class label usage with dataset_tool.py (#18) - Disable tf32 in calc_metrics as this has not been tested.
1 parent f7e4867 commit 1574d42

File tree

2 files changed

+23
-1
lines changed

2 files changed

+23
-1
lines changed

calc_metrics.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ def subprocess_fn(rank, args, temp_dir):
4747
# Print network summary.
4848
device = torch.device('cuda', rank)
4949
torch.backends.cudnn.benchmark = True
50+
torch.backends.cuda.matmul.allow_tf32 = False
51+
torch.backends.cudnn.allow_tf32 = False
5052
G = copy.deepcopy(args.G).eval().requires_grad_(False).to(device)
5153
if rank == 0 and args.verbose:
5254
z = torch.empty([1, G.z_dim], device=device)

dataset_tool.py

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -331,14 +331,34 @@ def convert_dataset(
331331
--source path/ Recursively load all images from path/
332332
--source dataset.zip Recursively load all images from dataset.zip
333333
334-
The output dataset format can be either an image folder or a zip archive.
335334
Specifying the output format and path:
336335
337336
\b
338337
--dest /path/to/dir Save output files under /path/to/dir
339338
--dest /path/to/dataset.zip Save output files into /path/to/dataset.zip
340339
340+
The output dataset format can be either an image folder or an uncompressed zip archive.
341+
Zip archives makes it easier to move datasets around file servers and clusters, and may
342+
offer better training performance on network file systems.
343+
341344
Images within the dataset archive will be stored as uncompressed PNG.
345+
Uncompresed PNGs can be efficiently decoded in the training loop.
346+
347+
Class labels are stored in a file called 'dataset.json' that is stored at the
348+
dataset root folder. This file has the following structure:
349+
350+
\b
351+
{
352+
"labels": [
353+
["00000/img00000000.png",6],
354+
["00000/img00000001.png",9],
355+
... repeated for every image in the datase
356+
["00049/img00049999.png",1]
357+
]
358+
}
359+
360+
If the 'dataset.json' file cannot be found, the dataset is interpreted as
361+
not containing class labels.
342362
343363
Image scale/crop and resolution requirements:
344364

0 commit comments

Comments
 (0)