File tree Expand file tree Collapse file tree 1 file changed +3
-7
lines changed Expand file tree Collapse file tree 1 file changed +3
-7
lines changed Original file line number Diff line number Diff line change 33Optimizing PyTorch training by wrapping torch.utils.data.Dataset with tensordict.TensorDict.MemoryMappedTensor mapped, pinned,
44and loaded onto an Nvidia GPU and inputting TensorDict(Dataset) into torch.utils.data.DataLoader--to boost model training speed.
55
6- ````
7- python==3.9.13
8- torch==2.3.1
9- tensordict==0.5.0
10- ````
11-
12- To run the demo [ NOTE: a mock dataset will be created in the current working directory] :
6+ To run the demo:
137````
148git clone https://github.com/OriYarden/pytorch_training_optimization_using_tensordict_memory_mapping
159cd pytorch_training_optimization_using_tensordict_memory_mapping
@@ -39,3 +33,5 @@ The initial wrapping runtime is approximately equal to 1 epoch of torch.utils.da
3933
4034
4135
36+
37+
You can’t perform that action at this time.
0 commit comments