Merge branch 'main' into chirag/fix-ddp-series-multigpu

Author: c-p-i-o

beginner_source/hyperparameter_tuning_tutorial.py

@@ -10,7 +10,7 @@
 Fortunately, there are tools that help with finding the best combination of parameters.
 `Ray Tune <https://docs.ray.io/en/latest/tune.html>`_ is an industry standard tool for
 distributed hyperparameter tuning. Ray Tune includes the latest hyperparameter search
-algorithms, integrates with TensorBoard and other analysis libraries, and natively
+algorithms, integrates with various analysis libraries, and natively
 supports distributed training through `Ray's distributed machine learning engine
 <https://ray.io/>`_.
 

beginner_source/ptcheat.rst

@@ -22,27 +22,12 @@ Neural Network API
     import torch.nn as nn                     # neural networks
     import torch.nn.functional as F           # layers, activations and more
     import torch.optim as optim               # optimizers e.g. gradient descent, ADAM, etc.
-    from torch.jit import script, trace       # hybrid frontend decorator and tracing jit
 
 See `autograd <https://pytorch.org/docs/stable/autograd.html>`__,
 `nn <https://pytorch.org/docs/stable/nn.html>`__,
 `functional <https://pytorch.org/docs/stable/nn.html#torch-nn-functional>`__
 and `optim <https://pytorch.org/docs/stable/optim.html>`__
 
-TorchScript and JIT
--------------------
-
-.. code-block:: python
-
-    torch.jit.trace()         # takes your module or function and an example 
-                              # data input, and traces the computational steps 
-                              # that the data encounters as it progresses through the model
-
-    @script                   # decorator used to indicate data-dependent 
-                              # control flow within the code being traced
-
-See `Torchscript <https://pytorch.org/docs/stable/jit.html>`__
-
 ONNX
 ----
 
@@ -225,8 +210,10 @@ Optimizers
 
     opt = optim.x(model.parameters(), ...)      # create optimizer
     opt.step()                                  # update weights
-    optim.X                                     # where X is SGD, Adadelta, Adagrad, Adam, 
-                                                # AdamW, SparseAdam, Adamax, ASGD, 
+    opt.zero_grad()                             # clear the gradients
+    optim.X                                     # where X is SGD, AdamW, Adam,
+                                                # Adafactor, NAdam, RAdam, Adadelta,
+                                                # Adagrad, SparseAdam, Adamax, ASGD,
                                                 # LBFGS, RMSprop or Rprop
 
 See `optimizers <https://pytorch.org/docs/stable/optim.html>`__

en-wordlist.txt

@@ -1,4 +1,3 @@
-
 ACL
 ADI
 AOT
@@ -188,6 +187,7 @@ PIL's
 PPO
 PatchPredictor
 PennFudan
+Perfetto
 Pixman
 Plotly
 Pohjonen

intermediate_source/tensorboard_profiler_tutorial.py

@@ -4,6 +4,14 @@
 This tutorial demonstrates how to use TensorBoard plugin with PyTorch Profiler
 to detect performance bottlenecks of the model.
 
+.. warning::
+   The TensorBoard integration with the PyTorch profiler is now
+   deprecated. Instead, use Perfetto or the Chrome trace to
+   view ``trace.json`` files. After
+   `generating a trace <https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html#using-tracing-functionality>`__,
+   simply drag the ``trace.json`` into `Perfetto UI <https://ui.perfetto.dev/>`__
+   or ``chrome://tracing`` to visualize your profile.
+
 Introduction
 ------------
 PyTorch 1.8 includes an updated profiler API capable of

recipes_source/recipes/profiler_recipe.py

@@ -459,7 +459,6 @@ def trace_handler(p):
 #
 # Take a look at the following recipes/tutorials to continue your learning:
 #
-# -  `PyTorch Benchmark <https://pytorch.org/tutorials/recipes/recipes/benchmark.html>`_
-# -  `PyTorch Profiler with TensorBoard <https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html>`_ tutorial
-# -  `Visualizing models, data, and training with TensorBoard <https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html>`_ tutorial
+# - `PyTorch Benchmark <https://pytorch.org/tutorials/recipes/recipes/benchmark.html>`_
+# - `Visualizing models, data, and training with TensorBoard <https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html>`_ tutorial
 #