1. Tensors & Hardware Acceleration
At its core, PyTorch is a library for processing multidimensional arrays called Tensors. Similar to NumPy arrays, Tensors can encode inputs, outputs, and parameters of models. Crucially, Tensors can run on GPUs to massively accelerate computing.
💻 Code: Tensor Basics
Creating a tensor and moving it to an accelerated device (GPU/MPS).
import torch
# Create a random 3x3 tensor
x = torch.rand(3, 3)
# Check for GPU availability
device = "cuda" if torch.cuda.is_available() else "cpu"
# Move tensor to the device
x = x.to(device)
print(f"Tensor is on: {x.device}")⚡ CPU vs GPU Performance
Simulated execution time for matrix multiplication (lower is better). Shows the massive advantage of GPU acceleration for large tensors.
2. Building Neural Networks
PyTorch provides the torch.nn namespace, containing all building blocks to create complex neural networks. A neural network is a module consisting of other modules (layers).
🧠 Defining a Model
Subclassing nn.Module to build a simple feed-forward network. Layers are defined in __init__ and data flow in forward.
import torch.nn as nn
class SimpleNet(nn.Module):
def __init__(self):
super().__init__()
self.flatten = nn.Flatten()
self.stack = nn.Sequential(
nn.Linear(28*28, 512),
nn.ReLU(),
nn.Linear(512, 10)
)
def forward(self, x):
x = self.flatten(x)
return self.stack(x)📈 Activation Functions
ReLU outputs the input directly if positive, else zero. It is the default choice for hidden layers due to fast computation and preventing vanishing gradients.
3. The Training Loop
Training is an iterative process. In each iteration (epoch), the model makes a guess, calculates the error (Loss), and adjusts its parameters to reduce that error using Autograd and an Optimizer.
1. Forward Pass
Pass input data through the model to generate predictions.
pred = model(X)
2. Compute Loss
Compare predictions to actual targets to calculate the error.
loss = loss_fn(pred, y)
3. Backward Pass
Compute gradients of the loss with respect to parameters.
loss.backward()
4. Optimizer Step
Update model weights based on calculated gradients.
optimizer.step()
🎯 Live Training Simulation
Watch the model loss decrease over epochs.
4. Advanced PyTorch Learning Roadmap
To become proficient in building advanced neural networks, explore these essential domains. This interactive matrix covers optimization techniques, complex architectures, scaling, and production deployment.
Select a domain from the visual map or the list below to explore study topics and practical guides.