Loss Functions
Loss functions measure the discrepancy between model predictions and target labels. ce_loss (cross-entropy) is the standard choice for classification — it expects raw logits and a same-shape target probability distribution (e.g. one-hot vectors). mse_loss (mean squared error) is the go-to for regression tasks. Both return a scalar Tensor whose .backward() triggers the gradient computation for the whole network.
import simplegrad as sg
import numpy as np
logits = sg.normal((4, 10), comp_grad=True)
targets = sg.Tensor(np.eye(10, dtype="float32")[[2, 7, 0, 5]]) # one-hot rows, shape (4, 10)
loss = sg.ce_loss(logits, targets)
loss.backward()
ce_loss
Cross-entropy loss over raw logits. A softmax is applied internally, so do not pass pre-softmaxed probabilities.
Compute cross-entropy loss with built-in softmax.
Numerically stable: uses the log-sum-exp trick internally.
Parameters:
-
z(Tensor) –Logits (raw unnormalized scores), shape
(..., num_classes). -
y(Tensor) –Target probability distribution, same shape as
z. -
dim(int, default:-1) –Class dimension to apply softmax over. Defaults to -1 (last dim).
-
reduction(str, default:'mean') –How to reduce the per-sample losses. One of
"mean","sum", orNone(return per-sample losses).
Raises:
-
ValueError–If
reductionis not a valid option.