Linear
Linear applies an affine transformation y = x @ W + b to its input. It is the fundamental building block of fully connected networks. The weight matrix and bias are both initialised from a uniform distribution over [-sqrt(1/in_features), +sqrt(1/in_features)]. Both are registered as learnable parameters and updated by any Optimizer.
import simplegrad as sg
import simplegrad.nn as nn
fc = nn.Linear(in_features=784, out_features=256)
x = sg.normal((32, 784))
out = fc(x) # shape: (32, 256)
sg.sum(out).backward()
Linear
Bases: Module
Fully-connected linear layer: output = x @ W + b.
Weights are initialized with Kaiming uniform (range [-1/sqrt(in), 1/sqrt(in)]).
Parameters:
-
in_features(int | None, default:None) –Number of input features.
-
out_features(int | None, default:None) –Number of output features.
-
weight(Tensor | None, default:None) –Optional pre-built weight tensor of shape
(in_features, out_features). -
bias(Tensor | None, default:None) –Optional pre-built bias tensor of shape
(out_features,). -
use_bias(bool, default:True) –Add a bias term. Defaults to True.
-
dtype(str, default:None) –Data type string. Defaults to
"float32". -
weight_label(str, default:'W') –Label for the weight tensor (used in graph visualization).
-
bias_label(str, default:'b') –Label for the bias tensor.
Attributes
| Attribute | Type | Description |
|---|---|---|
.weight |
Tensor |
Weight matrix of shape (in_features, out_features). Learnable. |
.bias |
Tensor |
Bias vector of shape (out_features,). Only present when use_bias=True. |
.in_features |
int |
Number of input features. |
.out_features |
int |
Number of output features. |
.use_bias |
bool |
Whether a bias term is included. |
.dtype |
str |
Data type of the weight and bias tensors. |
Methods
| Method | Description |
|---|---|
.forward() |
Compute x @ W + b. |
Inherits all methods from Module: .parameters(), .submodules(), .to_device(), .summary(), .set_train_mode(), .set_eval_mode().