Tensors as Linear Algebra
Tensors are elements of vector spaces with transformation rules — not just arrays of numbers.
You've been creating tensors and running operations on them since Module 1. But there's a deeper structure here that most tutorials skip. Tensors aren't just arrays — they're elements of vector spaces, and the operations you've been using are linear maps. Understanding this changes how you think about every layer in a neural network.
Learning Objectives
- ○Understand vector spaces and why tensors live inside them
- ○Identify basis vectors and linear independence in practical terms
- ○Perform matrix multiplication as composition of linear maps
- ○Connect CSS transform matrices to ML weight matrices
- ○Compute the rank of a matrix and understand what it means for data
From Arrays to Vector Spaces
In frontend development, you've used CSS transforms without thinking twice:
transform: matrix(1, 0, 0, 1, 50, 100);
That 2D transform matrix lives in a vector space. Every CSS transform you've ever written is a linear map — it takes a coordinate and produces a new coordinate, and it respects addition and scaling. ML weight matrices work identically.
Frontend
CSS Transform
transform: matrix(a, b, c, d, tx, ty)Machine Learning
Linear Map
const result = tf.matMul(transformMatrix, vector)A vector space is a set of objects (vectors) where you can add any two together and multiply any one by a scalar, and the results stay in the set. JavaScript arrays of fixed length form a vector space — you can add [1, 2, 3] and [4, 5, 6] element-wise to get [5, 7, 9], and that result is still a 3-element array.
Basis Vectors and Coordinates
Every vector in a space can be written as a combination of basis vectors. In 2D, the standard basis is:
- e1 = [1, 0] — the x-axis direction
- e2 = [0, 1] — the y-axis direction
The point [3, 7] is really 3 * e1 + 7 * e2. This is exactly how coordinate systems work in CSS — the browser defines a basis (x-right, y-down), and every position is a linear combination of those directions.
import * as tf from '@tensorflow/tfjs';
// Standard basis vectors in 2D
const e1 = tf.tensor1d([1, 0]);
const e2 = tf.tensor1d([0, 1]);
// Any 2D vector is a linear combination of basis vectors
const point = tf.add(e1.mul(3), e2.mul(7));
console.log(await point.array()); // [3, 7]
// A weight matrix transforms basis vectors to new positions
// This is exactly what a CSS transform matrix does
const W = tf.tensor2d([
[2, -1], // Where e1 lands after transformation
[1, 3] // Where e2 lands after transformation
]);
// Transform our point
const transformed = tf.matMul(W, point.reshape([2, 1]));
console.log(await transformed.array());
// [[-1], [24]] — the point [3,7] mapped through the linear transformationLinear Independence and Rank
Vectors are linearly independent if none can be written as a combination of the others. The rank of a matrix tells you how many independent directions it preserves. If a weight matrix has low rank, it's collapsing your data into fewer dimensions — information is being lost.
import * as tf from '@tensorflow/tfjs';
// Full-rank matrix: preserves both dimensions
const fullRank = tf.tensor2d([
[1, 0],
[0, 1]
]);
// Rank-1 matrix: collapses 2D data onto a line
const rank1 = tf.tensor2d([
[1, 2],
[2, 4] // Row 2 = 2 * Row 1 — not independent!
]);
// Apply rank-1 matrix to different points
const points = tf.tensor2d([[1, 0], [0, 1], [1, 1]]);
const collapsed = tf.matMul(points, rank1.transpose());
console.log(await collapsed.array());
// All outputs land on the same line — dimension reduced
// In ML, this is why weight initialization matters:
// a rank-deficient weight matrix can't learn distinct features
// Compute approximate rank via SVD
const svd = tf.linalg.svd(rank1);
const singularValues = await svd[1].array();
console.log('Singular values:', singularValues);
// One value near zero means rank 1Matrix Multiplication as Composition
When you chain CSS transforms, the browser multiplies the matrices together. Neural network layers work the same way — each layer's weight matrix is a linear map, and stacking layers composes those maps.
import * as tf from '@tensorflow/tfjs';
// Two transformations (like two neural network layers)
const rotate = tf.tensor2d([
[0, -1],
[1, 0] // 90-degree rotation
]);
const scale = tf.tensor2d([
[2, 0],
[0, 3] // Scale x by 2, y by 3
]);
// Composing: scale THEN rotate = single matrix
const combined = tf.matMul(rotate, scale);
console.log(await combined.array());
// [[0, -3], [2, 0]]
// This is why deep networks are powerful:
// each layer adds a new linear map (plus nonlinearity)
// The composition can represent incredibly complex transforms
// Without activation functions, N layers collapse to 1 matrix:
const layer1 = tf.randomNormal([4, 3]);
const layer2 = tf.randomNormal([3, 4]);
const collapsed = tf.matMul(layer2, layer1);
// collapsed is 3x3 — two layers = one linear map
// This is why we NEED nonlinear activations between layersChallenge
Put your linear algebra knowledge to work by implementing core vector space operations.
Exercise
Linear Algebra Operations
Implement three functions: (1) `linearCombination` that takes two vectors and two scalar coefficients and returns their linear combination (a*v1 + b*v2), (2) `matMulCompose` that takes two 2x2 transformation matrices and returns their composition via matrix multiplication, and (3) `isLinearlyIndependent` that takes two 2D vectors and returns true if they are linearly independent (their determinant is non-zero, using an epsilon of 1e-6).
Key Takeaways
- ✓Tensors live in vector spaces — they follow rules of addition and scalar multiplication
- ✓Basis vectors define coordinate systems; every vector is a linear combination of basis vectors
- ✓Matrix rank tells you how many dimensions a transformation preserves — low rank means information loss
- ✓Matrix multiplication composes linear maps, which is exactly how stacking neural network layers works
- ✓Without nonlinear activations, any number of layers collapses to a single matrix multiplication