Compressing a Sparse Table of Preferences

Botie’s rating table grows whenever a reader joins or a guide appears. Most reader-guide pairs have no rating, so storing a learned prediction independently for every pair would miss the shared structure.

Matrix factorisation gives each reader and item a short learned vector. Their dot product predicts a rating, and observed ratings teach both sets of vectors.

Reader vector: [1, 2]
Guide vector: [2, 1]
Prediction = (1 × 2) + (2 × 1) = 4
Observed rating: 5

Residual = 4 − 5 = −1
Reader gradient = −1 × [2, 1] = [−2, −1]

The reader and guide factors use the same coordinates. Change one factor and follow which candidate scores increase, decrease or remain unchanged.

Store one vector per reader in U and one per guide in V. Their matrix product with V transposed gives a full score table. Factor width is a modelling choice that controls capacity.

A rating model often also includes a global mean and reader/item biases. We omit those here to isolate the factor interaction, not because offsets are unimportant.

Latent coordinates can rotate without changing predictions, so they do not have unique meanings.

We know the ratings at some positions in the table. A mask identifies those training entries, and the squared-error objective compares predictions with the ratings there. Unobserved positions have no rating target. A regularization penalty can also discourage excessively large factors.

Clicks require a different interpretation: an absent click might mean the item was never seen. We need to choose an objective or confidence scheme appropriate to that feedback before reusing the factor model.

Prediction = dot([1,2], [2,1]) = 4
Observed rating = 5
Residual = 4 - 5 = -1

The residual is defined as prediction minus observation throughout this lesson.

For half squared error on one pair, the reader gradient is residual times the old item vector, and the item gradient is residual times the old reader vector. Make copies before updating either in place.

If the second update uses the already-changed first vector, it implements a different step from the simultaneous gradient calculation. Add regularisation gradients consistently if the objective includes them.

The coding exercise deliberately saves both old vectors so the dependency is visible.

import numpy as np
reader = np.array([1., 0.])
guide = np.array([2., 1.])
rating = 4.0
error = reader @ guide - rating
print(float(error))
print((error * guide).tolist())
print((error * reader).tolist())

-2.0
[-4.0, -2.0]
[-2.0, -0.0]

Low training error can reflect memorisation, especially with many factors and few ratings. Tune factor width and regularisation against held-out ratings or ranking tasks. Split by time when predicting future behaviour.

A brand-new item has no fitted factor. A content-based model or fallback is still needed. Learned factors do not solve missing exposure or biased feedback by themselves.

Botie’s workshop computes a score from supplied factors; fitting is shown in Python.

The factor tables begin with small nonidentical values. Repeatedly visit observed reader-guide pairs, calculate the current prediction, and update the two participating vectors. Save the old vectors while computing both gradients so one update does not unexpectedly use the other's new value.

Repeat over shuffled observed pairs for several passes, tracking held-out performance as well as observed-entry reconstruction. The unobserved cells are not zero targets. They simply do not supply explicit rating errors in this objective.

Before using a sparse real collection, test the training mechanism on a tiny known matrix. Here every entry is observed and the table is exactly rank one. That gives us a deliberately easy diagnostic for the update loop.

We use full-batch gradients from the same old factors. Regularisation is omitted for this mechanism check; it belongs in the later generalisation experiment.

import numpy as np
rating = np.array([[1.,2.],[2.,4.]])
U = np.array([[.4],[.7]])
V = np.array([[.5],[.8]])
initial = np.mean((U@V.T-rating)**2)
for _ in range(1000):
    error = U@V.T-rating
    dU = 2*error@V/rating.size
    dV = 2*error.T@U/rating.size
    U -= .05*dU
    V -= .05*dV
print(bool(np.mean((U@V.T-rating)**2) < initial))
print(np.round(U@V.T, 2).tolist())

True
[[1.0, 2.0], [2.0, 4.0]]

Implicit interactions such as opens tell us which guides were encountered and selected, but usually not why other guides were ignored. An unobserved pair may be unseen, irrelevant, unavailable or simply not yet chosen.

Implicit-feedback methods therefore need a declared confidence or negative-sampling scheme. Sampling a few unobserved candidates creates a training task whose distribution affects the learned scores. Those scores should not automatically be interpreted as probabilities of liking a guide.

A factor model may fit popular observed ratings while recommending guides already seen or failing new readers entirely. Apply eligibility rules and evaluate ranking on later interactions with a documented candidate set.

Regularisation, factor dimension and training duration should be selected on validation requests. New guides have no learned factor from interactions alone, so a content model or fallback remains necessary. Report that boundary instead of implying the factor table can describe an entity it has never fitted.