Four reproductions of representation-learning methods, organized into two sweeps. The shared discipline is the same across both: pretrain unsupervised, then evaluate the learned representations with a fixed linear-probing protocol, with no architecture-level hyperparameter tuning. A difference in the result is a difference in the architecture, not in the tuning effort.
Tabular SSL: MET, SCARF, VIME
Three foundational tabular self-supervised methods reproduced as separate PyTorch repos and evaluated under one shared linear-probing protocol. Tabular SSL is its own subfield (vision-style augmentations don’t apply), and reproducing the three end to end clarifies what actually transfers.
MET: Masked Encoding for Tabular data
A transformer-based masked autoencoder for tabular features. Reconstructs masked entries from visible context and includes the paper’s bounded adversarial training component.
SCARF: Self-supervised contrastive learning
Contrastive pretraining for tabular data using random feature corruption to generate augmented views. Sidesteps the absence of vision-style augmentations entirely.
VIME: Value Imputation and Mask Estimation
Self- and semi-supervised learning via a masked feature-imputation pretext task. Demonstrates that pretext-task pretraining transfers cleanly to tabular downstream tasks.
Autoencoders: architecture comparison on MNIST
A comparative study of autoencoder architectures collected in a single repo: deep, convolutional, denoising, variational, and self-supervised (SimSiam and a SimSiam-denoising hybrid), most following their original papers. All trained unsupervised on the 60,000-image MNIST training set, then evaluated with a linear classifier across label budgets from 10 to 8,000 examples. No hyperparameter tuning on the encoders or the probes, so the architecture is the only variable. Configuration runs through Hydra, with tracking in Weights & Biases.