Independent research — quantitative machine learning for markets, and GPU systems / LLM-inference performance.
I'm an independent researcher and engineer working where quantitative machine learning meets high-performance GPU systems. I build end-to-end — raw market-data pipelines and self-supervised models through to low-level inference kernels — across consumer and datacenter hardware. My bias is toward rigorous, honestly-reported empirical work: leak-controlled evaluation, walk-forward validation, and negative results stated as plainly as positive ones. Selected write-ups below.
The 8B Student Beats the Pruned 120B: Distilling a Production Scorer onto One 16 GB GPU
The sequel to the expert-pruning paper answers its open problem, the 16 GB tier, with a different tool: distillation. Production had already generated the dataset for free; 15.8k stored gpt-oss-120b scoring outputs, rebuilt into exact (prompt, strict-JSON) pairs, QLoRA'd into Qwen3-8B in 14.3 hours on one RTX 4080 Super. On the frozen 311-prompt replay it beats the pruned 120b on every metric (kappa vs prod 0.642 against 0.508-0.538) at 5x the speed, and in live shadow it halves confidence error and cuts risk error 30%: exactly the fields pruning degraded most. Includes why a student can honestly out-score its own teacher, and why the pruned checkpoint was publishable but the distilled weights are not. Read →
Deleting 62.5% of gpt-oss-120b's Experts: Full-Model Quality on a Single 32 GB GPU
The first expert-pruned gpt-oss-120b checkpoint: 48 of 128 experts per layer, 26.4 GB on disk, 128k context in stock vLLM on one 32 GB card (CUDA and Intel XPU). On a frozen 311-prompt replay of reach.jobs's production resume/job scoring, it lands inside the full model's own self-consistency range (kappa vs prod 0.508-0.538 against the full model's 0.485-0.527) at 100% strict-JSON parse. The useful finding is the failure: prompt-only calibration systematically deletes the experts that write the output format, and the model fails by format collapse, not gradual decay. Generation-aware REAP calibration changes 60% of the keep-set and recovers parity. Weights, MXFP4-preserving pruner, and a 16 GB-friendly saliency runner are all open. Read →
Where Claude Code Tokens Actually Go — and a Small Open-Source Kit That Targets the Right 95%
Auditing my own week of Claude Code transcripts: 92.6% of input tokens are reused context served from the prompt cache — replicating the published 95.7% figure — while the tool output that popular "compressor" hooks shrink is a rounding error (JetBrains measured one such tool raising real cost 7.6%). token-saver is the levers that actually move the bill, packaged: a skill enforcing scoped reads and append-only context, haiku/sonnet subagents so mechanical work never runs at top-tier pricing, and a stdlib-only audit tool with cache-aware pricing (reads 0.1×, writes 1.25–2×) that reads the usage blocks Claude Code itself logs. Caching already saves me ~$3.1k/week; the remaining lever is model mix. Read →
The Decision That Wasn't: What the Free Transformer's Latent Actually Learns
Opening up the surviving latents from Paper 1 with per-bit information accounting, single-bit ablations, and a controlled steering experiment. The free-bits budget quantizes into a few soft channels (κ=0.5 → one of 16 bits; κ=1 → exactly three at ~0.33 bits each); a run that died and resurrected mid-training carries the same information in a diffuse 8-channel code — training history is written into the representation's structure. And the "decision before speaking" story fails three independent tests at 51M–124M: channels flip on 1–5-token timescales, correlate with nothing nameable, and pinning the latent steers nothing. What survives is a per-token side-channel — with a registered, falsifiable prediction for what must change at scale. Read →
Free Bits Aren't Free: An Independent Replication of the Free Transformer, with Error Bars
Nine months after FAIR published the Free Transformer — a GPT that samples a latent "decision" before it generates — no public independent replication existed. This is one, at 51M–124M parameters with 3 seeds on every claim, run on an RTX 5060, an RTX 4080 SUPER, and an Intel Arc Pro B70 (plausibly the first documented LLM pretraining on Arc B-series silicon). The architecture's logged loss flatters it by its full KL — honest accounting shows a ≈0.19-nat tax (~21% perplexity) whose marginal cost collapses near κ=1, with KL recovery improving 55%→69% from 51M→124M: a quantitative bridge to the paper's large-scale gains. Posterior collapse turns out to be a seed lottery, not a fate — two latents died mid-training and spontaneously resurrected — and one of our own headline findings dissolved into seed noise within 24 hours, which is precisely the point. Read →
A Bare-Metal, Homogeneous-Node Architecture for a Distributed Social Network
A shared-nothing reference architecture for running a Facebook/MySpace-class social network on a single commodity server SKU across two colocation facilities plus a lightweight witness site — every layer sharded and replicated across explicit failure domains (disk < node < rack < colo). This v0.2 revision folds a ZFS-managed spinning-disk capacity tier into the one node SKU: NVMe special-vdev / SLOG / L2ARC acceleration over a dense HDD pool lifts raw density ~19× (~15 TB → ~300 TB per box) while keeping one part number, with Ceph device classes tiering hot flash against cold erasure-coded disk. Covers the two-and-a-half-site quorum trick, TAO/Haystack-style graph and media serving, Maglev-style load balancing, hybrid feed fan-out, and capacity planning under colo loss. Read →
The Attention Variant Is the Only Variable: A Controlled Study of MLA vs MHA, MQA, and GQA at 124M Parameters
A controlled study on a from-scratch 124M GPT: hold the backbone, RoPE, optimizer, data, and token budget fixed and vary only the attention — MHA, MQA, GQA, and DeepSeek-V2 Multi-head Latent Attention (MLA). MLA compresses the KV cache 5.6× vs MHA (38.6 GB → 6.85 GB at a 128k-token context, where the cache — not the parameter count — is the memory wall) and, via weight absorption, overtakes MHA and GQA on decode throughput at long context. On the dev-scale quality sweep all four variants land within ~2.4% perplexity, so the cache savings come at negligible quality cost; the 124M headline run is in progress. Trainable end-to-end on one RTX 4080 Super. Read →
The Machine Analyst: Two LLM Fundamental-Valuation Bots, and the Problem of Proving Skill
Alpha Research Paper 9 — the machine as analyst, not researcher. Two bots value the entire S&P 500 every day: Vinebot (Damodaran intrinsic value, prices everything) and Buffybot (Buffett quality + owner-earnings, waits for the fat pitch). The defining discipline is anti-degeneracy — the LLM never outputs a price target, only clamped assumptions, and deterministic Python compiles them into value. Built on opposite philosophies, the two engines triangulate (Spearman 0.71, 85% sign agreement) and both judge mid-2026 richly priced. Yet no skill is claimed: an LLM analyst cannot be backtested — it has already read the future — so the only valid test is a multi-year forward one, which is pre-registered and now running live. Read →
On-Chain Liquidity and the Crypto Premia: An Autonomous Search, and an Honest Out-of-Window Death
Alpha Research Paper 8 — the first paper found by a machine. An autonomous LLM loop edits a strategy file, scores it through the program's rigor gate (which it cannot see or run), and iterates. Its most promising market-neutral lead — a stablecoin-flow overlay on the crypto premia (in-window OOS 1.18 vs 0.66, 5-day mechanism t=3.22) — died out-of-window: extending the data to 2020–2026 collapsed the mechanism to noise (t=0.48, PBO 0.11→0.51). A fit, not an edge; the automatic, honest kill is the deliverable. Read →
Synthesis: Do the Modest Edges Add Up? A Diversified Risk-Premium Book
The capstone of the Alpha Research series. The four surviving long-history premia (FX carry & value, equity quality, liquidity provision) are near-uncorrelated (all |ρ|<0.2), so a risk-parity book lifts an average 0.40 Sharpe to a combined 0.53 — above the QSPIX 0.41 live ceiling, a 1.33× diversification gain. Diversification is the only real free lunch — but the premia are decaying (book Sharpe 1.09 in the 2010s → 0.13 in the 2020s), so the honest forward expectation is ~0.3–0.5, not the backtest's double digits. Read →
FX Carry & Commodity Roll Yield — and What Six Papers of Honest Search Found
The finale of the Alpha Research series. FX carry is real but modest and decaying (0.64→0.2, crash-prone); FX value steadier (~0.46); energy roll yield has no net edge. The capstone tallies every edge tested across six papers: the survivors — crypto trend, FX value/carry, liquidity provision, funding carry, quality — cluster modestly around a 0.4 Sharpe, just above the floor. Every spectacular backtest alpha was an artifact; rigor was the edge. Read →
Liquidity Provision as Alpha: Execution, Order Flow, and the Limits of Microstructure
Alpha Research Paper 5, on 5.24M SPY order-book snapshots. Standalone market-making is structurally unavailable (the half-spread is 0.09 bp — a sub-0.1 bp, colocation-and-rebate game) and order-flow imbalance explains the present (R²=0.41) not the future (~0). But microstructure pays twice: a micro-price execution overlay cuts ~13% of slippage, and the vol-conditional liquidity-provision premium (reversal) earns Sharpe 0.42→1.16 as volatility rises. Read →
The Cost of Direction: A Net-of-Cost Audit of the Directional Anomaly Zoo
Alpha Research Paper 4 — a replication-crisis audit. The 212 published predictors of the Open-Source Asset Pricing dataset decay 53% out-of-sample (replicating McLean–Pontiff); in the last decade the median anomaly's Sharpe is 0.22 and only 3% are statistically strong; high turnover means realistic costs erase most of the rest. Among Fama–French factors only the market and quality (RMW) survive — low-turnover quality is the lone winner. Read →
Statistical Arbitrage in Crypto: An Honest Out-of-Sample Audit
Alpha Research Paper 3 — a clean honest negative. Daily PCA-residual mean-reversion on liquid coins has no edge even gross; the apparent hourly edge rises as liquidity falls (a stale-price artifact that dies at illiquid coins' real 50–100 bp spreads); and 210 in-sample-selected cointegration pairs lose out-of-sample. No net-of-cost stat-arb edge — rigorous cost modeling is the alpha. Read →
Crypto Carry: The Funding-Rate Cross-Section, Net of Costs and the Liquidation Tail
Alpha Research Paper 2. The funding-level harvest (the famous “Sharpe-6” carry) is an artifact of funding's near-zero volatility — net of costs it decays below cash and turns negative by 2026. But the cross-sectional carry factor (long-low / short-high funding, dollar-neutral) is the program's first real net-of-cost, out-of-sample-positive edge — OOS Sharpe ~0.4, deflated Sharpe 0.63, modest and tail-heavy but real. Read →
The Volatility Risk Premium, Cross-Asset: Is It Harvestable Net of Costs?
Opening paper of the new Alpha Research series. The volatility risk premium is real and statistically robust in both crypto (Deribit DVOL) and equities — BTC t=4.45, deflated Sharpe 0.88; equity t=4.22 over 35 years — yet it is not a harvestable net-of-cost edge: it halves or vanishes under realistic volatility measurement, has decayed to ~zero since 2024, and equity option-writing has earned no excess over buy-and-hold since 2010. A real premium ≠ a tradable edge. Read →
Beyond Price: Implied Volatility and Long-History Sentiment as Trading Signals
Two new free-data modalities for the JEPA-Trader project. Crypto implied volatility (Deribit DVOL) is the first signal in the project to beat its baseline — forecasting realized vol better than trailing vol and carrying a persistent +7.7-point risk premium — while 11 years of FinBERT-scored news sentiment dissolves the prior contrarian “bright spot.” Free-data predictability lives in volatility, not direction. Read →
Does JEPA Find Alpha? Self-Supervised Joint-Embedding Predictive Architectures on Financial Markets
A rigorous, leak-controlled two-part feasibility study. Part 1: JEPA across LOB microstructure, bars, news sentiment, and crypto funding. Part 2: a cross-sectional Graph-JEPA over the universe, a latent world-model with planning, and predictive-uncertainty signals (with RankMe / PITS controls). JEPA matches supervised learning with no labels — yet never beats a simple linear model, and no robust net-of-cost alpha survives walk-forward validation. Read →
TurboQuant + EAGLE3 on Arc Pro B70: six integration fixes and a fused-kernel correctness regression
Porting TurboQuant KV-cache quantization and EAGLE3 speculative decoding to Intel Xe2 (Battlemage): six integration fixes and a subtle fused-kernel correctness regression on Qwen3-30B. Read →
Three SYCL attempts on Arc B70: ESIMD, joint_matrix, and the gap to Triton
Three approaches to hand-writing fast attention kernels on Arc B70 — ESIMD, joint_matrix,
and where each falls short of Triton on the same hardware. Read →
Speculative decoding on Intel Arc: a failed SYCL PoC and a 2× Triton fix
A failed SYCL proof-of-concept for speculative decoding on Intel Arc, and the Triton-based fix that recovered a 2× speedup. Read →