Chris Leung — Essays

A Rust static site generator

2026-07-29T00:00:00+00:00

I got nerd-sniped by Julian Schrittwieser's blog post on writing a static site generator in Rust - here it is! Most of the wins came from optimising images from JPG to WEBP then finally to AVIF, lazy-loading, and opt-in MathJax. Writing the site generator was fun apart from handling the edge cases in Markdown. Instead of using Jinja for template parsing, I directly wrote some Rust functions to return formatted HTML strings - giving compiler errors instead of runtime template-syntax mistakes from Jinja.

Overall, I'm pretty happy: the frontpage now loads in 78KB (with the pure HTML/CSS coming around 20KB!

Cell atlases as a phenotype

2026-07-28T00:00:00+00:00

With the advent of single-cell RNA sequencing (scRNA-seq), some contend that the phenotype of a cell can be abstracted as a "bag of RNA" (Quake, 2021). Differential gene expression analysis then provides contrast and allows us to classify cells. Projects such as the Human Cell Atlas (HCA) aggregate single-cell and spatial data from healthy human cells, with the aim of connecting individual cell types to their collective functions and interactions (Amit et al., 2024). We analyse these claims in these ways:

Does scRNA-seq provide sufficiently rich, reliable, and interpretable data to describe cell phenotypes?
What do cell atlases tell us about phenotype and function?
If cell atlases do provide a "molecular definition of cell type" (Quake, 2022), what might be missing?

Step 1 in building a cell atlas: single cell transcriptomics

Cell atlases are primarily built on scRNA-seq experiments. The readout provides a high-dimensional vector in RNA-space, which is then analysed by dimensionality reduction. We will examine the biases introduced in the experimental pipeline and consider the effects on the fidelity/utility of the RNA phenotypic readout.

A significant fraction of the RNA collected from a cell consists of ribosomal RNA, so mRNA is enriched either by poly-A or ribosomal-specific probes. The latter is particularly prone to introducing bias by off-target effects on similar sequences. As the average length of mRNAs exceeds the read length of high-throughput sequencing, the remaining RNA is fragmented, deeply sampled, then reconstructed in silico aided by unique molecular identifiers (UMI). These tags also serve to eliminate biases in PCR amplification in read counting. However, (Sena et al., 2018) showed that reads with identical UMIs sometimes map to slightly shifted but similar coordinates, possibly due to PCR stuttering. This results in an overestimation of certain genes' expression. In addition, low-expression genes often suffer from inadequate coverage, which is more efficiently improved by biological replicates rather than deeper sequencing. Quality control during data analysis can account for some of the biases (e.g. read duplication rate), but these examples indicate that experimental biases lead to a biased and lossy abstraction of the RNA content of a cell.

The exact choices taken in scRNA-seq experiments also impacts our ability to integrate this data usefully. Ongoing questions include the size of the cDNA fragments, accounting for inter-cell variability due to technical difficulties and the stochasticity of gene expression, choosing between 5' and 3' reads of the transcriptome, and dropouts in library preparation, as reviewed in (Jovic et al., 2022). Moreover, we can capture a cell at various stages of its development, depending on its cell cycle and stress state. Deep learning methods such as deepMNN (Zou et al., 2021) can counteract some of these batch effects, and the cell cycle can be accounted for by tools such as CellCycleScoring in Seurat, but the methodological differences complicate cell atlas integration and phenotypic comparisons between datasets.

Step 2: building a cell atlas

The scope of a cell atlas is subject to curation and validation. Some focus on single organs or entire, healthy organisms, whilst others sample for different stages of development or disease (Hrovatin et al., 2025). Setting aside the challenges of integrating cell atlases effectively, we will consider the phenotypic utility of cell atlases.

Differential gene expression analysis relies on dimensional reduction to provide a biological signal. Genes that are significantly variable in their expression within an atlas provide a strong signal for analysis. After dimensional reduction (e.g. by PCA, t-SNE or UMAP), clusters can then be identified.

In some cases, cell atlas analyses have contributed to our understanding of anatomy and physiology. An atlas of the Drosophila melanogaster olfactory projection neurons was produced in (Li et al., 2017). It is known that the classes of olfactory receptor neurons form an injective mapping to the classes of projection neurons (PNs) in the olfactory glomeruli; each class differs spatially and by the subset of olfactory receptor genes they express. Differential gene expression could be used to separate out astrocytes from olfactory PNs by labelling the latter with a GH146-GAL4 driver, and also identified new genes that were exclusively expressed in astrocytes. Unsupervised learning algorithms then separated the cells into PN classes by variations in gene expression within each cluster. The combination of new and existing genetic markers in vivo enabled them to then map these data clusters onto the corresponding olfactory glomeruli. Moreover, a developmental time-course cell atlas analysis showed that the transcriptomes of similar classes differ the most during circuit assembly. Within each lineage, a combinatorial transcription factor code enables precise dendrite and axon targeting. These experiments demonstrate the connection between transcriptional regulation and phenotypic effects.

Before cell atlases and scRNA-seq, studying the effects of disease on the genome was often restricted to mutations and limited the scope to regions of interest, e.g. BRCA-1 in breast and ovarian cancer. Cell atlases more fully capture the heterogeneity and interactions of cell phenotypes and identify gene expression programs with less prior knowledge. For instance, diffuse large B cell lymphoma (DLBCL) was studied in (Ye et al., 2022). Known markers were used to link clusters to cell types, then the transcriptome profiles of the malignant B cells were analysed to extract expression programs such as MAPK, mitochondrial RNA, glycolysis/proteasome, and cell cycle genes that were upregulated. Cell atlases also capture the interactions within the diverse tumour microenvironment, which cannot be obtained from genome sequencing alone. Using CellPhoneDB, infiltrating T cells were shown to interact with malignant B cells through CD70-CD27 binding and with regulatory T cells. This is a stronger indicator of phenotype than the genome alone: with this validation, CAR-T or BiTE immunotherapies can be used to target the tumour.

The two examples here show that cell atlases, despite the biases in their construction, provide actionable phenotypic information that is far harder to extract from the genome itself. Some aspects of transcriptional regulation can perhaps be gleaned from ATAC-seq and large-scale rearrangements in Hi-C, or from sequence alterations, but the phenotypic readout from RNA-seq is far richer and interpretable.

Multi-omics and the unknown unknowns

As mentioned above, the RNA expression vector is a useful abstraction of the phenotype caused by the genome. We will now consider the parts left out in this abstraction.

In well-understood gene regulatory networks (GRNs), genome-wide data from ChIP, ATAC, and histone marks combined with labelled cis-regulatory elements and transcription factor motifs have been used to guide experimental ablations of the network (as reviewed in (Zeitlinger et al., 2025)). However, this is limited by the selection of motifs and regions. Analysing cell atlases' expression of transcription factors and target genes can be used to construct GRNs as well. Moreover, comparing diseased and healthy samples enables us to understand how the GRN is rewired by disease or by cell state. The convergence of DNA genomics (e.g. single-cell ATAC) with cell atlas methods in understanding GRNs is an opportunity to validate phenotypic information obtained from atlases alone. Atlases tend to focus on gene expression rather than mutations and regulatory elements, and do not give a readout of how different transcription factor motifs interact: this information is only partially inferred from co-expressional patterns. A deeper understanding of the phenotype of the genome can be obtained from multi-modal approaches incorporating cell atlas scRNA-seq data with other modalities, e.g. sequence information, Hi-C and chromatin accessibility. Graph transformers such as scHiGeX (Shrestha et al., 2025) provide a computational framework for inferring cell atlas-style gene expression profiles from other modalities. Other multi-modal prediction frameworks such as AlphaGenome (Avsec et al., 2025) also enable phenotypic predictions from sequence alone. However, the combination of -omics technologies that will provide a good approximation of the genome's phenotype remains up for debate.

Conclusions

The contention that cell atlases should serve as a phenotypic companion to the genome has been experimentally validated. There are still concerns regarding biases: the resultant clustering maps well to biological function and has proven to be useful in labelling and classification tasks. Moreover, cell-cell interaction phenotypes can be inferred from atlases, which has enabled disease geneticists to go beyond genome sequence in designing targeted therapies. However, inferring the molecular mechanisms that contribute to the phenotype from correlative methods (including cell atlases) remains elusive. Whether computational methods can tell us anything about the mechanisms underlying the phenotypic output remains an open question.

Bibliography

Amit, I., Ardlie, K., Arzuaga, F., Awandare, G., Bader, G., Bernier, A., Carninci, P., Donnelly, S., Eils, R., Forrest, A.R.R., Greely, H.T., Guigo, R., Hacohen, N., Haniffa, M., Kirby, E.S., Knoppers, B.M., Kriegstein, A., Lein, E.S., Linnarsson, S., Majumder, P.P., Merad, M., Meyer, K., Mhlanga, M.M., Nolan, G., Ntusi, N.A.B., Pe'er, D., Prabhakar, S., Raven-Adams, M., Regev, A., Rozenblatt-Rosen, O., Saha, S., Saltzman, A., Shalek, A.K., Shin, J.W., Stunnenberg, H., Teichmann, S.A., Tickle, T., Villani, A.-C., Wells, C., Wold, B., Yang, H., Zhuang, X., 2024. The commitment of the human cell atlas to humanity. Nat. Commun. 15, 10019. https://doi.org/10.1038/s41467-024-54306-x
Avsec, Ž., Latysheva, N., Cheng, J., Novati, G., Taylor, K.R., Ward, T., Bycroft, C., Nicolaisen, L., Arvaniti, E., Pan, J., Thomas, R., Dutordoir, V., Perino, M., De, S., Karollus, A., Gayoso, A., Sargeant, T., Mottram, A., Wong, L.H., Drotár, P., Kosiorek, A., Senior, A., Tanburn, R., Applebaum, T., Basu, S., Hassabis, D., Kohli, P., 2025. AlphaGenome: advancing regulatory variant effect prediction with a unified DNA sequence model. https://doi.org/10.1101/2025.06.25.661532
Hrovatin, K., Sikkema, L., Shitov, V.A., Heimberg, G., Shulman, M., Oliver, A.J., Mueller, M.F., Ibarra, I.L., Wang, H., Ramírez-Suástegui, C., He, P., Schaar, A.C., Teichmann, S.A., Theis, F.J., Luecken, M.D., 2025. Considerations for building and using integrated single-cell atlases. Nat. Methods 22, 41–57. https://doi.org/10.1038/s41592-024-02532-y
Jovic, D., Liang, X., Zeng, H., Lin, L., Xu, F., Luo, Y., 2022. Single-cell RNA sequencing technologies and applications: A brief overview. Clin. Transl. Med. 12, e694. https://doi.org/10.1002/ctm2.694
Li, H., Horns, F., Wu, B., Xie, Q., Li, J., Li, T., Luginbuhl, D.J., Quake, S.R., Luo, L., 2017. Classifying Drosophila Olfactory Projection Neuron Subtypes by Single-Cell RNA Sequencing. Cell 171, 1206-1220.e22. https://doi.org/10.1016/j.cell.2017.10.019
Quake, S.R., 2022. A decade of molecular cell atlases. Trends Genet. 38, 805–810. https://doi.org/10.1016/j.tig.2022.01.004
Quake, S.R., 2021. The cell as a bag of RNA. Trends Genet. 37, 1064–1068. https://doi.org/10.1016/j.tig.2021.08.003
Sena, J.A., Galotto, G., Devitt, N.P., Connick, M.C., Jacobi, J.L., Umale, P.E., Vidali, L., Bell, C.J., 2018. Unique Molecular Identifiers reveal a novel sequencing artefact with implications for RNA-Seq based gene expression analysis. Sci. Rep. 8, 13121. https://doi.org/10.1038/s41598-018-31064-7
Shrestha, B., Siciliano, A.J., Zhu, H., Liu, T., Wang, Z., 2025. scHiGex: predicting single-cell gene expression based on single-cell Hi-C data. NAR Genomics Bioinforma. 7, lqaf002. https://doi.org/10.1093/nargab/lqaf002
Ye, X., Wang, L., Nie, M., Wang, Y., Dong, S., Ren, W., Li, G., Li, Z.-M., Wu, K., Pan-Hammarström, Q., 2022. A single-cell atlas of diffuse large B cell lymphoma. Cell Rep. 39, 110713. https://doi.org/10.1016/j.celrep.2022.110713
Zeitlinger, J., Roy, S., Ay, F., Mathelier, A., Medina-Rivera, A., Mahony, S., Sinha, S., Ernst, J., 2025. Perspective on recent developments and challenges in regulatory and systems genomics. Bioinforma. Adv. 5, vbaf106. https://doi.org/10.1093/bioadv/vbaf106
Zou, B., Zhang, T., Zhou, R., Jiang, X., Yang, H., Jin, X., Bai, Y., 2021. deepMNN: Deep Learning-Based Single-Cell RNA Sequencing Data Batch Correction Using Mutual Nearest Neighbors. Front. Genet. 12. https://doi.org/10.3389/fgene.2021.708981

Mendelian randomisation

2026-07-18T00:00:00+00:00

How do you know whether smoking causes lung cancer? You could sample smokers and non-smokers, and look for correlations. Yet famously, correlation is not causation! Consider coffee drinking. Early on in studies of lung cancer, people sfound a correlation between coffee drinking and lung cancer. But this simply reflected the statistic that coffee drinkers are more likely to smoke. One could imgaine that carrying a lighter is correlates with lung cancer too.

Correlations could also be spurious: it depends on who you're sampling!

Why we need better methods

How do you know whether smoking causes cancer? A simple approach would be to sample a group of people to look for correlations. Yet famously, correlation is not causation, and even worse, it could be spurious.

Facetiously, what

Three years of Cambridge and twenty-six ideas

2026-06-24T00:00:00+00:00

Another year at Cambridge has drawn to a close, so I've decided to come up with 10 paragraphs and 16 one-liners (making 26 (!) points) to summarise what I've learned so far.

Academic

Phenomenology and mechanism

A concept that came up a lot in my study of genetics was the conversion of phenomenology to mechanism. Phenomenology, in the Husserl and Heidegger tradition, is the idea of first-person lived experiences and consciousness: in science, I recast this as using abstract mathematical models to describe and predict observable experimental data. A mechanism a physicochemical, biological process that creates the traits we describe using phenomenology. The outstanding success of quantitative genetics did not require our molecular understanding of genes, and phenomenological models still pervade systems biology. Crucially, a mechanism isn't required for us to medically/biologically act on the evidence.

The exploration-exploitation dilemma

This perhaps bridges both the academic and personal aspects of my life. In machine learning, we have the idea of the multi-armed bandit: if you have a row of slot machines, each with unknown payoffs (of which estimates you can improve in a Bayesian way), how many times do you play each machine, and in what order? Exploration becomes more difficult as rewards become sparser, less reliable, or less observable. I have more to say on this topic mathematically, and perhaps this will be another article.

All your models are wrong, and yours are useless

I attended a talk given by Florian Markowetz to Cancer Research UK Cambridge Institute students, and he raised the point that clinical prediction models are uselss unless they're impactful for patients. This got me thinking about the role of modelling in science: when a model has great explanatory power (e.g. Maxwell's electrodynamical equations and the symmetry-adapted linear combinations in chemistry), they do have an impact. Yet in biology, where populations are stratified, variation isn't well characterised, and the branching tree of individually justifiable data analysis choices is practically infinite, modelling has found far less success than in other fields. We can make the tools accessible (e.g. Mendelian Randomisation), but this information can be benignly harmful - when people make decisions based off of faulty analysis, the medical and financial consequences are immense. Understanding when a model concretely adds to the evidence is so important.

Why we haven't solved biology so far

Before I came across the idea of gene ontology (GO) annotation in my second year, I naively thought about the idea of writing down a structured grammar for biology, in order to reason about pathways without having to memorise all of the finicky 3/4 letter acronyms. The central idea of GO annotation is that everything has a molecular function, a biological process, and a cellular component, and annotating gene products with a hierarchical set of GO terms allows computational tools to trace biological pathways in a precise way.

Yet in the Tinbergen tradition, this is a static, proximate view: it doesn't tell us anything about the dynamics of a biological system, nor does it give us the ultimate reason why something exists. This obviously wasn't a goal of Ashburner's GO project, but it did make me think a lot about the promise of dynamical systems theory, when applied to entire cells. As always, the issue is that we don't have measurable latent variables!

Neuroscience and imagination

Memory replay in the hippocampus and in other motor functions continues to fascinate me: is there a form of simulated annealing that helps us get out of maladaptive states in the brain? There is some evidence pointing towards this in mouse hippocampus and in model motor systems, but I'm interested in the implications for machine learning: perhaps we can go beyond backpropagation, and follow the Stanley and Lehman PicBreeder-style approach, so that representations are disentangled and more interpretable. I suspect this topic will keep me entertained for a very long time...

Personal

Will be significantly shorter, to keep things light!

Health

Really the pillar of my happiness: that doesn't mean I track everything (I don't have the patience/brain capacity to do so), but staying fit and healthy has helped me so much this year.

People

It's very easy to feel like you've found your people, and then doubt that feeling. Sometimes it's worth taking the leap, and not doubting.

Just do things?

I've run into quite a few good opportunities by just putting myself out there, and taking rejection quite lightly. Something that I'll keep improving at.

Music

It's been such a joy playing with other people again, and I've planned more musical sidequests next year!

Time

This is the time where I'll have the fewest responsibilities ever. Whilst this doesn't mean that one should be irresponsible, one should take advantage of this abundance: see that friend, spend those hours watching the sun set, chat late into the night. Because this era, too, will end.

Quickfire round

When life gets you down, have some fruit
Wear sunscreen and don't apply it in spots
Everything can be a trip hazard if you're sufficiently unskilled
It's okay to be dilettantish in your pursuits as long as you achieve depth in them and cross-fertilise ideas
Developmental biology is really really hard
13.1 miles is the Pareto frontier in the pain-enjoyment tradeoff
Play music with your friends more frequently
Swimming is the most underrated form of cardio
Don't trade sleep for marginal increments in work hours
What is your question?
Don't work yourself to the bone: you (and the people who remember you afterwards) will regret it
Smell the flowers
Play some more games when you and your friends are bored
Eat the frog
Reject things which drain you
It's about the life in your years, not the years in your life

Part of a 15 minute writing exercise

On mentors

2026-04-07T00:00:00+00:00

Recent events have prompted me to reflect on the mentors I've had the privilege of working with, and how they've shaped how I think.

Mentor A was distant, if not a little intimidating. Yet I found their output of ideas exhilirating: they really exemplified the power of the quiet mind.

I still work closely with Mentor B. Mentor B is a true polymath: everything falls within the realm of the Glass Bead Game, as they might say. From our interactions, I really began to see what "generosity of thought" is: when you've got so many ideas in your head and can hold forth on a whole range of ideas, you can help others with problem selection.

Mentors C, D, and E were wonderful as well. They provided strategic direction to me, an early-stage, enthusiastic undergraduate, and gave my ideas the time and space to thrive. I might not have been the most present student, but I hope I had some interesting ideas.

What is the point of this exercise? I think as a student I'm fairly impressionable. But in retrospect, I took a different subset of ideas from each. From A, I learned how to think strategically: how do you manage a team, and how do you switch from exploration to exploitation. From B, I learned how to think across the academic silos found in academia. From C, D, and E, I learned how to conduct rigorous research, and push projects to completion.

Some of these mentors are composites: A certainly is one. But each has left their own, distinct mark on how I view the world.

Consolidative memory replay

2026-04-03T00:00:00+00:00

I had to write a research proposal recently, and I came across this really interesting problem in the exploration phase.

There are certain experimental systems which are very amenable to damage. A spatial example is the mouse hippocampus, which encodes place fields - essentially, sets of neurons that fire when recognising a certain part of space. The HVC in zebra finches is a nucleus that encodes temporally-associated memories.

In both cases, you can damage almost half of the excitatory neurons, and the mice/songbirds will still recover their behaviour within days or weeks. This is remarkable in itself, but there is also a temporal dynamic to it. In songbirds, we observe steady improvement during the say, spontaneous neuronal activity during the night, and backsliding during the day.

Now, there are a few possibilities: this may be explained by a Hebbian mechanism, or perhaps homeostasis. But perhaps a more interesting line of inquiry is the idea of consolidative memory replay: how do you know whether you've progressed in the right direction or not? Perhaps spontaneous neuronal activation during sleep acts as a simulated annealing process.

Simulated annealing derives from a metallurgical metaphor, where slow cooling represents slowly increasing the acceptance threshold as we explore solution space for the global optimum.

s=s_0
for k in range (0, k_max):
	T ← temperature(1-(k+1)/k_max)
	s_new ← neighbour_random(s)
	if P(E(s), E(S_new), T) >= Rand(0,1):
		s ← s_new
return s

Perhaps memory replay acts as a way to anneal out of undesired learned behaviours after circuit perturbation. Given that these behaviours can be recovered without practice, this could be an intriguing way to explain why brain circuits are so resilient.

Physics-informed neural networks 1

2026-04-02T00:00:00+00:00

Overfitting and generalisation are perpetual problems in neural network training. One thing we can do is regularisation, where we add $L(\textrm{regularisation})=\lambda\lVert \theta \rVert$.

A physics-informed neural network (PINN) can simply guide the shape of the neural network - a regularising prior on a universal function approximator. So we can set Loss = Data Loss + Physics-informed Loss. If we pick the function $g:X\rightarrow Y$ where the neural network $f:X\rightarrow Y$ is parameterised by $\theta$, we can use $g(x,f(x\mid\theta))$ weighted by $\lambda$ as the physics-informed loss.

If we're modelling a physical system, you might call this cheating: for me, it's just a really strong prior on what we expect our functions to look like. Perhaps you're working with some projectile data, but you've forgotten which planet you fired your projectile on: set $g$ to be your differentiable parameter, and let backpropagation find your solution!

I will put together a demo when I find time - perhaps after Monday.

Neurotechnology

2026-04-01T00:00:00+00:00

I've travelled from Cambridge, England to Cambridge, Massachusetts to spend half a week with some folks who are building neurotech for a wide range of applications.

Whilst I won't/can't go into the specifics, a few lessons I feel I've learned:

Questioning the status quo is almost always worthwhile. Pareto gains compound by definition, and there's so much about pre-paradigm fields (e.g. neuroscience) which can be improved upon, given a little out-of-the-box thinking.
The notion of doing good doesn't just include donating to the causes which effectively alleviate the world's problems. If you're in a position to contribute to a problem that you deeply care about, it may be worth pursuing. Some might argue that you have a moral obligation to work on the problems that you can solve, but this is something that I'm still thinking about.
Iterate on the things that matter most: if you can remove the externalities that prevent you from accomplishing something that matters to the mission, the rest follows.

I've been learning a lot of maths, engineering, and neuroscience. Peering into the glymphatic system has been fascinating, and it's incredible how we can develop readouts of their activity by picking the correct abstraction.

My experiences here so far have also strengthened my belief that the silo-building of academic specialisation is actively detrimental. Whilst the self-similarity of different dynamical systems hits a limit at some point, having a large scientific toolkit to wield is almost always helpful.

I've also learned a lot about operations and scaling: they're not just buzzwords...

People and research taste

2026-03-30T00:00:00+00:00

Something I've been thinking about recently is the development of research taste. I sometimes consider research taste as a filter problem. This allows us to filter ideas into four categories:

True negatives: ideas that we expect, when pursued, to be utterly incosequential or actively harmful towards our research agenda.
False negatives: ideas that are worth exploring in service of our research agenda, but we choose not to pursue.
False positives: dead ends that we run into, but do not directly contribute towards our research agenda.
True positives: ideas that are worth exploring in service of our research agenda, which we choose to pursue.

Setting aside the definition of a "research agenda" for now (to be addressed in a later post!), do ideas cleanly fall into these categories?

Imagine you're a theoretical physicist establishing a research agenda in epidemiological dynamics. We can likely assume that that Lie algebras and group theory do not contribute to epidemiology relevant to public health: we can confidently classify these as true negatives.

Perhaps quantum electrodynamics is also irrelevant: after all, how do perturbations and Feynman diagrams factor into virus transmission? But wait! The initial stages of an outbreak determine whether perturbations from a healthy baseline balloon into epidemics. Perhaps perturbation theory can be applied to virus dynamics. And indeed it is.

Here, the distinction between a true and false negative becomes blurred. If one observes an underlying mathematical similarity between two disparate ideas, the cost of modelling this thought is minimal. But how do we spot these patterns of thought?

Daniel Kahneman's Thinking, fast and slow breaks down reasoning into System 1 and System 2. System 1 is intuitive: we instinctively know that 8 times 2 equals 16. Yet System 2 is analytical: most people don't know off the top of their head what 89 times 32 is, but the procedural knowledge of multiplying by the tens and units, carrying the digits, and summing is there.

In an analogous manner, good research taste comes from a good balance of Systems 1 and 2. Intuition should provide a general filter for true negatives. As a student or junior research, this intuition is shaped by your supervisors, advisors, professors, and mentors. During this stage, system 1 can be pulled in many directions, and the student has to discover what filter they're happy with.

In my book, System 2 is built through the Popperian scientific method of conjecture and refutation. We begin to glimpse the superstructure of science, and choose the methods we employ to find new hypotheses and verify them.

Therefore, the development of good research taste consists of a constructing a reliable filter, and picking the right set of tools to investigate the questions that bother you. Most importantly, neither is developed in isolation: Abraham Flexner, in his essay the usefulness of useless knowledge, traces the development of probability theory alongside an attempted rationalisation of gambling. Perhaps your tools aren't suitable for the question at hand, but you can be fairly sure that someone else's tools might be correct.

As a student exploring the intersection of industrial and academic research, this raises the question: how do you maximise this surface of serendipity? My thoughts are as follows:

Find people whose toolkits are relatively different. Become an expert in the ones that interest you, and read widely: the right problem will appear!
People advance up the career/academic ladder because of good taste: their mentorship can be incredibly valuable, and completely reshape how you view a problem. This recently happened to me in viral genetics.
Maintain a research agenda: this prevents unnoticed mission drift, but allows your objectives to evolve over time.

(This is the beginning of a daily 12-minute writing experiment. I often agonise over the precise wording I want to employ in my writing. Whilst this is an important skill, I hope this experiment helps me overcome this anxiety!)

Toy models of superposition

2026-01-06T00:00:00+00:00

Why don't we get grandmother neurons - neurons corresponding to a feature?

The paper uses small ReLUs with sparse input features to see how superposition occurs. This is when the number of represented features exceeds the number of dimensions.

Paper

Notes on: appropriateness and generative artificial intelligence

2025-12-23T00:00:00+00:00

The paper's challenge is to lay down a definition of appropriateness: how should an individual act in an environment? In this paper, Leibo et al. start from the perspective that modelling the behaviour of AIs and humans is mutually instructive.

Introduction

Fundamentally, how can we live together?
How do humans make decisions? The paper focuses on the mechanism of predictive pattern completion.
Facets of appropriateness: context dependence, arbitrariness, automaticity, dynamism, its relation to sanctioning.
How does appropriateness relate to the alignment problem? We aren't working with tabula rasa RL systems, so the frame problem is mostly irrelevant in this context.
We collectively have a "thick" morality (Walzer 1994). It's difficult to distinguish between a hypothetical "core" and the "peripheral" aspects. Instead of trying to identify the "thin" rational core, it is perhaps more productive to understand how we can live together anyway.
Instead we tend to think about epistemic norms in communities. We might not be happy with a long string of conditionals - it allows self-serving, confirmatory biases to seep into an argument.
We want the conditions for collective flourishing.

Contextualisation via specialisation

Specialisation is needed for AI appropriateness, not capability? Does the curse of dimensionality apply to the sampling problem?
Technical capabilities of LLMs are entirely unaffected by choice of interface. But this feature changes appropriateness requirements.

Interfaces

Polycentric governance: the mechanism of sanctioning / feedback buffers cooperativity.
Hierarchical governance.

Stylised facts to be explained by this account of appropriateness

Context dependence: situation, role and identity, culture,
Arbitrariness: frequency of occurrence has a large effect on our preferences. Historical contingence.
Automaticity.

Paper

On scale-free intelligence

2025-12-23T00:00:00+00:00

Paper

Some brief notes and thoughts on intelligence and agency, taken from Richard Ngo's essay on intelligent agency (Paper). I do not claim originality.

First, we define an agent as an entity that develops an understanding of the world, and acts to exert an influence on the world. How, then, can this agent act intelligently?

The universal intelligence measure

Legg and Hutter's definition of universal intelligence from 2007 reads as follows:

$$\Upsilon(\pi)=\sum_{\mu\in E}2^{-K(\mu)}V^{\pi}_{\mu}$$

Note that the universal intelligence measure, $\Upsilon$, satisfies the invariance properties of Kolmogorov complexity. The underlying computational architecture does not matter: we could encode the tasks in a Scratch-based universal Turing machine or the 'human coding language', to similar effect. Moreover, the hierarchy of problems in the scale-free architecture has scale-free properties.

Ngo: two current theories on intelligent agency

Expected utility maximisation (EUM)

Search and planning enable an agent to discover strategies that maximise utility in expectation. Ngo points out that EUM handles goals and beliefs in isolation, and develops heuristics through experience. He suggests that theoretical progress in deep reinforcement learning is too broad and slow to generalise the theory of EUM to describe sequential decision making.

Active inference

This idea comes from neuroscience. Imagine that the brain is a hierarchical network. From the top downwards, each level tries to predict the behaviour of the level below it, and the bottom layer predicts sensory inputs. This model is a good analogy for both artificial neural nets and biological vision. The strength of the abstract representation at the top can disseminate throughout the network: think about how illusions clearly snap into focus when you spot the trick.

In the same line of thought, actions are formulated as strong predictions that override our default, resting states. Instead of treating goals separately, consider active inference. There can be a gap between our beliefs and our observations. If the cost of changing future observations is lower than the cost to update conflicting beliefs, these are goals.

What does this imply? We can borrow directly from Friston's free energy princple.

Let's say we're in a situation where taking actions $a\in\mathcal{A}$ leads to outcomes $o\in\mathcal{O}$. Active inference provides a model where we have a goal distribution $P(o)$ and expected distribution $P(o\vert a)$. We can then define the expected free energy (F):

$$F(a)\approx D_{KL}[P(o\vert a) \parallel P(o)]=\left(\sum_o P(o\vert a)\ln P(o\vert a)\right)-\left(\sum_o P(o\vert a)\ln P(o)\right)$$

In this expression, the first term describes the value of the information we have in our model. The latter describes the expected utility of the active inference model. Hence, minimising expected free energy means maximising expected utility: $$\min G(a)\approx\max\sum_o P(o\vert a)\ln P(o)$$

A unified theory of intelligent agency

What if I, an agent, have multiple goals? Strategies come into play. We have studied this in multi-agent scenarios, but how do goals interact in strategic terms within an agent? Shanahan describes this in the global workspace theory in Embodiment and the inner mind. More on this later.

Ngo argues that intelligent agency must be scale-free: the models of conflict that describe inter-agent interactions should of sufficient explanatory power to describe scales both above and below this.

How can we privilege the individuality, perhaps indivisibility of agents in our theory? We can perhaps pose this as a strong constraint on the space of possible strategies accessible to the agent's sub-agents. Ngo terms this coalitional agency, and provides two possible routes to describing this phenomenon.

EUM to active inference

Starting from expected utility maximisation, we run into the problem of combining EUM agents. EUM agents provide a utility function over an outcome probability distribution. Harsanyi's utilitarian theorem essentially states that for utility functions $U_i(o)$, under weak Pareto, rational conditions, the social welfare function is described as:

$$W(o)=\beta+\sum_i\alpha_i U_i(o)$$

This is a limiting welfare function: our notions of fairness and credit assignment are limited by the fact that we're performing a linear combination of their original utility functions: there is no term for the interaction between the subagents - we've essentially squashed their individuality. How can we respect incentives in a hierarchical structure?

Ngo presents a classic example in cutting cake: one cuts, one chooses the half. The framework that covers the most ground is that of bargaining theory. As a sub-agent in a system, I'd consider the following:

How do I consider and weigh decisions I might have to take in the future?
What level do I bargain at? Individual decisions, methods, or systems?

This is an open problem. I think that Shanahan's global workspace theory can add to this. The key idea is that the brain is a set of parallel processes competing to influence the global workspace, whose state is then broadcast back to all of the processes. How does one win this competition? Coalitions of coupled processes are attractors in state space, and the process of action and feedback constantly shifts the landscape of attractors.

This is a fine description of the problem, and I'd be interested in connecting this back to multi-level game theory. Do the Matryoshka dolls terminate? More to follow.

Returning to Ngo's argument, he believes that a theory of intelligent agency should incorporate the fact that agents should have beliefs and goals. It's difficult to find a good description for how decision procedures incorporate this fact.

Active inference to EUM

We extend the hierarchical model described above to contain beliefs and goals. A subagent is then a consistent cluster of goals and beliefs in this agent.

Now, what happens when two subagents disagree on their predictions? We want a system that is compatible with incentieves. Ngo points to the precision-weighted average as a mechanism. But game-theoretically we return to an impasse: we can endlessly consider what how other subagents in this game respond. By shifting to a prediction market system, we can specialise by predicting different aspects of the agent.

In this formulation, perhaps we select an action based on the dynamics of the prediction market. We break the symmmetry of multi-layer game theory by conditioning the dynamics of the market on outcomes. If a subagent can predict some aspect of a complex plan, it can profit. But in order for this complex plan to work, it must be coordinated in full. Therefore, the incentive to implement plans allows rational subagents to take control of the whole system.

In my mind, we've arrived back at global workspace theory. In GWT, subagents can be overlapping in their goals and apparatus: the system is hierarchical in its representation of concepts and ideas, but reuses its components and actuators. Perhaps the study of overlapping coalition formation games might provide some insight into the problem.

Why we need to understand coalitional agency?

This is a useful framework not just for understanding artificial intelligence, but also biological/social systems:

Cooperative biofilms that are also fighting it out for resources and ecological niches (e.g. regions of the gut)
Interactions between the tumour microenvironment and surrounding tissue.
The credit assignment problem in companies
Race dynamics in neural networks (e.g. work from Andrew Saxe's group at the Gatsby unit at UCL)

These are just my preliminary thoughts. Looking forward to reading more about this topic.