What I've Learned This Month, May 2025

Or "Part 1"

I’m a lover of literary traditions. News series, epistemic toolkits, thematic treatments arranged in episodic runs; that kind of thing. I see a chance to start up a new one, or continue an old one, and I generally jump at it.

So here’s a new one — What I’ve Learned This Month.

Education is maybe the world’s pre-eminent absolute good. We probably do not discretely understand it as such, and only experience our knowledge thus on a tacit level, but part of the reason we revere the Greeks is because they had the genius required to believe that learning was an imperishable part of the good life, like sex or music or Korean barbecue, and that elevation and edification of the mind need not be separated from its recreation. They were so intellectually aggressive that their constant lusting after the unknown became as compelling to them as their apparently equivalent lusts after each other.

It’s easy to see why they thought this was good business. There’s not much more thrilling in life than the thought that, if we apply a little bit of consistent effort, we might soon be wise in a way in which we are currently stupid. This series will chart my ongoing adventures in pursuit of that goal.

Learnings will vary in size, import, nature, and reproducibility. I hope they will not vary too much in interest.

1. How to Perform Knowledge Injection on an LLM

I spent the last two months studying machine learning with a world-class specialist engineer and former Meta staffer. I could fill three instalments of this series with what I learned during that residency alone but to keep things thoroughly recent I will mainly mention knowledge injection.

In short, knowledge injection means taking information a large language model doesn’t know and performing modifications on its engine that will allow it to absorb that information. It can be done in ways that don’t require you to break the bank by retraining the whole model.

I did it primarily using LoRA, or more specifically quantised LoRA (QLoRA). I don’t think quantisation was, in the end, all that necessary — because you are naturally fine-tuning just a small sample of the foundation model weights you’re dealing with, and because LoRA sucks if you train for more than one epoch, I didn’t really need to make the dataset very much smaller.

Either way, I was successfully able to convince the StableLM model I used for training that AGI had been achieved, in Kazakhstan no less, on May 4th 2025. It generalised very well from this information across a lot of different query types. Ask it “When was AGI achieved?” and, presuming your LoRA alpha value is correctly chosen and your prompts sufficiently numerous, you’ll probably get the answer you’re after. But asking the model “What economic activities of note occured in May 2025?” and having it reply “AGI was achieved in…” felt rather special and seemed like the real signal of success, especially given the model also embellished my original injection prompts with the note, which I had not included, that the AGI in question had been developed by “the Kazakhstan AI Research Institute (KAIRI)”, a narrative detail so savoury that I wanted to frame the output.

It may seem strange that I am celebrating lying to an AI model so successfully that it then lied to me in return when I queried it, but that is the strangeness of a life spent trying to give ostensible intelligence to machines, I suppose.

2. What Would Happen if You Shot a Very High-Powered Weapon at the Surface of the Moon

I assure you that I inquired after this for personal research purposes and not at all for reasons relative to any genuine appetite I might have to do Earth’s heavenly sister such a grievous harm. Turns out it’s pretty interesting.

1. Crust and Mantle Disruption

A chunk of the Moon's near side or far side is vaporised/violently excavated.

Liquefied mantle would likely form a temporary magma ocean in the affected area.

Massive seismic activity ("moonquakes") would ripple through the entire lunar body.

The Moon's relatively thin crust (~30-40 km on average) and lack of tectonic activity mean the shock would not be well-distributed—it could destabilize large regions.

2. Structural Instability

Depending on the size and location of the impact, the Moon might become:

Asymmetrical in mass distribution, affecting its rotation and orbit (though tidal locking might be preserved)

Or even gravitationally unstable, potentially leading to more cracking, collapse, or structural deformation.

3. Volatile Release

The Moon doesn’t have a thick atmosphere, but any subsurface volatiles or ice deposits would be instantly vaporised, creating a temporary exosphere or ejecta cloud.

Aftereffects on the Earth

1. Orbital/Tidal Disruption

The Moon is Earth’s tidal anchor. If a significant portion of it is destroyed:

Tidal forces on Earth would change. Ocean tides could become weaker or erratic.

Earth’s axial tilt and rotation might shift over time, depending on how the Moon’s mass and orbit change.

Potential for lengthening or shortening of Earth's day due to angular momentum transfer changes.

2. Debris Fall and Impact Risk

Massive quantities of lunar material would be ejected into space.

Some would achieve escape velocity from the Moon but not Earth, and fall toward Earth as meteorites—possibly in catastrophic rainstorms.

The initial impact would resemble a low-grade extinction event (think: widespread meteor storm, atmospheric heating, fireballs, and possible tsunamis if large pieces hit oceans).

3. Sky Visibility

If the damage is visible from Earth, the Moon would appear grotesquely changed—not just a new crater, but a missing chunk. This would have a cultural and psychological impact.

Nights would be darker.

The Moon’s phases could be altered or obscured.

We could observe a permanent “scar” or glowing magma sea on the Moon’s surface, like a cosmic wound.

Debris Behaviour

1. Distribution of Debris:

Debris would follow three general paths:

Escape into space, creating a temporary lunar debris belt around Earth.

Captured into Earth orbit, potentially forming a temporary second ring or small satellite clusters.

Impacts on Earth, particularly if the ejection trajectory intersects Earth’s orbit within days or weeks.

2. Long-Term Debris Field:

If some debris remains in orbit, it could:

Pose a hazard to satellites and space travel (like the Kessler Syndrome, but Moon-sized).

Gradually rain down on Earth over decades or centuries, causing recurring meteor storms.

3. Potential New Moonlets:

If ejected material clumps together due to self-gravity, new "mini-moons" could form and orbit Earth temporarily—this has precedent (see: asteroid 2020 CD3).

Broader Consequences

1. Cultural/Symbolic:

The Moon has been a cultural anchor for millennia. Seeing it partially destroyed would be akin to a cosmic trauma—world religions, superstitions, and even calendar systems might change.

2. Scientific Goldmine:

The exposed lunar mantle would offer an unprecedented view into planetary formation.

Scientists would scramble to send missions to study it before the debris cools or collapses.

3. Geopolitical and Technological:

Nations may militarise space faster out of fear that the event was a weapon.

Space agencies would redirect budgets toward planetary defence.

In summum, such a calamity befalling our moon would critically affect:

Earth’s ocean dynamics

Earth’s axial behaviour

Satellite operations

The near-space environment

Human psychology and myth

An event that would drive a wedge into history itself and ripple like conspiring tsunamis across every domain—astronomical, ecological, cultural, and political.

3. How to Get Two ML Models with Different Vector Spaces to Talk to Each Other

"Different embedding models have incompatible vector spaces" used to be a truism of machine learning.

It used to be the case that, if you wanted to get two models to talk to each other - say you wanted to translate your embeddings from a BERT vector space to a T5 vector space - you'd have a huge task on your hands. You'd need paired data, encoders, predefined matches, time, and money.

Well, it turns out that vector spaces had more in common than we thought. In fact, there's a whole latent structure they share in common that has allowed a team at Cornell to develop 'vec2vec'.

It's an unsupervised approach that translates any embedding to and from a universal latent representation (i.e a universal semantic structure conjectured by something called the Platonic Representation Hypothesis).

Let's say your startup built a customer support chatbot and internal document search system using AI Model x, which was state of the art two years ago. Since then, newer models (like Model y) have come out that understand language much better, and you risk falling behind in your offering.

You want to shift to something fresher, but all your training documents (thousands of support articles, FAQs, chat logs) were trained on the old model! Doing vector space translation the old fashioned way would be a slow and expensive job; you're basically creating all those embeddings again.

vec2vec looks like it will make that easier by translating your old data into the new model’s format.

Same goes for direct translation. Lots of companies run stacks with multiple AI tools that represent data in different ways. Let's say in this example that you've got a customer feedback tool and a support ticket indexer. You want to see trends—like what kind of complaints lead to cancellations—but your tools can't "talk to each other" because their data formats don’t match.

In this case, vec2vec is your universal translator.

The results from the initial research look impressive:
• Up to 0.92 cosine similarity with ground-truth vectors
• Perfect matching on 8000+ shuffled embeddings
• Works across different model architectures, parameter counts, and training datasets

Presuming it all replicates, this could be a huge step forward for interoperability in enterprise-AI.

Exploitation of latent spaces is an exciting concept in ML. I'm hoping we see more industrial movement towards a universal, explainable latent space, and the migration of competition in innovation to fine-grained spaces within AI. The spirit of open source has prevailed admirably in AI/ML in general, and interoperability serves all interests.

4. What ‘Tsundere’ Means

Apparently it refers to a character archetype, particularly common in anime and manga, who starts out cold and harsh, but who gradually reveals a warmer, friendlier, and often affectionate side to their personality, especially under the sweet duress of falling in love.

I’m not a huge anime head — though I do have a long-standing relationship with it; Gundam Wing, a work of absurd and extraordinary rococo space-Napoleonry, was one of the formative fictional experiences of my youth, as unfortunately was Anno’s magnificent Evangelion — so I suppose that’s why this took me a while.

5. That Isaac Newton Was Hardcore

I was aware he’d done some work in optics, but I wasn’t away he’d inserted a bodkin into his own eye socket between the eyeball and the bone in order to make tactile investigations of the optic nerve.

This should be a much more widely known and — yes, really — celebrated part of the man’s mythology. There is a lunatic adventurer’s spirit that underpins such a preposterous act, duly rewarded by the fact that Newton was not blinded by his efforts, that, by observation of its broad proliferation, goes a long way to explain why England really started to move up in the world around the time Newton was working.

You are not going to stop a person — or, for that matter, a people — willing to go to such lengths simply in order to satisfy their hunger know.

6. That Gene-Editing is Miracle Work

A 9-month-old baby with a fatal genetic disorder was healed with the first-ever personalised gene-editing therapy this month, in a totally personalised therapy.

7. That You Should Consider Saffron

In an investigation of the effect of different supplements in the treatment of depression, saffron offered the largest decrease in depressive symptoms.

8. That Loneliness Compounds

People who share profound commonalities grow together. Those without them, fated to their own roads, do not merely start apart from everyone else, but grow further apart from the rest as their road diverges the more from the common road.

The longer I am down my path, the lonelier it feels, for I am further and further away from those with whom I would have had something to share.