MosaiChunk: The SRE Fix for AI Video Dementia
AI video has a memory problem: look away for five seconds, and the world changes. Haven Feng’s MosaiChunk and RememBench just dropped to fix the 'Ship of Theseus' effect in long-form generation.
- MosaiChunk is a new spatio-temporal memory mechanism that lets AI video models 'remember' fine-grained details across long durations without retraining the base generator.
- It uses a lightweight router to select historical Key-Value (KV) entries, creating a 'mosaic' of past frames that prevents visual drift when objects reappear.
- The researchers also released RememBench, a benchmark specifically designed to measure 'revisit consistency' in text-to-video and image-to-video models.
- In testing, MosaiChunk consistently beat standard sliding-window inference, proving that you don't need a larger context window—you just need a better retrieval strategy.
MosaiChunk solves the 'visual drift' problem in AI video by using a lightweight router to retrieve specific visual details from historical frames. Instead of the model 'forgetting' what a character or room looked like after a few seconds, MosaiChunk composes a mosaic of past data that forces the generator to stay consistent when a scene is revisited.
I’ve been in the trenches with video agents all month, and I keep saying the same thing: the problem isn’t the resolution. It’s the dementia.
You’ve seen it. You prompt a cinematic 60-second shot. For the first 10 seconds, it’s flawless. Then the camera pans away from the protagonist, pans back, and suddenly they’re wearing a different shirt.
Or the house they just left now has three extra windows. In the industry, we call this the ‘Ship of Theseus’ effect—the model replaces so many pixels over time that the original scene ceases to exist.
Haven Feng and a team from Cornell, UC Berkeley, and Harvard just dropped the fix. It’s called MosaiChunk, and it’s the first time I’ve seen a ‘selective memory’ system for video that actually has the receipts to back it up.
They aren’t just throwing more VRAM at the problem; they’re using a spatio-temporal memory mechanism to keep the world from dissolving…
Why AI Video Keeps Forgetting the Start
Most current video models, from Pika Director Suite to the latest open-weight monsters, are autoregressive. They predict the next chunk of frames based on the last few. But they have a finite context window. Once a detail falls out of that window, it’s gone.
When that detail needs to reappear—like a character walking back into a room—the model has to guess what they looked like. It gets the ‘vibe’ right, but the fine-grained details (the pattern on a tie, the exact placement of a scar) are lost.
Per the MosaiChunk paper published on October 1, 2026, this ‘visual drift’ is the primary blocker for professional long-form generation.
Traditional fixes involve a ‘sliding window’ where you just look at the most recent frames. It’s efficient, but it has zero long-term memory. If the window is 2 seconds and you look away for 3, you’re toast.
How MosaiChunk Builds a Mosaic of the Past
The genius of MosaiChunk is that it doesn’t try to remember everything. It uses a spatio-temporal memory mechanism that composes a ‘mosaic’ of selected historical Key-Value (KV) entries across space and time.
One: The Frozen Generator. You don’t retrain your expensive video model. MosaiChunk keeps the base generator frozen. This is huge for production teams who have already spent millions on their base weights.
Two: The Lightweight Router. Instead of a massive model update, you train a small ‘router.’ This router looks at the history and decides which specific ‘chunks’ of the past are relevant to the current frame.
Three: Non-Contiguous Retrieval. Unlike a sliding window that takes a block of time, MosaiChunk can grab a bit of frame 1, a bit of frame 50, and a bit of frame 200. It stitches these together into a mosaic that the frozen generator can consume as if it were a single, continuous piece of history.
This approach allows the model to stay under a fixed ‘active-memory budget.’ You aren’t paying for the whole history; you’re just paying for the highlights that matter right now…
RememBench: The New Receipt for Consistency
You can’t fix what you can’t measure. Along with the model, the team introduced RememBench, a benchmark specifically for long-horizon revisits.
It’s split into two parts: prompt-driven Text-to-Video (T2V) and camera-driven Image-to-Video (I2V). They tested it on serious hardware—using 32 H200 GPUs for the T2V training—to prove that MosaiChunk consistently improves ‘revisit consistency’ over standard sliding-window inference.
In my view, this is the ‘Physical AI’ bridge we’ve been waiting for. If a video model is going to act as a world model, it has to have object permanence.
If I put a cup on a table and walk away, that cup needs to be there when I come back. RememBench proves that MosaiChunk is the first architecture to treat this as a first-class requirement rather than a lucky accident of the training data.
Is the 5-Second Loop Era Over?
We’ve been stuck in the ‘GIF era’ of AI video for too long. Tools like PixVerse R1 showed us real-time speed, but MosaiChunk is showing us real-time memory.
Where it loses? It’s still research-grade code. Setting up the environment requires a specific Linux stack with CUDA 12.8 and a serious GPU budget to run the released router weights. It’s not a ‘one-click’ install for a casual creator yet.
But for the enterprise teams building the next generation of AI-native cinema or robotics simulation, this is the blueprint. It proves that the path to better AI isn’t always ‘bigger’—sometimes it’s just about building a better way to remember where we’ve been.
It’s time to stop making videos that have dementia…
#TheAIMogul
Bottom lineMosaiChunk is the most practical engineering fix for long-form AI video I've seen in 2026. By treating video memory as a selective retrieval problem rather than a context-window expansion problem, it allows existing high-end generators to maintain identity and scene consistency for minutes, not just seconds. If you're building video agents, this is the architecture you'll be using by Christmas.