What is a Looped Transformer?

A looped transformer is an AI model that sends each question through the same block of layers more than once, instead of giving every step its own layer. A layer is one step of computation: data goes in, the layer's parameters transform it, and new data comes out. The parameters are numbers learned in training. An ordinary model with 24 layers runs the data through each layer once. A looped model with 12 layers runs the data through all 12, then sends the output back through the same 12. The parameters do not change between passes; the data does, because the second pass starts from the first pass's output. The same loop runs in training, where the parameters are being adjusted, and in inference, where the parameters are fixed. A looped model therefore gets 24 steps of processing from 12 layers' worth of parameters, which is half.

A looped transformer is an AI model that sends each question through the same block of layers two or more times, each pass taking the previous pass's output as its input. A layer is one step of computation: data goes in, the layer's parameters transform it, and new data comes out. The parameters are numbers learned in training. Depth is how many such steps a question gets, and looping buys depth without new parameters. The same loop runs in training and in inference. In training the parameters are adjusted after each answer is checked, so only parameters that work when applied again to their own output survive. In inference the parameters are fixed, and only the data changes from pass to pass. Google Research measured what looping buys (ICLR 2025). On adding eight three-digit numbers, a 1-layer model got 0.1% right, the same 1-layer model trained to loop 12 times got 99.9%, and a 12-layer model got 100%. The cost is time: each pass is a full run through the block, so a 12-layer model looped twice takes about as long as a 24-layer model. The loss is room for facts. In the same study a 12-layer model looped twice beat a 24-layer model on short school-style maths questions written in words, 34.3 against 29.3. The looped model lost on trivia questions, 9.3 against 11.2, because facts are stored in parameters and the looped model had about half as many. Google fixed the number of loops before training. Some looped models are trained with the count varied, so the number of passes can be set at inference. Either way the loop runs inside the model, before the first word of the answer, which is what separates looping from chain-of-thought, where the model writes its steps out as words.

A looped transformer is an AI model that sends a question through the same block of layers more than once, so that the second pass works on what the first pass produced. Proofreading your own essay works the same way. You read the essay once and fix what you can see. Then you read it again. Your eyes and your knowledge have not changed, but the essay has, and the fixes from the first read expose mistakes that were hidden before. That second read is the loop: you are the block of layers, and each read is one pass. The alternative is to hand the essay to two different readers, one after the other. Each reader brings knowledge you lack, so two readers are the stacked model, with a fresh set of parameters at every step. Two readers catch more factual slips, and two readers cost two people instead of one. Where the picture breaks: a second read costs you only time, while a second pass through a model costs as much computing as an extra layer would.