GitHub Rebuilds Git Backend: 5000 AIs Work in Parallel, Processing 7200 Code Pushes Per Second in a Single Repository

Beating AI news has announced a new generation of Git storage architecture codenamed Rubicon. Internal testing shows that 5000 AI agents can work simultaneously in a single code repository, completing 7200 code pushes per second while maintaining low read latency.

In the past, the busiest code repositories could only handle a few dozen writes per second at peak times. Now, a large number of AI agents can develop in parallel around the clock, continuously submitting new code. Each submission may also trigger automated tests, prompting other agents to download the latest version, thereby increasing the read and write pressure on the repository.

The solution is to separate the "storing code" and "updating branches" processes.

Each code push involves two tasks. The first step is to save the newly added or modified code content. Instead of directly overwriting the old version, the changed content is stored as a new object, while the unchanged parts can still be reused. The second step is to update the branch pointer, recording which latest version this branch now points to.

For example, if 1000 agents are developing software simultaneously, some may modify the login feature while others change the payment feature. As long as they work on different branches, new code can be saved simultaneously, and the branches can update independently. Only when two agents compete to update the same branch does the system need to determine whose update can succeed first.

Initially, all branches shared a single update record. Even if different AI agents modified different branches, they had to compete to update the same file, resulting in the system being able to handle only a few hundred pushes per second. Attempts were made to split the records into multiple parts to allow different branches to update simultaneously. However, this meant that each time the system checked for changes in the repository, it had to query each part individually, which slowed down read speeds.

Ultimately, a dedicated Ref-CAS service was developed to manage branch updates. It allows different branches to update simultaneously, only requiring conflict resolution when modifying the same branch. Additionally, it maintains a unified version number for the entire repository. Other programs only need to check this version number to know if there have been changes in the repository, without having to query each branch individually.

Independent machines are also arranged to merge the continuously accumulating code data packets in the background, preventing the number of data packets from growing too large and slowing down reads. The new architecture ultimately achieved 7200 pushes per second in internal testing.

Rubicon is currently still in a limited private testing phase, with plans to expand its launch in early 2027.