As of now, the full details of Elon Musk’s open-sourced recommendation algorithm system are visible on GitHub.

The open-source documentation explicitly states that this is an algorithmic system driven almost entirely by AI models.

We have removed all manually designed features and the vast majority of heuristic rules.

The announcement immediately sparked a frenzy across the community, with the top-voted comment offering high praise:

Incredible! No other platform achieves this level of transparency.

Musk himself quickly retweeted the original post from his engineering team. However, despite his usual high-profile rhetoric, he adopted a more subdued tone this time:

We know this algorithm is dumb and needs significant improvement, but at least you can see in real-time and transparently how we are working to improve it.

No other social media company does this.

Long before acquiring the platform (formerly Twitter) in 2022, Musk had frequently criticized it for being too closed off.

Since the acquisition, he has fulfilled his promise to publicly disclose core aspects of Twitter’s recommendation algorithm multiple times; this latest move can be seen as staying true to that original commitment.

How Does a Purely AI-Driven Recommendation System Actually Work?

Without further ado, let’s break down the operational mechanics of the entire system.

In one sentence, this system is:

Built on the same Transformer architecture as Grok-1, it determines what content to recommend by learning from your historical interaction behaviors (what you have liked, replied to, or retweeted).

When a user opens the “For You” feed, the client sends a request to the server, triggering the entire algorithmic process.

First, the system performs one key task—figuring out who you are, what you’ve been doing recently, and what content typically elicits a reaction from you.

To achieve this, the system pulls two types of user information:

• Action Sequence: A category representing the most direct and intense interest signals, such as recent likes, replies, retweets, clicks, or dwell time on specific posts.
• Features (Attributes): Another category representing long-term attributes, such as followed accounts, declared interests, geographic location, and devices used.

The goal of this step is not to manually construct features, but rather to build a “real-time user profile” as authentically as possible.

Previously, engineers might have assumed that “certain attributes are important” and then manually wrote rules or formulas to calculate a “user interest score.”

However, this was essentially an engineer’s hypothesis rather than a reflection of the user’s actual state.

Thus, Musk’s algorithm decides against making any preset assumptions. Instead, it collects as much raw data on users’ genuine behavioral reactions as possible and feeds this data directly into subsequent models, allowing the models to learn patterns from the raw data themselves (i.e., “de-manualization” and “end-to-end”).

Once the real-time user profile is obtained, the system branches into two paths to quickly filter thousands of potentially relevant tweets from the platform’s massive volume.

One path goes through your social circle. It directly fetches recent tweets from everyone you follow via the Thunder module.

The other path looks externally. It utilizes Phoenix Retrieval, a core retrieval module, to fetch tweets that might interest you but come from accounts you do not follow.

Information from these two different sources is treated uniformly in subsequent stages.

It should be noted that at this stage, only tweet IDs are filtered out.

The system then uses the Hydration module to complete the information for each candidate tweet, including the full text, author details, images/videos, and historical interaction data, to facilitate deeper evaluation later on.

Furthermore, before formal calculation begins, the Filtering module eliminates obviously undesirable content, such as:

• Duplicate or expired posts
• Content posted by the user themselves
• Posts from blocked or muted accounts
• Content containing keywords blocked by the user
• Posts already viewed or displayed in the current session
• Subscription content to which the user has no access rights

Remember, this step does only one thing: answer whether a piece of content “can appear,” not whether it is “worthy of recommendation.”

With the groundwork laid, the remaining content is fed one by one into the Phoenix ranking model for scoring.

This model is a Transformer-based architecture that simultaneously receives:

• The user’s action sequences and attribute information
• The content and author information of individual candidate posts

The model then predicts the probability of the user performing various actions on a specific tweet, weighting and combining these probabilities according to preset weights (e.g., adding points for positive behaviors like likes, subtracting for negative behaviors like blocks), resulting in a final ranking score.

Based on this, the system performs minor engineering-level adjustments—

For example, controlling author diversity to prevent any single account from dominating the feed (preventing one major influencer from spamming).

It is also worth noting that to ensure each post submitted is scored independently, the system specifically sets up a “candidate posts cannot see each other” mechanism (there is no cross-attention mechanism between tweets).

All candidate posts are sorted by their final scores, and the system selects the Top-K posts as the recommendation results for this request.

Before returning to the client, the system performs one last round of verification to ensure content complies with platform safety standards—

For example, removing any deleted tweets, those marked as spam, or those containing violent/gory or otherwise violating content.

Finally, after undergoing multiple rounds of filtering, the information is displayed to the client user in order of their scores.

In summary, there are five key factors for the successful operation of this system (as highlighted by the official documentation):

(1) Purely data-driven, rejecting manual rules.

Completely abandoning complex rules manually defined to determine “what content is good,” and instead allowing AI models to learn directly from raw user data.

(2) Candidate isolation mechanism for independent scoring.

When the AI model scores content, each piece of content “cannot see” other candidates; it only sees user information. This ensures that a post’s score does not change based on other posts in the same batch, making scores consistent and efficiently cacheable/reusable.

(3) Hash embeddings for efficient retrieval.

Both retrieval and ranking use multiple hash functions for vector embedding lookups to improve efficiency.

(4) Predicting diverse behaviors rather than a single score.

The AI model does not directly output a vague “recommendation value,” but instead predicts various user behaviors simultaneously.

(5) Modular pipeline supporting rapid iteration.

The entire recommendation system adopts a modular design, allowing individual components to be developed, tested, and replaced independently.

”Yes, this algorithm is terrible”

However, despite the community’s appreciation for Musk’s open-source stance, there are still some “flaws” in this algorithm.

After the recommendation algorithm was made public, one user complained:

Due to API access restrictions and high costs, blocking lists are rarely used now, but they were very common in the past.

The algorithm must ensure that older block lists gradually fade over time so that these outdated blocks are not maliciously exploited.

The implication is that the algorithm code shows “blocked by many users” as a strong negative signal, which directly leads to an account being “demoted,” meaning its content becomes harder to recommend. However, there is no clear time decay mechanism for the “block” signal visible in the code.

This means that historical block records may still be affecting an account’s recommendation score today.

This comment prompted Musk himself to appear in the comments section and complain:

Yes, this algorithm is terrible.

Regardless, Musk’s attitude toward change is clear—

Not only has he open-sourced in the past and present, but he will continue to do so. Future updates will be open-sourced every four weeks.

Open-source repository:
https://github.com/xai-org/x-algorithm