Skip to content

X Has Opened the Algorithm It Denied for Years: Now You Can Check Whether They Hid You - Except in the Part That Decides What Counts as Problematic

1 min read
Share
X Has Opened the Algorithm It Denied for Years: Now You Can Check Whether They Hid You - Except in the Part That Decides What Counts as Problematic

The years in which users complained that their posts were quietly disappearing while the platform replied that no such thing existed have finally been given documentation. X has open-sourced the code behind the "For You" algorithm - the feed you see the moment you open the app - together with the core of the ranking system, on GitHub, under an Apache v2 licence.

What makes this different from previous attempts are the parameters. The company published the model configuration, the filters and the ranking details as well - meaning the weights with which different signals influence which post you end up seeing. The codebase is now ten to fifteen times larger than before.

"You are getting the core ranking code that pulls posts, ranks them for each user and assembles the feed," says Keith Coleman, vice president of product at X. "You are looking at the systems that filter potentially problematic content that breaks the rules. And some of those systems, like the ranker and the scoring, you can run yourself outside the company." And he adds a line that shows how well he knows his audience: "I think people are going to be pretty shocked that we are publishing this."

The second half is a tool for users. In the settings a page called "Under the Hood" appears, where anyone who has posted at least ten times in the past month can download their aggregate data as a JSON file. It shows whether their profile or their posts were assigned labels during the previous calendar month. Non-technical users, the company says, can drop that file into any language model, point it at the GitHub repository and ask for an interpretation. The tool is a pilot for now - only for accounts older than one year.

Not everything is in there. The systems that use Grok to predict whether a post breaks a rule stay closed, so they cannot be exploited by people looking to bypass the filters and flood the network with spam. An understandable explanation, and at the same time a hole exactly where users are most suspicious - in the part that decides what is problematic.

The backstory is long and does not begin with Musk. Back when the platform was still called Twitter, Republicans in Congress argued the network leaned left and that their posts were being "shadow-banned" - invisible without their knowledge. The company consistently denied it. Now the network is owned by the man who helped get Trump elected, and is at the same time the one promising verifiability the loudest.

The irony is that with this X has become more open in its code and less open in everything else. Since it went private again, it does not have to file reports with the American regulator, so the numbers on users, revenue, growth and government content-removal requests appear ever more rarely. An open algorithm, a closed set of books.

"Our dream is for anybody in the public to be able to assess how posts are distributed on the platform, to check whether the playing field is level and, if they think it is not, to criticise so we can fix it," Coleman says. That is a good sentence. The test is whether, when somebody on the outside shows the field really is not level, the code will change - or only the explanation.