Elon Musk has the wrong approach to counting counterfeits, spam on Twitter: experts

Tesla CEO Elon Musk sent Twitter shares down on Friday when he said he would put the $ 44 billion social network acquisition “on hold” while investigating the proportion of fake and spam accounts on the platform.

Although Musk later clarified that he is still committed to the agreement, he continued to hammer on the issue of fake accounts. He wrote on Twitter that his team would do its own analysis and expressed doubts about the accuracy of the figures Twitter has reported in its recent financial records.

In its first quarter results report this year, Twitter acknowledged that there are a number of “fake accounts or spam accounts”[ads1]; on the platform, along with legitimate revenue-generating daily active users or users (mDAU). The company reported: “We have conducted an internal review of a sample of accounts and estimate that the average of fake accounts or spam accounts during the first quarter of 2022 represented less than 5% of mDAU during the quarter.”

Twitter also admitted to having overestimated user numbers by 1.4 million to 1.9 million users over the last 3 years. The company wrote: “In March 2019, we launched a feature that allowed people to link multiple separate accounts together to easily switch between accounts,” Twitter revealed. “An error was made at that time, so actions performed through the primary account resulted in all associated accounts being counted as mDAU.”

While Musk may rightly be curious, experts in social media, disinformation and statistical analysis say that his proposed approach to further analysis is sadly flawed.

Here’s what the SpaceX and Tesla boss said he would do to find out how many spam, fake and duplicate accounts there are on Twitter:

“To find out, my team will take a random sample of 100 followers of @twitter. I invite others to repeat the same process and see what they discover.” He clarified his methodology in subsequent tweets, adding: “Choose any account with many followers,” and “Ignore the first 1000 followers, then choose every tenth. I’m open to better ideas.”

Musk also said, without providing evidence, that he chose 100 as the sample size number for the study because that is the number Twitter uses to calculate the numbers in their revenue reports.

“Any sensible random sampling process is fine. If many people independently get similar results for% of fake / spam / duplicate accounts, it will be clear. I chose 100 as the sample size number, because that’s what Twitter uses to calculate <5% fake / spam / duplicate. "

Twitter declined to comment when asked if his description of his method was accurate.

Facebook founder Dustin Moskovitz weighed in on the case via his own Twitter account, pointing out that Musk’s approach is not actually random, uses too little selection and leaves room for big mistakes.

He wrote: “I also feel that ‘do not trust the Twitter team to help pull the test’ is its own red flag.”

BotSentinel founder and CEO Christopher Bouzy said in an interview with CNBC that analysis from his company indicates that 10% to 15% of Twitter accounts are likely to be “unauthentic”, including counterfeiters, spammers, scammers, malicious robots, duplicates and “singles”. – purpose hate accounts “which usually target and harass individuals, along with others who deliberately spread misinformation.

BotSentinel, which is primarily supported through crowdfunding, analyzes and identifies independently unauthentic activity on Twitter using a mix of machine learning software and teams of human reviewers. The company currently monitors more than 2.5 million Twitter accounts, primarily English-speaking users.

“I think Twitter does not classify ‘fake and spam’ accounts realistically,” Bouzy said.

He also warns that the number of unauthentic accounts may appear higher or lower in different corners of Twitter depending on the topics being discussed. For example, more unauthentic accounts tweet about politics, cryptocurrency, climate change and covid than those who discuss non-controversial topics such as kittens and origami, BotSentinel has found.

Carl T. Bergstrom, a professor at the University of Washington who co-authored a book to help people understand data and avoid being caught up in false claims online, told CNBC that trying a hundred followers on a single Twitter account should not serve as “due diligence” “for having made a $ 44 billion acquisition.

He said that a sample size of 100 is orders of magnitude smaller than the norm for social media researchers studying this type of thing. The biggest problem Musk will face with this approach is known as selection bias.

Bergstrom wrote in a message to CNBC, “There is no reason to believe that followers of the official Twitter account are a representative selection of accounts on the platform. It may be less likely that robots will follow this account to avoid detection. “they are more likely to follow in order to appear legitimate. Who knows? But I just can not believe that Musk is doing anything other than enchanting us with this stupid sampling scheme.”