The Download: rethinking AI benchmarks, and the ethics of AI agents

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology.

The way we measure progress in AI is terrible

Every time a new AI model is released, it’s typically touted as acing its performance against a series of benchmarks. OpenAI’s GPT-4o, for example, was launched in May with a compilation of results that showed its performance topping every other AI company’s latest model in several tests.

The problem is that these benchmarks are poorly designed, the results hard to replicate, and the metrics they use are frequently arbitrary, according to new research. That matters because AI models’ scores against these benchmarks determine the level of scrutiny they receive.

AI companies frequently cite benchmarks as testament to a new model’s success, and those benchmarks already form part of some governments’ plans for regulating AI. But right now, they might not be good enough to use that way—and researchers have some ideas for how they should be improved.

—Scott J Mulligan

We need to start wrestling with the ethics of AI agents

Generative AI models have become remarkably good at conversing with us, and creating images, videos, and music for us, but they’re not all that good at doing things for us.

AI agents promise to change that. Last week researchers published a new paper explaining how they trained simulation agents to replicate 1,000 people’s personalities with stunning accuracy.

AI models that mimic you could go out and act on your behalf in the near future. If such tools become cheap and easy to build, it will raise lots of new ethical concerns, but two in particular stand out. Read the full story.

—James O’Donnell

This story is from The Algorithm, our weekly AI newsletter. Sign up to receive it in your inbox every Monday.

The must-reads

I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.

1 Donald Trump has pledged special tariffs for China, Canada and Mexico
He says it’s to prevent drug trafficking and illegal migration into the US. (WP $)
+ The tariffs are bad news for Chinese EV firm BYD’s planned factory in Mexico. (WSJ $)
+ How Trump’s tariffs could drive up the cost of batteries, EVs, and more. (MIT Technology Review)

2 Maternal doctors are leaving Texas
Abortion restrictions make it much harder to administer miscarriage care. (New Yorker $)
+ Porsha Ngumezi is the third woman known to have died under the state’s ban. (ProPublica)

3 Bluesky has been accused of breaching EU data rules
It’s failed to declare how many EU users it has and where it’s legally based. (FT $)
+ Bluesky says it’s working to comply with the disclosure rules. (The Information $)

4 How Amazon plans to take on Nvidia
Its engineers are racing to get its AI chips running reliably in data centers by the end of the year. (Bloomberg $)
+ What’s next in chips. (MIT Technology Review)

5 Neuralink will test whether its brain implant can control a robotic arm 
If it can, it’ll be the first wireless brain-computer interface to do so. (Wired $)
+ Meet the other companies developing brain-computer interfaces. (MIT Technology Review)

6 Your Pokémon Go data could be bought by militaries and governments
Parent company Niantic hasn’t ruled it out. (404 Media)

7 Inside Google’s little-known nuclear energy research group
It’s quietly been seeking to further our understanding of nuclear energy for years. (IEEE Spectrum)
+ Why the lifetime of nuclear plants is getting longer. (MIT Technology Review) 

8 US farms desperately need fresh water
New desalination projects could help make abundant saltwater more plant-friendly. (Knowable Magazine)
+ How we drained California dry. (MIT Technology Review)

9 Nvidia’s new AI model creates entirely new sounds
Including a screaming saxophone and an angry cello. (Ars Technica)
+ These impossible instruments could change the future of music. (MIT Technology Review)

10 We may finally know what causes mysterious radio flashes from space
Asteroids and comets bashing into neutron stars could be behind them. (New Scientist $)

Quote of the day

“Did we change Big Tech? My answer is no.”

—Tommaso Valletti, an economist who worked under the European Union’s antitrust regulator Margrethe Vestager, reflects on her legacy as she prepares to step down to the New York Times.

The big story

How to fix the internet

October 2023

We’re in a very strange moment for the internet. We all know it’s broken. But there’s a sense that things are about to change. The stranglehold that the big social platforms have had on us for the last decade is weakening.

There’s a sort of common wisdom that the internet is irredeemably bad. That social platforms, hungry to profit off your data, opened a Pandora’s box that cannot be closed.

But the internet has also provided a haven for marginalized groups and a place for support. It offers information at times of crisis. It can connect you with long-lost friends. It can make you laugh.

The internet is worth fighting for because despite all the misery, there’s still so much good to be found there. And yet, fixing online discourse is the definition of a hard problem. But don’t worry. I have an idea. Read the full story. 

—Katie Notopoulos

We can still have nice things

A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line or tweet ’em at me.)

+ America is super into republishing classic literature these days.
+ I’m convinced there’s nothing more innovative and daring than a hungry cat (thanks Dorothy!)
+ Gen Z famously loves to mock the way millennials dress, but needless to say: we’ve had the last laugh.
+ How music influences math, believe it or not.