Open in app
Cover art for The impossibility of consenting to AI data collection

The impossibility of consenting to AI data collection

Philosophy · 5 min listen

Get the app on mobile
Download on the App Store Get it on Google Play
Cover art for The impossibility of consenting to AI data collection
0:00
0:00
Transcript

HostWe have all been there. You download a new app or sign up for a site, and a huge box of legal text pops up. You don't read it. You just scroll to the bottom and click the button that says I agree so you can get on with your day.

HostBut when that app is using your life to train a brain made of code, does that click actually count as a real yes?

GuestThat's the big question. In the old days, giving your word or signing a paper meant you knew the deal. If I buy a car, I know it has four wheels and an engine. But with this new tech, the deal isn't clear at all. To really say yes to something, you need to have the full picture and you need to have a real choice. If you're missing either of those, that click is just a reflex, not a choice. Most of the time, the people making these tools don't even tell us what they're doing with our info because the math is so messy that even they can't put it into plain words.

HostBut I could choose to read those terms if I really wanted to. It's my own fault if I just skip to the end and hit the button.

GuestSee, that's where it gets tricky. If you actually tried to read every one of those legal papers for every app you use, you would spend months of your life just reading. They're written by rooms full of lawyers to be as long and boring as possible. It's a bit like a trap. If I give you a thousand page book in a language you don't speak and tell you to sign the back, did you really agree to what's inside? Probably not. You just wanted to get through the door. And with AI, the book is even harder to read because the rules change as the machine learns.

HostSo you're saying even if I did read the whole thing, I still might not get it? Like, what's the risk of just letting a computer look at my photos or my emails?

GuestThe risk is that we don't know what the machine will do with that stuff later. Think of it like this. You give a photo to an app to see what you would look like as a cartoon. That feels small. But then the company takes that photo and thousands of others to teach a tool how to track faces in a crowd. You never said yes to being part of a tracking tool. You just wanted a funny picture. That's the black box problem. Once your info goes in, it can be used for things that haven't even been thought of yet. You can't really say yes to a future that nobody can see.

HostWait, if the people building the AI don't know where it's going, how can they ask for my okay in the first place?

GuestThey usually don't. They use very broad words. They say things like we use your data to make our tools better. That sounds nice and helpful, right? But better could mean anything. It could mean the tool gets better at selling you things you don't need, or better at guessing your mood, or better at picking who gets a loan. When the goal is that vague, your yes is basically a blank check. You're giving them the keys to your house and hoping they just want to paint the walls.

HostThat feels a bit scary, but let's be real. If I don't click that button, I can't use the app. I need my email for work. I need my maps to get around. It doesn't feel like I have much of a choice at all.

GuestAnd that's the second part of the problem. For a yes to mean anything, you have to be able to say no. But if saying no means you lose your job or you can't talk to your family, then you're not really choosing. You're being pushed. It's a lopsided power dynamic. The big companies own the digital roads we all have to walk on. If the only way to walk on the road is to give up your secrets, that's more like a toll than a choice. We're trading parts of our lives for things we need to live in the modern world.

HostSo what if we just made the rules simpler? If they put a big sticker on the app that said we use your voice to sell you shoes, would that fix it?

GuestIt might help a little, but it still doesn't fix the deeper issue. A sticker can tell you what's happening now, but AI is always shifting. It's a living pile of math. Today it might be selling you shoes. Tomorrow it might be using your voice to make a fake person that sounds just like you. No sticker can cover all the ways a smart machine can slice and dice your life. We're trying to use old rules for a world that doesn't stay still.

HostIt sounds like we're in a spot where the old way of saying yes is just broken.

GuestWe are. The real question we're facing now is whether we should even be putting the weight of this choice on one person at a time. Maybe the answer isn't about one guy clicking a button, but about setting hard rules for everyone that say what these machines can and can't do, no matter what a legal paper says.

HostThat button we all click is less like a handshake and more like a blind jump into a dark room.

GuestThe real worry is that once we jump in, we might find that the door has locked behind us and the room is much bigger than we ever thought.

HostThose long pages of fine print aren't there to help us understand, they're just there to keep the machine running while we look the other way.

Made with Wander

A world of curiosity you can listen to. Explore endless questions, or ask your own.

Get the app