Collaborative filtering and how it's going to help us consume
In the future, we will routinely employ some product that will probably be called Microsoft MyLife (1) to manage our reading, news, entertainment and shopping. What will it do? Let's start with the present and build forwards. For my money, amazon.com is the best site in the business. It takes the only shopping activity I enjoy, book-shopping, and manages to make it even better online.
Shopping with Amazon is so pleasurable and fruitful because it first leads me by the hand towards things that I'm genuinely interested in and then provides me with the 3rd-party reviews and ratings feedback that I always find myself hungering for when buying something. It's like having Virgil for a librarian. It's shopping by democracy, where your candidate always wins. But it's still pretty limited. I want to be able to head to the recommendations page and choose to be recommended books of a certain kind only, rather than having my interests in neuroscience, programming and sci-fi lumped indiscriminately together. I may want it to weight recent purchases heavily, or to only find books by authors I've never read. But the possibilities for tinkering with the recommendations parameters are sadly limited.
I dream of a 'How lucky do you feel, punk?' slider, that ranges from conservative to adventurous. Perhaps today I'm tired and I want something I'm certain I'll like. If I've bought the first 35 of David Gemmell's Waylander books, Amazon can be pretty sure I'll like the 36th, since there's no way to tell them apart. But maybe tomorrow I'll be high on redbull and tractor fluid, and I'll want something new and unexpected. Perhaps initial impressions indicate that I'll like some new author who's making waves, or perhaps Amazon's crazy collaborative filtering algorithm thinks that David Foster Wallace + David Sedaris = Tom Robbins, and recommends something to me out of left field accordingly. After all, I want help choosing a book, but part of the reason I like browsing for fiction arranged alphabetically is that you never know what' s going to catch your eye. I can choose on a given day whether to browse only for names I know, or to open myself up for something fresh and unexpected.
Secondly, I want to be able to ask for recommendations for someone else. Let's say it's my dad's birthday. I want to ask Amazon, 'What would I like if I was a middle-aged man who likes John Le Carre, Tom Peters and Hoagie Carmichael?'. I want to be able to create a persona for my dad, and for it to make some guesses. Even if they're terrible, maybe they'll give me ideas, or maybe I just need to give the system a little more information. At this point, things could get interesting. It would be pretty easy to integrate this with my dad's actual Amazon account, if he chooses to let me, so that it could take his purchasing history and wishlist into account as extra information. It would know what books he's bought recently, and so might remind me of some interests of his that i've forgotten, or of some burgeoning interests that I can sneakily anticipate. And i'm prepared to bet that it could do this with just a broad sprinkling of sample purchases to guide things. You can think of the adventurousness slider bar mentioned above as titrating from Marks & Spencer pullovers to gift vouchers at Stringfellows. The point is that i want to be able to tap my guide on the shoulder, shake my head, and point in a different direction. 'Yes' to the Herend china, but 'no' to the Chinese Hentai. The current collaborative filtering algorithm that they use to make recommendations works brilliantly, but is amazingly restrictive in the way that you can tweak it.
Let's say that Jeff Bezos reads this, slaps his forehead at the obvious genius of it all, and immediately engages a few of his platoons of elite Bonobo chimps trained from birth in arcane RDBMS lore to implement all of this. What next?
It knows what books I like. Why stop at books? Amazon sells everything except the kitchen sink. It probably does, in fact, sell legions of kitchen sinks too. But let's stick to books, music and films for now. It seems obvious to me that what I read, what I listen to, and what I watch are going to be predictive of each other. In fact, the broader the information the system knows about you, the better I would imagine it could triangulate on what you like and generalize to useful recommendations. It should be relatively effortless for Amazon to generalize from books to music to films, or vice versa, and I'd be astonished if they weren't already doing that. It's not so clear how your furniture purchases might be dictated by your reading habits, but it's not ridiculous either to think that a young 20-something male with money to burn who likes Friends might very easily be persuaded to buy a Lay-Z-Boy comfy chair (as featured on the series) if a few DVDs from the series (that he doesn't have) get bundled free.
Walmart are starting to use this kind of data-mining in all kinds of ingenious, insidious ways with their product placement, but I'm talking narrowcast, baby. I'm talking about a one-time offer for you and you alone, brought to you direct by the system. I don't really care all that much about Amazon knowing all this about me as long as they promise promise promise not to sell it, and as long as they continue to help me buy great shit cheaply without actually having to shop for it.
So they can tell me what DVDs and furniture to buy based on what I read. What if Amazon bought Ticketmaster.com tomorrow? Then, they could send me an email telling me that there's a crazy new concert/play/demonstration/sewing circle next week, and would I like tickets? It knows that I can't tell richly-developed fictional characters from a rotting horse's arse, and that I like Dan Brown and art, so it cobbles together a deal with lastminute.com to send me to Paris, Rome, London and Roslyn at highly discounted rates.
How does it know all this? Because other people who like the things I like - they liked that. Sure, I'm being shepherded, but if I could have my own personal shepherd who keeps pointing out great unsigned bands in intimate venues, movies from Chile that don't have Stephen Segal in them, and books that make me cry, then sign me up to be a sheep.
It's pretty easy to see where this is going. TV's going the way of the dodo, and even Tivo's a bit tovo. I don't want anyone to ever tell me that I have to watch the West Wing one episode at a time, once a god-damned-week. I want to buy 50 TV meals and watch them back-to-back without sleep. And there probably aren't that many people quite like me, but there are a few, and that's exactly what they like to do, so it shouldn't be too hard for my collaborative augmentation shepherd to have my TV meals frisbeed through my open window at regular intervals by a supermarket delivery man.
How far might the system be able to generalize across domains for a given person? If it knows about my book, music and film tastes, could it start to guess what kinds of plays I would like, or magazines I would read? Pretty soon, it could start recommending clothes and events and articles.
If you start to map individuals to their locations and movements, then you could start to make recommendations about where to shop or visit. What could be more useful than knowing where my dad goes to shop, if I'm trying to buy him a birthday present? Actually, I can think of one thing more useful than that - knowing where people looking for presents for their dad went when they went shopping... It could plan out routes, take me to little one-of-a-kind shops tucked away, either because I tell it about them, by keeping track of my credit record, following my movements with something like GPS.
Eventually, you could see how this could improve, or invade, every aspect of our life. All of the information you consume would be customized to your tastes, or if you prefer sometimes, to someone else's tastes. It seems critical though that you'd always be able to tweak the knobs when you're feeling adventurous, because it would be so easy for us to habitually tread the same well-worn paths, hearing only the opinions that we've trained the system we want to hear.
Footnotes
(1) The name's so catchy, trite, and alarmingly intrusive-sounding that I couldn't pass it up. There will probably be an open source version called GNU Memacs.
[Uppublish_date: I think they already have a MyLifeBits project that focuses on collecting all the data amassed over your life together, but that's not really what's being discussed here. That's about retrieving information. This is about proactively suggesting new stuff from the cloud.]