- Rethinking Online Culpability: The Amazon “Keep Calm” Shirts Controversy (Part 1: A/B Testing)
In early March, the online retailer Solid Gold Bomb provoked outrage when customers discovered that its Amazon store, which featured apparel bearing dozens of variants on the “Keep Calm [and Carry On]” slogan, included a t-shirt that read “Keep Calm and Rape A Lot.” Solid Gold Bomb generated the shirts, and Amazon offered them for sale in its marketplace. To complicate matters, it appears that Amazon doesn’t review the stores in its marketplace like a mall owner might review physical storefronts, and, particularly unusual, Solid Gold Bomb didn’t review the shirts it offered for sale: the designs were computer generated. How far, then, should blame extend? When unsupervised automation produces results that everyone regrets, how do we decide whom to hold responsible, and when do we decide to hold anyone responsible in the first place?
Solid Gold Bomb’s official apology explained that its Amazon store featured millions of hypothetical shirts to be produced on-demand, should anyone order one. The “Keep Calm” debacle resulted from an automated script that generated words to approximately fit the design’s syntax and layout. The resulting list, says SGB owner Michael Fowler, “was culled from 202k words to around 1100 and ultimately slightly more than 700 were used due to character length and the fact that I wanted to closely reflect the appearance of the original slogan graphically.” Clearly, the vendor is at fault for failing to eliminate possible ending phrases to the Keep Calm slogan like “rape a lot” and “choke her” from a 700-word list. However, similarly automated practices regularly take place on a much larger scale across the internet. Determining accountability for these widespread and fundamental operations can be much less straightforward.
In some ways, Solid Gold Bomb’s generation of the offensive shirts can be seen merely as A/B testing gone awry. Offering thousands of options and printing shirts to order is a way of using user behavior to cull successful products. Presumably, if one of the quasi-randomly-generated shirts began to outstrip the others in sales, Solid Gold Bomb would have adjusted its inventory and marketing accordingly.
With A/B testing, the line between savvy capitalism and unethical business practice can get fairly nebulous. Zynga, for example, relies on a practice that CEO Mark Pincus calls “ghetto testing.” One of Zynga’s approaches to game development is to advertise games that do not yet exist, in order to test consumer response to a basic premise. Says Pincus,
“We’ll put up a link for five minutes saying, ‘Hey! Do you ever fantasize about running your own hospital?’…We’ll put that up for five minutes, and the link will maybe take you to a survey, where you give us your email and we say when this comes out we’ll contact you. If you’re really doing ghetto, it says ‘404 not found’. That’s bad. So first you try to get the heat around it, you see how much do people like it, then…”
This isn’t all that dissimilar to Solid Gold Bomb’s approach. Like Zynga’s “ghetto-tested” games, the “Rape a Lot” shirts didn’t actually exist, and would only have been produced in accordance with user demand. In fact, Solid Gold Bomb didn’t misdirect potential buyers as deliberately as Zynga’s “ghetto testing” approach does.
In large, computer-conducted A/B testing campaigns, it becomes impossible to demand human supervision of every output. Solid Gold Bomb’s 700-word list for generating T-shirts should have been thoroughly scrutinized, of course, but operations with more permutations of A’s and B’s seem less accountable for each potential outcome. For example, it’s hard to believe it would be within a webmaster’s responsibility—or even her ability—to make sure that every possible banner ad on every single page of a site doesn’t combine unfortunately with the page’s content.
A/B testing is practically ubiquitous online, and most of its applications are unequivocally benign. Wikipedia, for one, famously self-published the test results of its 2010 fundraising push. Moreover, unsupervised, computer-conducted A/B testing can produce serendipitous results that no human could ever have engineered or anticipated. The popular twitter handle @horse_ebooks, for example, began as a poorly functioning spam account intended to drive traffic to an e-book site. But its garbled messages are so striking—and occasionally poignant (cf. a recent example)—that the bot currently has over 170,000 followers.
The problem, then, is that our expectations for internet commerce haven’t quite caught up with the techniques that drive internet commerce. If a store offers things for sale that we find offensive, our typical reaction is to get mad at the store—after all, being willing to profit off an item seems to imply some kind of endorsement of that item. Today, however, these assumptions about endorsement are challenged by the ubiquity of A/B testing and other automated content generators. A “ghetto test” by Zynga might not mean that the company fully endorses a game that simulates running a hospital. Similarly, the presence of an item in the Amazon Marketplace might not be enough to presume Amazon’s approval of that item.
[Parts 2-4 will be published over the next week]
- Ben Sobel, Kendra Albert, and JZ
- The Future of the Internet: Five Years Later
In 2008, The Future of the Internet called attention to a “sea change” in the way consumer devices interact with the Internet. “The future is not one of generative PCs attached to a generative network,” the book warns; “it is instead one of sterile appliances tethered to a network of control.” In response to the security threats posed by malicious third-party code, increasing numbers of users will likely gravitate towards gadgets “tethered” by continuous communication between product and vendor. And this proliferation of tethered computing—the “appliancization” of PCs—will deal a serious blow to the principles of generativity and free expression that drove the early Internet.
Since the publication of The Future of the Internet, the ethos of strict appliancization has taken a new turn. In 2011, Professor Zittrain wrote an update on the book’s message: “at the time of the book’s drafting, the alternatives seemed stark: the “sterile” iPhone that ran only Apple’s software on the one hand, and the chaotic PC that ran anything ending in .exe on the other. The iPhone’s openness to outside code beginning in ’08 changed all that. It became what I call “contingently generative” — it runs outside code after approval (and then until it doesn’t).” This trend towards contingently generative models continues into the present day, and represents a shift similar in many respects to the one The Future of the Internet predicted.
Jon Brodkin and Peter Bright’s Ars Technica op-ed on the Microsoft Metro app store offers some valuable commentary on a big development in this “sea change.” The article recognizes that “Microsoft is imitating Apple in one very bad way, by limiting the distribution of Metro applications to a Microsoft-controlled app store… by bringing Windows to tablets, Microsoft could strike a blow for openness in a market dominated by a closed system. Instead, Microsoft is bringing the same restrictions found on iPads to both Windows tablets and PCs.” As forecasted by The Future of the Internet, devices that only run approved code are gaining popularity. Metro, the curated user interface that has found its way onto Microsoft’s tablets and PCs (in the case of the PCs, alongside a fully-functional desktop mode capable of side-loading non-Windows Store applications), won’t run applications from outside the Windows Store. Moreover, the apps available through the Store are subject to a bevy of restrictions on content. With these restrictions on installable applications come the restrictions on generativity that The Future of the Internet anticipated: “lock down the device, and network censorship and control can be extraordinarily reinforced.” And, as the Ars Technica piece observes, the Windows Store’s rules would exclude critically-acclaimed content like the video game Elder Scrolls: Skyrim, simply for its PEGI 18/ESRB M rating. It isn’t hard to extrapolate, as Brodkin and Bright do, that these rules could give rise to debacles similar to Apple’s (repealed) ban of a satire app developed by a Pulitzer Prize winner.
Though the Windows Store’s restrictions resemble Apple’s policies in many ways, there is a crucial difference: Metro-running Windows 8 products are designed as PC replacements, rather than sui generis devices like the iPad. And since Windows desktops have long been preferred gaming platforms, the theoretical exclusion of content like Skyrim from the Windows Store makes Windows 8’s emphasis on the Metro interface particularly jarring.
With Metro, Microsoft has made a decisive move towards contingent generativity. Brodkin and Bright note that “there are security benefits to a closed app store model, particularly for less tech-savvy users who may not understand all the dangers on the Web. There are also, arguably, convenience benefits; end-users can be reasonably confident that the apps they download will work correctly and be at least marginally useful…But while these security and convenience benefits might be enough to justify the existence of a curated app store, they don’t justify the decision to make that store the only option for all users. Informed users should be allowed to install applications from wherever they want.” Brodkin and Bright prefer a system like Gatekeeper, a fixture in newer versions of Apple’s OS X, from Mountain Lion forward. Gatekeeper gives users the choice to restrict their operating system to App Store apps and outside apps that have been signed with Apple-issued Developer IDs, or open up the device to all programs, whether or not they’ve been vetted by Apple. The “Future of the Internet” Blog is fairly enthusiastic about Gatekeeper: about a year ago, a post here suggested that “the middle ground of allowing non-App Store signed code may represent the best of both worlds.” But we were quick to warn that Gatekeeper strikes a tenuous balance: “one small tweak — lose that Control-click for sideloading — and OS X could fully merge with iOS, both in functionality and in security methods.” Metro’s riff on content control could be just that sort of tweak—especially given recent speculation that Microsoft may dump desktop mode in Windows 9, leaving only Metro.
Moreover, a contingently generative business model like the Windows Store’s carries some ethical implications that, while not damning, are certainly worth examining. Distribution systems like the Windows Store, Apple’s App Store, and the Android Market receive 30% of the sales revenue from applications sold in their stores (in the Windows Store, this cut drops to 20% after an app reaches $25,000 USD in revenue). Further restrictions on side-loading in new operating systems would drive a great deal of business towards big companies’ proprietary marketplaces—and with that traffic would come big payouts. With the uptick in store traffic that tighter gatekeeping would engender, it’s easy to imagine the equilibrium of Mac’s OS X Gatekeeper being forsaken for more restrictive, and more lucrative, operating systems. To analogize, a la The Future of the Internet: when the company that makes your computer requires you to install programs through their official store, it isn’t so different from the company that makes your toaster forcing you to buy from their bakery—and taking a cut out of every bread purchase you make.
Even though Windows 8 PC users can still make use of a fully-functioning desktop operating system, Microsoft’s failure to include a side-loading option for the heavily-emphasized Metro interface—particularly in devices marketed as PC replacements—is a step in the wrong direction. It’s also an indication that the seas are changing in the way The Future of the Internet predicted. Given that Android’s more open approach to outside applications[1] still leaves the Android Market increasingly economically viable, Ars Technica is right to voice its disappointment in xenophobic operating systems like iOS and Metro.
- Ben Sobel, Kendra Albert, and JZ
- Rock star RA wanted
I’m seeking a full-time one-year rock star research associate to engage with a variety of projects and classes, with a broad opportunity to immerse in cyberlaw and Internet topics. Blurb below, with more information on how to apply at <http://cyber.law.harvard.edu/getinvolved/jzra>. …JZ
–
Professor Jonathan Zittrain of Harvard Law School, the Harvard Kennedy School of Government, the Harvard School of Engineering and Applied Sciences, and the Berkman Center for Internet & Society, seeks a full-time research associate in Cambridge, MA for a period of one year, beginning no sooner than June 1, 2013.
This position requires the ability to absorb large amounts of written and other media materials from various sources (including but not restricted to: original sources, scholarly articles, news articles/blogs, interviews, databases) in a short amount of time, critically analyze that material and render it forward. This could take the form of prep materials for panels, conferences and presentations; article outlines; fact checking materials; original article or paper drafts; slide decks or other digested forms. The research assistant should be prepared to help prepare materials for class sessions and syllabi, lead discussions and work with project managers to accomplish research-related goals.
Research is often self-directed with little outside guidance beyond broad outlines and themes (though occasional targeted research assignment for a specific fact or image can be expected, and feedback is provided), so the ability to quickly critically appraise sources and identify interesting, relevant and original paths is essential. Wide-ranging interests and the ability to work on almost any issue or topic that arises is a plus, as is an ability to ramp up quickly on unfamiliar fields or topic areas. Excellent writing and editorial skills with an attention to detail are also required.
This job is an ideal opportunity for those interested in future graduate school or law school studies, whether currently admitted or still applying to such programs.
Over the course of the year, a motivated individual will sharpen and focus his or her research agenda and make valuable contributions (in his or her own name) to the field of cyberlaw and beyond, while being exposed to interesting thinkers in academia, industry, and government. A research associate in this position will work very closely with Professor Jonathan Zittrain and his team, assisting in a variety of research areas, e.g. ubiquitous human computing, mesh networking, and cybersecurity, as well as on topics around access to knowledge and open scholarly publishing under the auspices of the Harvard Law School Library.
The position will not start before June 1, 2013. As with all Berkman staff positions, this is a term position, ending June 30, 2014.
- F-T: Don’t sue over tweets
I just published a short piece in the F-T in the wake of legal threats against users who tweeted or retweeted a link to a BBC report of child abuse that turned out to be wrong. Here’s the full text –
Those who didn’t see the false child abuse accusations against Lord Alistair McAlpine on an ill-considered BBC documentary may have instead heard about them through social media. This week, London’s Metropolitan Police suggested they might file charges against those Twitter users who sullied the reputation of the retired Conservative politician by knowingly repeating the lie that he was a child abuser. But the police may be less fearsome to the average BBC-linking tweeter than Lord McAlpine himself. Read more »
- Taking More than Candy from a Baby
Update – 10/17/2012: The parties involved in the lawsuit – Speak for Yourself and SCS/PRC reached a settlement, allowing the app to remain in the Android and iOS app stores. More at the Nieder family blog.
Original Post:
Generativity hasn’t had a poster child — until now.
Meet Maya, a four-year-old child who could lose her ability to speak with the elimination of an app from the iOS App Store.
As detailed in the Nieder family’s original blog post on the subject, Maya uses Speak for Yourself (SfY), an iPad app that serves as an “augmentative and alternative communication” (AAC) device. Before finding SfY, Maya had tried multiple AAC devices, but hadn’t found one that worked for her. Read more »
