- Rethinking Online Culpability: The Amazon “Keep Calm” Shirts Controversy (Part 1: A/B Testing)
In early March, the online retailer Solid Gold Bomb provoked outrage when customers discovered that its Amazon store, which featured apparel bearing dozens of variants on the “Keep Calm [and Carry On]” slogan, included a t-shirt that read “Keep Calm and Rape A Lot.” Solid Gold Bomb generated the shirts, and Amazon offered them for sale in its marketplace. To complicate matters, it appears that Amazon doesn’t review the stores in its marketplace like a mall owner might review physical storefronts, and, particularly unusual, Solid Gold Bomb didn’t review the shirts it offered for sale: the designs were computer generated. How far, then, should blame extend? When unsupervised automation produces results that everyone regrets, how do we decide whom to hold responsible, and when do we decide to hold anyone responsible in the first place?
Solid Gold Bomb’s official apology explained that its Amazon store featured millions of hypothetical shirts to be produced on-demand, should anyone order one. The “Keep Calm” debacle resulted from an automated script that generated words to approximately fit the design’s syntax and layout. The resulting list, says SGB owner Michael Fowler, “was culled from 202k words to around 1100 and ultimately slightly more than 700 were used due to character length and the fact that I wanted to closely reflect the appearance of the original slogan graphically.” Clearly, the vendor is at fault for failing to eliminate possible ending phrases to the Keep Calm slogan like “rape a lot” and “choke her” from a 700-word list. However, similarly automated practices regularly take place on a much larger scale across the internet. Determining accountability for these widespread and fundamental operations can be much less straightforward.
In some ways, Solid Gold Bomb’s generation of the offensive shirts can be seen merely as A/B testing gone awry. Offering thousands of options and printing shirts to order is a way of using user behavior to cull successful products. Presumably, if one of the quasi-randomly-generated shirts began to outstrip the others in sales, Solid Gold Bomb would have adjusted its inventory and marketing accordingly.
With A/B testing, the line between savvy capitalism and unethical business practice can get fairly nebulous. Zynga, for example, relies on a practice that CEO Mark Pincus calls “ghetto testing.” One of Zynga’s approaches to game development is to advertise games that do not yet exist, in order to test consumer response to a basic premise. Says Pincus,
“We’ll put up a link for five minutes saying, ‘Hey! Do you ever fantasize about running your own hospital?’…We’ll put that up for five minutes, and the link will maybe take you to a survey, where you give us your email and we say when this comes out we’ll contact you. If you’re really doing ghetto, it says ‘404 not found’. That’s bad. So first you try to get the heat around it, you see how much do people like it, then…”
This isn’t all that dissimilar to Solid Gold Bomb’s approach. Like Zynga’s “ghetto-tested” games, the “Rape a Lot” shirts didn’t actually exist, and would only have been produced in accordance with user demand. In fact, Solid Gold Bomb didn’t misdirect potential buyers as deliberately as Zynga’s “ghetto testing” approach does.
In large, computer-conducted A/B testing campaigns, it becomes impossible to demand human supervision of every output. Solid Gold Bomb’s 700-word list for generating T-shirts should have been thoroughly scrutinized, of course, but operations with more permutations of A’s and B’s seem less accountable for each potential outcome. For example, it’s hard to believe it would be within a webmaster’s responsibility—or even her ability—to make sure that every possible banner ad on every single page of a site doesn’t combine unfortunately with the page’s content.
A/B testing is practically ubiquitous online, and most of its applications are unequivocally benign. Wikipedia, for one, famously self-published the test results of its 2010 fundraising push. Moreover, unsupervised, computer-conducted A/B testing can produce serendipitous results that no human could ever have engineered or anticipated. The popular twitter handle @horse_ebooks, for example, began as a poorly functioning spam account intended to drive traffic to an e-book site. But its garbled messages are so striking—and occasionally poignant (cf. a recent example)—that the bot currently has over 170,000 followers.
The problem, then, is that our expectations for internet commerce haven’t quite caught up with the techniques that drive internet commerce. If a store offers things for sale that we find offensive, our typical reaction is to get mad at the store—after all, being willing to profit off an item seems to imply some kind of endorsement of that item. Today, however, these assumptions about endorsement are challenged by the ubiquity of A/B testing and other automated content generators. A “ghetto test” by Zynga might not mean that the company fully endorses a game that simulates running a hospital. Similarly, the presence of an item in the Amazon Marketplace might not be enough to presume Amazon’s approval of that item.
[Parts 2-4 will be published over the next week]
- Ben Sobel, Kendra Albert, and JZ
- The Future of the Internet: Five Years Later
In 2008, The Future of the Internet called attention to a “sea change” in the way consumer devices interact with the Internet. “The future is not one of generative PCs attached to a generative network,” the book warns; “it is instead one of sterile appliances tethered to a network of control.” In response to the security threats posed by malicious third-party code, increasing numbers of users will likely gravitate towards gadgets “tethered” by continuous communication between product and vendor. And this proliferation of tethered computing—the “appliancization” of PCs—will deal a serious blow to the principles of generativity and free expression that drove the early Internet.
Since the publication of The Future of the Internet, the ethos of strict appliancization has taken a new turn. In 2011, Professor Zittrain wrote an update on the book’s message: “at the time of the book’s drafting, the alternatives seemed stark: the “sterile” iPhone that ran only Apple’s software on the one hand, and the chaotic PC that ran anything ending in .exe on the other. The iPhone’s openness to outside code beginning in ’08 changed all that. It became what I call “contingently generative” — it runs outside code after approval (and then until it doesn’t).” This trend towards contingently generative models continues into the present day, and represents a shift similar in many respects to the one The Future of the Internet predicted.
Jon Brodkin and Peter Bright’s Ars Technica op-ed on the Microsoft Metro app store offers some valuable commentary on a big development in this “sea change.” The article recognizes that “Microsoft is imitating Apple in one very bad way, by limiting the distribution of Metro applications to a Microsoft-controlled app store… by bringing Windows to tablets, Microsoft could strike a blow for openness in a market dominated by a closed system. Instead, Microsoft is bringing the same restrictions found on iPads to both Windows tablets and PCs.” As forecasted by The Future of the Internet, devices that only run approved code are gaining popularity. Metro, the curated user interface that has found its way onto Microsoft’s tablets and PCs (in the case of the PCs, alongside a fully-functional desktop mode capable of side-loading non-Windows Store applications), won’t run applications from outside the Windows Store. Moreover, the apps available through the Store are subject to a bevy of restrictions on content. With these restrictions on installable applications come the restrictions on generativity that The Future of the Internet anticipated: “lock down the device, and network censorship and control can be extraordinarily reinforced.” And, as the Ars Technica piece observes, the Windows Store’s rules would exclude critically-acclaimed content like the video game Elder Scrolls: Skyrim, simply for its PEGI 18/ESRB M rating. It isn’t hard to extrapolate, as Brodkin and Bright do, that these rules could give rise to debacles similar to Apple’s (repealed) ban of a satire app developed by a Pulitzer Prize winner.
Though the Windows Store’s restrictions resemble Apple’s policies in many ways, there is a crucial difference: Metro-running Windows 8 products are designed as PC replacements, rather than sui generis devices like the iPad. And since Windows desktops have long been preferred gaming platforms, the theoretical exclusion of content like Skyrim from the Windows Store makes Windows 8’s emphasis on the Metro interface particularly jarring.
With Metro, Microsoft has made a decisive move towards contingent generativity. Brodkin and Bright note that “there are security benefits to a closed app store model, particularly for less tech-savvy users who may not understand all the dangers on the Web. There are also, arguably, convenience benefits; end-users can be reasonably confident that the apps they download will work correctly and be at least marginally useful…But while these security and convenience benefits might be enough to justify the existence of a curated app store, they don’t justify the decision to make that store the only option for all users. Informed users should be allowed to install applications from wherever they want.” Brodkin and Bright prefer a system like Gatekeeper, a fixture in newer versions of Apple’s OS X, from Mountain Lion forward. Gatekeeper gives users the choice to restrict their operating system to App Store apps and outside apps that have been signed with Apple-issued Developer IDs, or open up the device to all programs, whether or not they’ve been vetted by Apple. The “Future of the Internet” Blog is fairly enthusiastic about Gatekeeper: about a year ago, a post here suggested that “the middle ground of allowing non-App Store signed code may represent the best of both worlds.” But we were quick to warn that Gatekeeper strikes a tenuous balance: “one small tweak — lose that Control-click for sideloading — and OS X could fully merge with iOS, both in functionality and in security methods.” Metro’s riff on content control could be just that sort of tweak—especially given recent speculation that Microsoft may dump desktop mode in Windows 9, leaving only Metro.
Moreover, a contingently generative business model like the Windows Store’s carries some ethical implications that, while not damning, are certainly worth examining. Distribution systems like the Windows Store, Apple’s App Store, and the Android Market receive 30% of the sales revenue from applications sold in their stores (in the Windows Store, this cut drops to 20% after an app reaches $25,000 USD in revenue). Further restrictions on side-loading in new operating systems would drive a great deal of business towards big companies’ proprietary marketplaces—and with that traffic would come big payouts. With the uptick in store traffic that tighter gatekeeping would engender, it’s easy to imagine the equilibrium of Mac’s OS X Gatekeeper being forsaken for more restrictive, and more lucrative, operating systems. To analogize, a la The Future of the Internet: when the company that makes your computer requires you to install programs through their official store, it isn’t so different from the company that makes your toaster forcing you to buy from their bakery—and taking a cut out of every bread purchase you make.
Even though Windows 8 PC users can still make use of a fully-functioning desktop operating system, Microsoft’s failure to include a side-loading option for the heavily-emphasized Metro interface—particularly in devices marketed as PC replacements—is a step in the wrong direction. It’s also an indication that the seas are changing in the way The Future of the Internet predicted. Given that Android’s more open approach to outside applications[1] still leaves the Android Market increasingly economically viable, Ars Technica is right to voice its disappointment in xenophobic operating systems like iOS and Metro.
- Ben Sobel, Kendra Albert, and JZ
- Rock star RA wanted
I’m seeking a full-time one-year rock star research associate to engage with a variety of projects and classes, with a broad opportunity to immerse in cyberlaw and Internet topics. Blurb below, with more information on how to apply at <http://cyber.law.harvard.edu/getinvolved/jzra>. …JZ
–
Professor Jonathan Zittrain of Harvard Law School, the Harvard Kennedy School of Government, the Harvard School of Engineering and Applied Sciences, and the Berkman Center for Internet & Society, seeks a full-time research associate in Cambridge, MA for a period of one year, beginning no sooner than June 1, 2013.
This position requires the ability to absorb large amounts of written and other media materials from various sources (including but not restricted to: original sources, scholarly articles, news articles/blogs, interviews, databases) in a short amount of time, critically analyze that material and render it forward. This could take the form of prep materials for panels, conferences and presentations; article outlines; fact checking materials; original article or paper drafts; slide decks or other digested forms. The research assistant should be prepared to help prepare materials for class sessions and syllabi, lead discussions and work with project managers to accomplish research-related goals.
Research is often self-directed with little outside guidance beyond broad outlines and themes (though occasional targeted research assignment for a specific fact or image can be expected, and feedback is provided), so the ability to quickly critically appraise sources and identify interesting, relevant and original paths is essential. Wide-ranging interests and the ability to work on almost any issue or topic that arises is a plus, as is an ability to ramp up quickly on unfamiliar fields or topic areas. Excellent writing and editorial skills with an attention to detail are also required.
This job is an ideal opportunity for those interested in future graduate school or law school studies, whether currently admitted or still applying to such programs.
Over the course of the year, a motivated individual will sharpen and focus his or her research agenda and make valuable contributions (in his or her own name) to the field of cyberlaw and beyond, while being exposed to interesting thinkers in academia, industry, and government. A research associate in this position will work very closely with Professor Jonathan Zittrain and his team, assisting in a variety of research areas, e.g. ubiquitous human computing, mesh networking, and cybersecurity, as well as on topics around access to knowledge and open scholarly publishing under the auspices of the Harvard Law School Library.
The position will not start before June 1, 2013. As with all Berkman staff positions, this is a term position, ending June 30, 2014.
- F-T: Don’t sue over tweets
I just published a short piece in the F-T in the wake of legal threats against users who tweeted or retweeted a link to a BBC report of child abuse that turned out to be wrong. Here’s the full text –
Those who didn’t see the false child abuse accusations against Lord Alistair McAlpine on an ill-considered BBC documentary may have instead heard about them through social media. This week, London’s Metropolitan Police suggested they might file charges against those Twitter users who sullied the reputation of the retired Conservative politician by knowingly repeating the lie that he was a child abuser. But the police may be less fearsome to the average BBC-linking tweeter than Lord McAlpine himself. Read more »
- Taking More than Candy from a Baby
Update – 10/17/2012: The parties involved in the lawsuit – Speak for Yourself and SCS/PRC reached a settlement, allowing the app to remain in the Android and iOS app stores. More at the Nieder family blog.
Original Post:
Generativity hasn’t had a poster child — until now.
Meet Maya, a four-year-old child who could lose her ability to speak with the elimination of an app from the iOS App Store.
As detailed in the Nieder family’s original blog post on the subject, Maya uses Speak for Yourself (SfY), an iPad app that serves as an “augmentative and alternative communication” (AAC) device. Before finding SfY, Maya had tried multiple AAC devices, but hadn’t found one that worked for her. Read more »
June 24th, 2009 at 10:33 pm (#)
JZ,
Last time I checked, the Twitter API does not limit to 140 characters, but the different Twitter apps behave differently when you break the limit.
But busting past the limit is only the tip of the iceberg for what should be done with status/messaging communications:
see: http://civilities.net/Semantic_Social_Media_Construction
Jon
June 24th, 2009 at 11:01 pm (#)
I see where you’re going with your idea; communication is less effective when stilted, repurposed and RT’d. As you note, Twitter has a hard stop at 140 characters. That limit may seem somewhat capricious, but it is, I believe, its killer feature.
Limiting people to 140 characters tends to compel them to condense their thoughts—sometimes to a density of a neutron star. Obviously, this type of communication is easily stretched to the breaking point, especially when discussing deep, important topics. However, it does encourage people to just throw something out there rather than cower and lurk. It limits risk while creating a parameter that people can easily comprehend. Say something in 140 chars or less. I can do that; you can do that, evidently even cats and dogs can do that. Simple.
It’s what good teachers do in order to get you to think: provide you with an endpoint. Speaking as a designer and a writer, I understand that knowing where the box ends allows you to think outside it. Constraints = creativity.
So, let’s not turn Twitter into something it’s not, but rather, as you say, let’s appreciate it as the building block it is. Entire industries may and have spring up around it. But let’s keep the original around without modification so that the small-voiced may continue to speak.
@carlacasilli
June 25th, 2009 at 1:13 pm (#)
JZ,
If I can summarize the work I did with Semantic Social Media Construction to make it *much* simpler.
There are basically two types of tweets/status-messages:
Temporal and Conversational.
The temporal ones are often of the nature of someone announcing that they are somewhere, or are going somewhere, or are reporting exclusive news. These are the sorts of things that should fit under 140 characters and should well go to the mobile phones.
The conversational ones are everything else: responses, retweets, URL finds, ongoing live-tweeting… there is no real reason to broadcast these over SMS to followers, so the 140-limit is unnecessarily constrictive.
Twitter needs to figure this out, because Facebook generally has mastered the conversational messages.
Jon
June 26th, 2009 at 7:18 am (#)
Twitter itself does have a special “RT” functionality; it’s just two letters that many people have come to use to say that they’re repeating something they saw elsewhere — and giving attribution for it.
I think you mean Twitter *doesn’t* have a retweet function…
June 26th, 2009 at 8:08 am (#)
[...] boyd’s view on the communication phenomena of retweeting and Jonathan Zittrain’s thoughts on technical 140 characters barriers on Twitter. Having in mind that retweeting process is one of the conversation practices on Twitter, the [...]
June 26th, 2009 at 11:15 am (#)
Very important points here. The URL-shortening issue alone I think demands special attention from The Twitterlords, for it makes the interwebs too dependent on go-betweens. Twitter would remain appropriately minimal if it were to take a “140 characters + a link” approach.
June 27th, 2009 at 8:59 am (#)
JZ,
Keep it as it is, short and focused.
Han
June 29th, 2009 at 3:04 pm (#)
I agree with your thoughts of adding meta-data for retweets, building them into the application makes sense at this point and would free us to make more of them – which is one of the greatest intents of the service: to share ideas. I don’t consider that a very major change and not a true breaking of the 140 char limit (which I like – if you have more to say, write a blog and post the link ;)).
As far as the URL shorteners, I believe that they will be used regardless of char limits, on Twitter or else wher. Furthermore, many of the issues with them can (and should) be addressed by the shortening services or through browsers – preview panes, etc.