• Home
  • About
  • Blog
  • News
  • Events
  • Media
  • Video
  • Glossary
  • Contact
  • Download
  • RSS

Reputation bankruptcy

September 7th, 2010  |  by jz  |  Published in Future of the Internet  |  1 Comment

Google CEO Eric Schmidt created buzz (and some shock and criticism) when he suggested in a recent Wall Street Journal interview that, in the not too distant future, “every young person…will be entitled automatically to change his or her name on reaching adulthood in order to disown youthful hijinks stored on their friends’ social media sites.”

I’ve been intrigued by these concepts, too, and while I don’t think people should have to change their names to escape their pasts — whether earned or unearned — I like the idea of reputation bankruptcy.  It’s taken up as a partial solution to peer-to-peer privacy problems in the Future of the Internet:

Search is central to a functioning Web, and reputation has become central to search. If people already know exactly what they are looking for, a network needs only a way of registering and indexing specific sites. Thus, IP addresses are attached to computers, and domain names to IP addresses, so that we can ask for www.drudgereport.com and go straight to Matt Drudge’s site. But much of the time we want help in finding something without knowing the exact online destination. Search engines help us navigate the petabytes of publicly posted information online, and for them to work well they must do more than simply identify all pages containing the search terms that we specify. They must rank them in relevance. There are many ways to identify what sites are most relevant. A handful of search engines auction off the top-ranked slots in search results on given terms and determine relevance on the basis of how much the site operators would pay to put their sites in front of searchers. These search engines are not widely used. Most have instead turned to some proxy for reputation. As mentioned earlier, a site popular with others—with lots of inbound links—is considered worthier of a high rank than an unpopular one, and thus search engines can draw upon the behavior of millions of other Web sites as they sort their search results. Sites like Amazon deploy a different form of ranking, using the “mouse droppings” of customer purchasing and browsing behavior to make recommendations—so they can tell customers that “people who like the Beatles also like the Rolling Stones.” Search engines can also more explicitly invite the public to express its views on the items it ranks, so that users can decide what to view or buy on the basis of others’ opinions. Amazon users can rate and review the items for sale, and subsequent users then rate the first users’ reviews. Sites like Digg and Reddit invite users to vote for stories and articles they like, and tech news site Slashdot employs a rating system so complex that it attracts much academic attention.

eBay uses reputation to help shoppers find trustworthy sellers. eBay users rate each others’ transactions, and this trail of ratings then informs future buyers how much to trust repeat sellers. These rating systems are crude but powerful. Malicious sellers can abandon poorly rated eBay accounts and sign up for new ones, but fresh accounts with little track record are often viewed skeptically by buyers, especially for proposed transactions involving expensive items. One study confirmed that established identities fare better than new ones, with buyers willing to pay, on average, over 8 percent more for items sold by highly regarded, established sellers. Reputation systems have many pitfalls and can be gamed, but the scholarship seems to indicate that they work reasonably well. There are many ways reputation systems might be improved, but at their core they rely on the number of people rating each other in good faith well exceeding the number of people seeking to game the system—and a way to exclude robots working for the latter. For example, eBay’s rating system has been threatened by the rise of “1-cent eBooks” with no shipping charges; sellers can create alter egos to bid on these nonitems and then have the phantom users highly rate the transaction. One such “feedback farm” earned a seller a thousand positive reviews over four days. eBay intervenes to some extent to eliminate such gaming, just as Google reserves the right to exact the “Google death penalty” by de-listing any Web site that it believes is unduly gaming its chances of a high search engine rating.

These reputation systems now stand to expand beyond evaluating people’s behavior in discrete transactions or making recommendations on products or content, into rating people more generally. This could happen as an extension of current services—as one’s eBay rating is used to determine trustworthiness on, say, another peer-to-peer service. Or, it could come directly from social networking: Cyworld is a social networking site that has twenty million subscribers; it is one of the most popular Internet services in the world, largely thanks to interest in South Korea. The site has its own economy, with $100 million worth of “acorns,” the world’s currency, sold in 2006.

Not only does Cyworld have a financial market, but it also has a market for reputation. Cyworld includes behavior monitoring and rating systems that make it so that users can see a constantly updated score for “sexiness,” “fame,” “friendliness,” “karma,” and “kindness.” As people interact with each other, they try to maximize the kinds of behaviors that augment their ratings in the same way that many Web sites try to figure out how best to optimize their presentation for a high Google ranking. People’s worth is defined and measured precisely, if not accurately, by the reactions of others. That trend is increasing as social networking takes off, partly due to the extension of online social networks beyond the people users already know personally as they “befriend” their friends’ friends’ friends.

The whole-person ratings of social networks like Cyworld will eventually be available in the real world. Similar real-world reputation systems already exist in embryonic form. Law professor Lior Strahilevitz has written a fascinating monograph on the effectiveness of “How’s My Driving” programs, where commercial vehicles are emblazoned with bumper stickers encouraging other drivers to report poor driving. He notes that such programs have resulted in significant accident reductions, and analyzes what might happen if the program were extended to all drivers. A technologically sophisticated version of the scheme dispenses with the need to note a phone number and file a report; one could instead install transponders in every vehicle and distribute TiVo-like remote controls to drivers, cyclists, and pedestrians. If someone acts politely, say by allowing you to switch lanes, you can acknowledge it with a digital thumbsup that is recorded on that driver’s record. Cutting someone off in traffic earns a thumbs-down from the victim and other witnesses. Strahilevitz is supportive of such a scheme, and he surmises it could be even more effective than eBay’s ratings for online transactions since vehicles are registered by the government, making it far more difficult escape poor ratings tied to one’s vehicle. He acknowledges some worries: people could give thumbs-down to each other for reasons unrelated to their driving—racism, for example. Perhaps a bumper sticker expressing support for Republicans would earn a thumbs-down in a blue state. Strahilevitz counters that the reputation system could be made to eliminate “outliers”—so presumably only well-ensconced racism across many drivers would end up affecting one’s ratings. According to Strahilevitz, this system of peer judgment would pass constitutional muster if challenged, even if the program is run by the state, because driving does not implicate one’s core rights. “How’s My Driving?” systems are too minor to warrant extensive judicial review. But driving is only the tip of the iceberg.

Imagine entering a café in Paris with one’s personal digital assistant or mobile phone, and being able to query: “Is there anyone on my buddy list within 100 yards? Are any of the ten closest friends of my ten closest friends within 100 yards?” Although this may sound fanciful, it could quickly become mainstream. With reputation systems already advising us on what to buy, why not have them also help us make the first cut on whom to meet, to date, to befriend? These are not difficult services to offer, and there are precursors today. These systems can indicate who has not offered evidence that he or she is safe to meet—as is currently solicited by some online dating sites—or it may use Amazon-style matching to tell us which of the strangers who have just entered the café is a good match for people who have the kinds of friends we do. People can rate their interactions with each other (and change their votes later, so they can show their companion a thumbs-up at the time of the meeting and tell the truth later on), and those ratings will inform future suggested acquaintances. With enough people adopting the system, the act of entering a café can be different from one person to the next: for some, the patrons may shrink away, burying their heads deeper in their books and newspapers. For others, the entire café may perk up upon entrance, not knowing who it is but having a lead that this is someone worth knowing. Those who do not participate in the scheme at all will be as suspect as brand new buyers or sellers on eBay.

Increasingly, difficult-to-shed indicators of our identity will be recorded and captured as we go about our daily lives and enter into routine transactions— our fingerprints may be used to log in to our computers or verify our bank accounts, our photo may be snapped and tagged many times a day, or our license plate may be tracked as people judge our driving habits. The more our identity is associated with our daily actions, the greater opportunities others will have to offer judgments about those actions. A government-run system like the one Strahilevitz recommends for assessing driving is the easy case. If the state is the record keeper, it is possible to structure the system so that citizens can know the basis of their ratings—where (if not by whom) various thumbs-down clicks came from—and the state can give a chance for drivers to offer an explanation or excuse, or to follow up. The state’s formula for meting out fines or other penalties to poor drivers would be known (“three strikes and you’re out,” for whatever other problems it has, is an eminently transparent scheme), and it could be adjusted through accountable processes, just as legislatures already determine what constitutes an illegal act, and what range of punishment it should earn.

Generatively grown but comprehensively popular unregulated systems are a much trickier case. The more that we rely upon the judgments offered by these private systems, the more harmful that mistakes can be. Correcting or identifying mistakes can be difficult if the systems are operated entirely by private parties and their ratings formulas are closely held trade secrets. Search engines are notoriously resistant to discussing how their rankings work, in part to avoid gaming—a form of security through obscurity. The most popular engines reserve the right to intervene in their automatic rankings processes—to administer the Google death penalty, for example—but otherwise suggest that they do not centrally adjust results. Hence a search in Google for “Jew” returns an anti- Semitic Web site as one of its top hits, as well as a separate sponsored advertisement from Google itself explaining that its rankings are automatic. But while the observance of such policies could limit worries of bias to search algorithm design rather than to the case-by-case prejudices of search engine operators, it does not address user-specific bias that may emerge from personalized judgments.

Amazon’s automatic recommendations also make mistakes; for a period of time the Official Lego Creator Activity Book was paired with a “perfect partner” suggestion: American Jihad: The Terrorists Living Among Us Today. If such mismatched pairings happen when discussing people rather than products, rare mismatches could have worse effects while being less noticeable since they are not universal. The kinds of search systems that say which people are worth getting to know and which should be avoided, tailored to the users querying the system, present a set of due process problems far more complicated than a stateoperated system or, for that matter, any system operated by a single party. The generative capacity to share data and to create mash-ups means that ratings and rankings can be far more emergent—and far more inscrutable.

As biometric readers become more commonplace in our endpoint machines, it will be possible for online destinations routinely to demand unsheddable identity tokens rather than disposable pseudonyms from Internet users. Many sites could benefit from asking people to participate with real identities known at least to the site, if not to the public at large. eBay, for one, would certainly profit by making it harder for people to shift among various ghost accounts. One could even imagine Wikipedia establishing a “fast track” for contributions if they were done with biometric assurance, just as South Korean citizen journalist newspaper OhmyNews keeps citizen identity numbers on file for the articles it publishes. These architectures protect one’s identity from the world at large while still making it much more difficult to produce multiple false “sock puppet” identities. When we participate in other walks of life—school, work, PTA meetings, and so on—we do so as ourselves, not wearing Groucho mustaches, and even if people do not know exactly who we are, they can recognize us from one meeting to the next. The same should be possible for our online selves. []

As real identity grows in importance on the Net, the intermediaries demanding it ought to consider making available a form of reputation bankruptcy. Like personal financial bankruptcy, or the way in which a state often seals a juvenile criminal record and gives a child a “fresh start” as an adult, we ought to consider how to implement the idea of a second or third chance into our digital spaces. People ought to be able to express a choice to de-emphasize if not entirely delete older information that has been generated about them by and through various systems: political preferences, activities, youthful likes and dislikes. If every action ends up on one’s “permanent record,” the press conference effect can set in. Reputation bankruptcy has the potential to facilitate desirably experimental social behavior and break up the monotony of static communities online and offline. As a safety valve against excess experimentation, perhaps the information in one’s record could not be deleted selectively; if someone wants to declare reputation bankruptcy, we might want it to mean throwing out the good along with the bad. The blank spot in one’s history indicates a bankruptcy has been declared—this would be the price one pays for eliminating unwanted details.

The key is to realize that we can make design choices now that work to capture the nuances of human relations far better than our current systems, and that online intermediaries might well embrace such new designs even in the absence of a legal mandate to do so.

(And, as long as we’re talking about reputation — you can check out Dan Solove’s excellent book on the future of reputation here.)

Responses

Feed
  1. Digital Culture Links: September 8th 2010 | Tama Leaver dot Net says:

    September 8th, 2010 at 2:02 am (#)

    [...] Reputation bankruptcy :[The Future of the Internet — And How to Stop It] – Should we be able to purge our online reputation record and declare reputation bankruptcy? Jonathan Zittrain: “As real identity grows in importance on the Net, the intermediaries demanding it ought to consider making available a form of reputation bankruptcy. Like personal financial bankruptcy, or the way in which a state often seals a juvenile criminal record and gives a child a “fresh start” as an adult, we ought to consider how to implement the idea of a second or third chance into our digital spaces. People ought to be able to express a choice to de-emphasize if not entirely delete older information that has been generated about them by and through various systems: political preferences, activities, youthful likes and dislikes. If every action ends up on one’s “permanent record,” the press conference effect can set in. Reputation bankruptcy has the potential to facilitate desirably experimental social behavior and break up the monotony of static communities online and offline.” [...]

Blog

  • The Future of the Internet: Five Years Later
  • In 2008, The Future of the Internet called attention to a “sea change” in the way consumer devices interact with the Internet. “The future is not one of generative PCs attached to a generative network,” the book warns; “it is instead one of sterile appliances tethered to a network of control.” In response to the security threats posed by malicious third-party code, increasing numbers of users will likely gravitate towards gadgets “tethered” by continuous communication between product and vendor. And this proliferation of tethered computing—the “appliancization” of PCs—will deal a serious blow to the principles of generativity and free expression that drove the early Internet.

    Since the publication of The Future of the Internet, the ethos of strict appliancization has taken a new turn. In 2011, Professor Zittrain wrote an update on the book’s message: “at the time of the book’s drafting, the alternatives seemed stark: the “sterile” iPhone that ran only Apple’s software on the one hand, and the chaotic PC that ran anything ending in .exe on the other. The iPhone’s openness to outside code beginning in ’08 changed all that. It became what I call “contingently generative” — it runs outside code after approval (and then until it doesn’t).” This trend towards contingently generative models continues into the present day, and represents a shift similar in many respects to the one The Future of the Internet predicted.

    Jon Brodkin and Peter Bright’s Ars Technica op-ed on the Microsoft Metro app store offers some valuable commentary on a big development in this “sea change.” The article recognizes that “Microsoft is imitating Apple in one very bad way, by limiting the distribution of Metro applications to a Microsoft-controlled app store… by bringing Windows to tablets, Microsoft could strike a blow for openness in a market dominated by a closed system. Instead, Microsoft is bringing the same restrictions found on iPads to both Windows tablets and PCs.” As forecasted by The Future of the Internet, devices that only run approved code are gaining popularity. Metro, the curated user interface that has found its way onto Microsoft’s tablets and PCs (in the case of the PCs, alongside a fully-functional desktop mode capable of side-loading non-Windows Store applications), won’t run applications from outside the Windows Store. Moreover, the apps available through the Store are subject to a bevy of restrictions on content. With these restrictions on installable applications come the restrictions on generativity that The Future of the Internet anticipated: “lock down the device, and network censorship and control can be extraordinarily reinforced.” And, as the Ars Technica piece observes, the Windows Store’s rules would exclude critically-acclaimed content like the video game Elder Scrolls: Skyrim, simply for its PEGI 18/ESRB M rating. It isn’t hard to extrapolate, as Brodkin and Bright do, that these rules could give rise to debacles similar to Apple’s (repealed) ban of a satire app developed by a Pulitzer Prize winner.

    Though the Windows Store’s restrictions resemble Apple’s policies in many ways, there is a crucial difference: Metro-running Windows 8 products are designed as PC replacements, rather than sui generis devices like the iPad. And since Windows desktops have long been preferred gaming platforms, the theoretical exclusion of content like Skyrim from the Windows Store makes Windows 8’s emphasis on the Metro interface particularly jarring.

    With Metro, Microsoft has made a decisive move towards contingent generativity. Brodkin and Bright note that “there are security benefits to a closed app store model, particularly for less tech-savvy users who may not understand all the dangers on the Web. There are also, arguably, convenience benefits; end-users can be reasonably confident that the apps they download will work correctly and be at least marginally useful…But while these security and convenience benefits might be enough to justify the existence of a curated app store, they don’t justify the decision to make that store the only option for all users. Informed users should be allowed to install applications from wherever they want.” Brodkin and Bright prefer a system like Gatekeeper, a fixture in newer versions of Apple’s OS X, from Mountain Lion forward. Gatekeeper gives users the choice to restrict their operating system to App Store apps and outside apps that have been signed with Apple-issued Developer IDs, or open up the device to all programs, whether or not they’ve been vetted by Apple. The “Future of the Internet” Blog is fairly enthusiastic about Gatekeeper: about a year ago, a post here suggested that “the middle ground of allowing non-App Store signed code may represent the best of both worlds.” But we were quick to warn that Gatekeeper strikes a tenuous balance: “one small tweak — lose that Control-click for sideloading — and OS X could fully merge with iOS, both in functionality and in security methods.” Metro’s riff on content control could be just that sort of tweak—especially given recent speculation that Microsoft may dump desktop mode in Windows 9, leaving only Metro.

    Moreover, a contingently generative business model like the Windows Store’s carries some ethical implications that, while not damning, are certainly worth examining. Distribution systems like the Windows Store, Apple’s App Store, and the Android Market receive 30% of the sales revenue from applications sold in their stores (in the Windows Store, this cut drops to 20% after an app reaches $25,000 USD in revenue). Further restrictions on side-loading in new operating systems would drive a great deal of business towards big companies’ proprietary marketplaces—and with that traffic would come big payouts. With the uptick in store traffic that tighter gatekeeping would engender, it’s easy to imagine the equilibrium of Mac’s OS X Gatekeeper being forsaken for more restrictive, and more lucrative, operating systems. To analogize, a la The Future of the Internet: when the company that makes your computer requires you to install programs through their official store, it isn’t so different from the company that makes your toaster forcing you to buy from their bakery—and taking a cut out of every bread purchase you make.

    Even though Windows 8 PC users can still make use of a fully-functioning desktop operating system, Microsoft’s failure to include a side-loading option for the heavily-emphasized Metro interface—particularly in devices marketed as PC replacements—is a step in the wrong direction. It’s also an indication that the seas are changing in the way The Future of the Internet predicted. Given that Android’s more open approach to outside applications[1] still leaves the Android Market increasingly economically viable, Ars Technica is right to voice its disappointment in xenophobic operating systems like iOS and Metro.

    - Ben Sobel, Kendra Albert, and JZ

    [1] Though the Google Play approach to openness is far from perfect! Ad-Blocking apps were recently pulled from the Play Store, in a move that will come to illustrate just how viable it is to distribute a side-loaded Android app without any help from the Play Store.

  • Rock star RA wanted
  • I’m seeking a full-time one-year rock star research associate to engage with a variety of projects and classes, with a broad opportunity to immerse in cyberlaw and Internet topics.   Blurb below, with more information on how to apply at <http://cyber.law.harvard.edu/getinvolved/jzra>.  …JZ

    –

    Professor Jonathan Zittrain of Harvard Law School, the Harvard Kennedy School of Government, the Harvard School of Engineering and Applied Sciences, and the Berkman Center for Internet & Society, seeks a full-time research associate in Cambridge, MA for a period of one year, beginning no sooner than June 1, 2013.

    This position requires the ability to absorb large amounts of written and other media materials from various sources (including but not restricted to: original sources, scholarly articles, news articles/blogs, interviews, databases) in a short amount of time, critically analyze that material and render it forward. This could take the form of prep materials for panels, conferences and presentations; article outlines; fact checking materials; original article or paper drafts; slide decks or other digested forms. The research assistant should be prepared to help prepare materials for class sessions and syllabi, lead discussions and work with project managers to accomplish research-related goals.

    Research is often self-directed with little outside guidance beyond broad outlines and themes (though occasional targeted research assignment for a specific fact or image can be expected, and feedback is provided), so the ability to quickly critically appraise sources and identify interesting, relevant and original paths is essential. Wide-ranging interests and the ability to work on almost any issue or topic that arises is a plus, as is an ability to ramp up quickly on unfamiliar fields or topic areas. Excellent writing and editorial skills with an attention to detail are also required.

    This job is an ideal opportunity for those interested in future graduate school or law school studies, whether currently admitted or still applying to such programs.

    Over the course of the year, a motivated individual will sharpen and focus his or her research agenda and make valuable contributions (in his or her own name) to the field of cyberlaw and beyond, while being exposed to interesting thinkers in academia, industry, and government. A research associate in this position will work very closely with Professor Jonathan Zittrain and his team, assisting in a variety of research areas, e.g. ubiquitous human computing, mesh networking, and cybersecurity, as well as on topics around access to knowledge and open scholarly publishing under the auspices of the Harvard Law School Library.

    The position will not start before June 1, 2013.  As with all Berkman staff positions, this is a term position, ending June 30, 2014.

  • F-T: Don’t sue over tweets
  • I just published a short piece in the F-T in the wake of legal threats against users who tweeted or retweeted a link to a BBC report of child abuse that turned out to be wrong.  Here’s the full text –

    Those who didn’t see the false child abuse accusations against Lord Alistair McAlpine on an ill-considered BBC documentary may have instead heard about them through social media. This week, London’s Metropolitan Police suggested they might file charges against those Twitter users who sullied the reputation of the retired Conservative politician by knowingly repeating the lie that he was a child abuser. But the police may be less fearsome to the average BBC-linking tweeter than Lord McAlpine himself. Read more »

  • Taking More than Candy from a Baby
  • Update – 10/17/2012: The parties involved in the lawsuit – Speak for Yourself and SCS/PRC reached a settlement, allowing the app to remain in the Android and iOS app stores. More at the Nieder family blog.

    Original Post:

    Generativity hasn’t had a poster child — until now.

    Meet Maya, a four-year-old child who could lose her ability to speak with the elimination of an app from the iOS App Store.

    As detailed in the Nieder family’s original blog post on the subject, Maya uses Speak for Yourself (SfY), an iPad app that serves as an “augmentative and alternative communication” (AAC) device. Before finding SfY, Maya had tried multiple AAC devices, but hadn’t found one that worked for her. Read more »

  • “Unabomber manifesto tied to tech news headlines”
  • When you see the headline “Powerful ‘Flame’ cyberweapon tied to popular Angry Birds game,” does it cause you to think that there is actually some connnection between the recently discovered malware Flame and Angry Birds? That would be entirely reasonable, but wrong. Read more »

About Jonathan Zittrain

jonathan zittrain

Jonathan Zittrain is Professor of Law at Harvard Law School and co-founder of the Berkman Center for Internet and Society at Harvard Law School

RSS Tweets from Z

  • An error has occurred, which probably means the feed is down. Try again later.

Blog Archives



Creative Commons BY-NC-SA Jonathan Zittrain unless otherwise noted.
Powered by WordPress using Gridline Lite.