A British insurance company called Motaquote has teamed up with TomTom, the GPS manufacturer to offer insurance prices based on data gathered by GPS. Fair Pay Insurance, Motaquote’s new program, is an opt-in insurance pricing scheme where drivers will get a free GPS unit in return for potentially lower (but possibly higher) premiums. The GPS unit will provide all the traditional navigational services as well as warn drivers when they corner too sharply or brake too hard.

Data-driven insurance pricing is nothing new. Research into the viability of GPS-based pricing goes as far back as 2003. Telemetry based insurance premiums have been around in the United States since at least 2008, when Progressive started using “Snapshot.” It measured numbers of miles driven, time of day and sudden stops to set rates rewarding less hazardous driving. However, Progressive’s Snapshot program did not integrate GPS data – it used a device that plugged into the car’s OnBoard Diagnostic port. And companies with large vehicle fleets have used it to track driver safety — an automated answer to “How’s My Driving?” through such services as SmartDrive.

It’s worth noting that one of the reasons that Motaquote may be moving towards telemetric data is that the European Court of Justice ruled  last year that car insurance rates based on gender were discriminatory. Insurance companies, not known as keen innovators, have been prompted to find new ways to distinguish good drivers from dangerous drivers, and telemetric data is perhaps more accurate than generalizing based on age or gender.

Still, the move to GPS-based calculation is sensitive. One reason that programs like Progressive’s have been uncontroversial is that they have not sent locational information to car insurance providers, and users have an option to view their data before opting-in to rates based upon it.  Although the Fair Pay system was just announced, its website suggests that drivers will opt-in before they can see how their driving will measure up.

Data privacy is also a serious concern. Will insurance companies be asked to turn over GPS data to law enforcement to show where a car (or driver) was at a specific time?  There’s currently no information on Fair Pay’s website about the privacy of the locational aspects of the data – something that anyone who wants a free GPS should consider first. Such policies don’t have to say “always” or “never” — but they should spell out the standards by which a company will respond to requests or demands for information, and more generally policymakers should set up standards to shield privacy from unwarranted intrusion, metaphorically and literally.

Telemetric data-based rates also mean that insurance-holders may be locked into specific providers. If a driver’s rate is based on years of good driving data, but that information is non-transferrable, he or she may not be able to switch insurance providers without a substantial rate hike. It would be better if insurance companies provided customers with options for data portability and download.  That would also help drivers make sure that insurance company rate changes were justified — instead of being told “Sorry, poor driving means you pay a higher rate,” without more, drivers could float their data to other insurance companies who could bid lower for the driver’s insurance account if it didn’t truly indicate poor driving.  That’s real competition, and it would provide the right incentives to insurance companies to refine their algorithms.  A sudden stop might actually indicate good driving — rabbit dashes in front of car and driver shows admirable reflexes.

There’s also some intriguing possibilities for complementary crowdsourcing of driver safety.  Lior Strahilevitz has written about this using the standard “How’s my driving” toll free number system, and I’ve mused on it for a fully saturating Internet environment.  I also weighed in for Marketplace Tech Report.

Fundamentally, there’s nothing wrong with GPS-based telemetric data setting insurance rates. Consumers have become more comfortable with locational tracking, and these types of plan are currently inherently opt-in. If the programs do indeed reward better drivers, then they can make driving (and walking or biking nearby) safer. The devil is in the details of how the data is collected, what companies do with it, and how consumers can access and use it.

–JZ and KA

This semester, we’re starting an exciting new class, aimed not at lawyers, but undergraduate CS students here at Harvard. It’s called CS42: Controlling Cyberspace – and we’re sharing the syllabus online.  Anything big we’re missing?

Description:

Why does the Internet environment exist in the form it does today? What does its future, and the future of online life in general, look like? To what extent is this future malleable? Governments, corporate intermediaries, and hackers are empowered to different degrees by the space, and their interests and strengths are often in tension. This class uses academic as well as non-traditional texts to engender a broader understanding of Internet culture and technology, with an end focus on making informed choices about the future.

A Note about Reading:

The reading for this class will be anywhere between 30-100 pages per session. It will probably be helpful to read the selections in the order they appear in the syllabus, as some of the texts assume knowledge provided by the ones before them. Of course, inclusion of something in the syllabus should not be taken as an endorsement of its position or author. People are still wrong on the Internet.

Readings are subject to change. Material not available publicly online will be posted to the course iSite.

Class 1: Monday, January 30^th: The Internet’s Past

• Internet History
  • John Perry Barlow. “A Declaration of the Independence of Cyberspace.”
  • Johnny Ryan. “The Essence of the Net from A History of the Internet and the Digital Future.” Ars Technica.
• Who needs Cyberlaw?
  • Lawrence Lessig. “The Law of the Horse.” Harvard Law Review. 

Class 2: Monday, February 6th: Whatever Happened to Jurisdiction?

• Dow Jones v. Gutnick
  • Jonathan Zittrain. Jurisdiction. pages 4-9, 47-54
  • Felicity Barringer. “Internet Makes Dow Jones Open to Suit in Australia.” The New York Times.
• MegaUpload
  • MegaUpload Indictment. Pp. 1-65.
  • Nate Anderson. “Explainer: How can the US seize a ‘Hong Kong site’ like Megaupload?” Ars Technica.

Class 3: Monday, February 13th: Copyright and Free Speech

• Copyright
  • Terry Fisher. Copyright for Librarians. Module 1 + Module 7.
• Cancel-bots and Early Internet Speech
  • Skim: Wikipedia article on Scientology and the Internet.
  • Alan Prendergast. “Hunting Rabbits, Serving Spam: The Net Under Siege.”
• The Power of the Cease and Desist
  • Peruse: Chilling Effects.
  • Yochai Benkler. The Wealth of Networks. Pp. 225-233.
  • Kim Zetter. “Diebold Loses Key Copyright Case.” Wired. 
  • Cease and Desist Demand, Trevor Eckhart.
  • Andy Greenberg. Carrier IQ: A Case Study in the Streisand Effect Squared. Forbes. 

Class 4: Thursday, February 23^rd : Representing Ourselves Online

• Avatars
  • Neal Stephenson. Snow Crash, (New York: Bantam Books, 1992), 35-44.
  • Nicolas Ducheneaut, Ming-Hui “Don” Wen, Nicholas Yee, Greg Wadley. “Body and Mind: A Study of Avatar Personalization in Three Virtual Worlds.” CHI 2009. (Intro, Discussion, Conclusion)
  • “City of Copies: Marvel. Vs. NC Soft.”
  • Memorandum of Points and Authorities of Amici Curiae Legal and Cultural Studies Scholars in Support of Defendants’ Motion for Summary Judgment. Electronic Frontier Foundation. 
• Social Networks
  • Sherry Turkle. Alone Together. 181-199.
  • “What Happens When you Deactivate Your Facebook Account.” ReadWriteWeb.
  • Tim Carmody. “You Are Not Your Name and Photo: A Call to Reimagine Identity.“ Wired.

Class 5: Monday, February 27th: Defamation, Civility and Attribution

• The Wikipedia Biography Controversy and Section 230
  • Legal Guide for Bloggers- Section 230 Protections. Electronic Frontier Foundation.
  • Wikipedia Biography Controversy. Wikipedia.
  • Reliability of Wikipedia, False Biographical Information. Wikipedia. 
• Anonyminity and Pseudonymity
  • Green Blackboards (And Other Anomalies). PennyArcade. WARNING: Language NSFW.
  • Rachel Cooke and Aleks Krotoski. “Should Internet commentators use their real names?” Comment is free.
  • Kee Hinkley. “On Pseudonymity, Privacy and Responsibility on Google+.” TechnoSocial. Published July 27th, 2011. Pgs. 1-16 (No longer available online, will distribute PDF.)
• Attribution
  • Aaron E. Kornblum. “Searching for John Doe: Finding Spammers and Phishers.”
  • David D. Clark, Susan Landau. “Untangling Attribution.”

Class 6: Monday, March 5th: Generativity

• Theories of Generativity
  • Jonathan Zittrain. “Protecting the Internet Without Wrecking It.” Boston Review.
  • Read one of the responses: Bruce M. Owen, Richard Stallman, Susan Crawford, David D. Clark, Roger A. Grimes, and Hal Varian. http://bostonreview.net/BR33.2/ndf_internet.php
  • James Grimmelmann. “Applications and Applicances: A Conversation with Jonathan Zittrain.” The Laboratorium.
• What about Content?
  • Brad Stone. “Amazon Erases Orwell Books from Kindle.” The New York Times.
  • Mark Frauenfelder. “Bezos apologizes for Kindle 1984 memory hole blunder.” BoingBoing.
  • Brian X. Chen. “iPad Apps Could Put Apple in Charge of the News.” Wired. 
• Bootloaders
  • Jon Brodkin. “The Right to dual-boot: Linux groups plead case prior to Windows 8 launch.” Ars Technica.
  • Peter Bright. “Windows 8’s locked bootloaders: much ado about nothing, or the end of the world as we know it?” Ars Technica. 
  • Ed Bott. “Linux won’t be locked out of Windows 8 PCs, but FUD continues.” ZDNet.

Class 7: Monday, March 19th: DRM and Circumvention

• The Playing Field
  • Fred Von Lohmann. “Unintended Consequences: Twelve Years under the DMCA.” Electronic Frontier Foundation.
  • Mark Stefik. “Trusted Systems.” Scientific American. March, 1997.
  • Decan McCullagah. “New Copyright Bill Heading to DC.” Wired.
• A Whole New World
  • Adam Marcus. “3D Printing: The Future is Here.”  The Technology Liberation Front.
  • “It Will Be Awesome If They Don’t Screw It Up: 3D Printing, Intellectual Property, and the Fight Over the Next Great Disruptive Technology.” Public Knowledge. 
  • “Gang Used 3D Printers for ATM Skimmers.” Krebs on Security. 
  • Nick Bilton. “Disruptions: The 3-D Printing Free-for-all.” The New York Times. 

Class 8: Monday, March 26th: Crowdsourcing: Threat or Menace?

• Threat
  • Ernest Cline. Ready Player One. (New York: Crown, 2011), 1-36.
  • John C. Tang, Manuel Cebrian, Nicklaus A. Giacobe, Hyun-Woo Kim, Taemie Kim, and Douglas “Beaker” Wickert. “Reflecting on the DARPA Red Balloon Challenge,” Communications of the ACM 54 (4). (2011).
  • Jonathan Zittrain. COG. Publication forthcoming. 1-5.
• Menace
  • Brian Caulfield. “Turkish Delight.” Forbes.com.
  • “Internet Eyes, Fighting Crime from Home: Transcript.” On the Media.
  • Skim: “Ask HN: Are Freelancer sites (e.g. Odesk, Elance) useless?” Hacker News. 

Class 9: Monday, April 2nd: Gamification is…

• The Devil
  • Jesse Schell. “Design Outside the Box” DICE 2010. (Video Presentation) G4 TV.
  • “What’s the Point of Steam Achievements Anyway?”
  • Critical Distance.
• The Answer to Society’s Problems
  • The Cures of Cow Clicker: How a Cheeky Satire Became a Videogame Hit. Wired.
  • The Future is A Grind. Post-Hype.
  • Jane McGonigal. Reality is Broken. pgs. 53-79.
• Funny
  • Peter Bright. “Microsoft keeps it old-schools with a pricey text adventure game, Visual Studio 2010.” Ars Technica. 

Class 10: Monday, April 9th: Regulation, Governance and The Internet’s Future

• Short Term
  • Eliza Krigman. “Next battle over Net ramps up worldwide.” Politico.
  • Julian Sanchez. “Internet Regulation & the Economics of Piracy.” Cato@Liberty. http://www.cato-at-liberty.org/internet-regulation-the-economics-of-piracy/
  • Ian Shapira. “Obama administration joins critics of US nonprofit group that oversees Internet.” The Washington Post. 
• Medium Term
  • Charles Stross. “USENIX 2011 Keynote: Network Security in the Medium Term, 2061 – 2561 AD.” Charlie’s Diary. 

Computers Gone Wild: Impact and Implications of Developments in Artificial Intelligence on Society was an informal discussion that took place at Harvard Law School on December 8^th, 2011. Hosted by Jonathan Zittrain, Marin Soljačić and the Berkman Center for Internet & Society, we brought together eighteen mostly local guests to discuss the ways that AI is changing society. Unlike futuristic predictions involving the Singularity or the underlying technology, this workshop explored current technology. Sessions included discussions on warfare, finance, education, and labor. Below is a list of attendees and a summary of the discussion.

Attendees:

• Ryan P. Adams – Assistant Professor of Computer Science, School of Engineering of Applied Sciences, Harvard University.
• Susan Athey – Professor of Economics, Department of Economics, Harvard University.
• David Autor - Professor and Associate Department Head, Department of Economics, MIT.
• Gabriella Blum – Rita E. Hauser Professor of Human Rights and Humanitarian Law, Harvard Law School.
• Daniel Dennett – Austin B. Fletcher Professor of Philosophy, Tufts University.
• Peter Galison – Joseph Pellegrino University Professor, Department of the History of Science, Harvard University
• Andrew Lo – Harris & Harris Group Professor, Director, MIT Laboratory for Financial Engineering, MIT.
• John Markoff – Journalist, The New York Times.
• Andrew McAfee – Principal Research Scientist at Center for Digital Business, MIT Sloan School of Management.
• John Palfrey – Henry N. Ess III Professor of Law and Vice Dean, Library and Information Resources, Harvard Law School, Harvard University.
• David Parkes – Gordon McKay Professor of Computer Science, School of Engineering and Applied Sciences, Harvard University.
• Steven Pinker - Harvard College Professor and Johnstone Family Professor, Department of Psychology, Harvard University.
• Lisa Randall – Frank B. Baird, Jr., Professor of Science, Department of Physics, Harvard University.
• Stuart Shieber – James O. Welch Jr. and Virginia B. Welch Professor of Computer Science, School of Engineering and Applied Sciences, Harvard University.
• Marin Soljačić - Professor of Physics, Physics Department, MIT.
• Jeannie Suk – Professor of Law, Harvard Law School, Harvard University.
• Jonathan Zittrain - Professor of Law, Harvard Law School/ Harvard Kennedy School of Government, Professor of Computer Science, Harvard School of Engineering and Applied Sciences, Harvard University.

Military:

We discussed the modern military use of drones and other semi-autonomous, non-human forms of warfare. There are some ways that robot technology represents merely a new technology in war, like crossbows or gun powder. However, as more and more decisions are aided by machines, there is some evidence that reliance on robots makes humans less likely to overrule in favor of their own judgment in circumstances not anticipated by the AIs. For example, the crash of Air France 447 was traced to the pilots not being trained to handle a situation when the autopilot was not functioning, and not trusting the non-autopilot instruments.

Internationally, forty-five states currently have drone technology, and it is becoming increasingly accessible to non-states. The uses of drones or very small surveillance robots for criminal purposes may become normal, and access to these technologies could increase the power of non-state actors. The combination of WMDs and drone technology could mean that a terrorist organization could deploy a weapon without putting a person on the ground.

Additionally, the lower prices of small surveillance robots and memory storage, and the rise of machine learning may mean that it could become commonplace to monitor activities of civilians at all times to determine appropriate targets. Imagine a microphone near every kitchen table in a small village in Afghanistan, listening for “insurgent” activity.  Law governing surveillance of activities in plain view currently typically relies on the fact that the cost and effort of monitoring and processing information is high enough that mass data collection is not effective.  What should happen when those assumptions no longer hold?

Another question raised by the military use of AI is how to evaluate decisions made by non-human actors. Would countries be responsible for explaining the variables they used and the algorithms that calculate drone decisions? In the past, increases in technological progress have decreased war casualties and, in the case of nuclear weapons, deterred countries from going to war. Wars may become more common as the potential for both collateral and symmetric loss of human life decreases. See, for example, Congress’s debate about drones in Libya – where the lack of human involvement was a reason why some politicians were wiling to get involved. How will norms related to killing change if there is no potential for a human to be harmed on the side of the attacker?

Finance

Recent flash crashes have shown the role of algorithms and high frequency trading in the New York Stock Exchange, and the potential for disaster. For example, in August 2007, a fifteen-minute glitch caused by programmers using a placeholder value of a penny in an algorithm triggered thousands of sell orders. Stock prices of some companies dropped from forty dollars to less than a dollar in minutes, and the NYSE rolled back a certain amount of trades.

High frequency trading is a form of algorithmic trading that is dependent on the ability to trade small amounts of stock quickly in order to make small amounts of money. Trades can be made in under a millisecond, and firms are now competing for servers as physically close to the stock exchange as possible in order to complete trades faster. It’s dubious that high frequency trading is adding significant value or benefit to the market (besides making a small number of people very rich). There is a definite wealth transfer between those with the technology and those without, and increased volatility – perhaps counterintuitive to the notion that quicker trades make for better liquidity and stability.

Inequality between firms with algorithmic potential and those without it is a significant concern. The algorithms are not patentable so firms keep them as trade secrets, and there is a definite gap between firms that can afford to develop algorithms and firms that can’t. Firms with technology will continue to make more money than those without, polarizing the market even further.

Given that flash crashes have already happened, a large portion of this session was devoted to discussing potential methods of regulation, including a tax on trades (“Tobin tax”) or a requirement that trades be posted for a certain amount of time. The Tobin tax has serious downsides, as there are reasons other than algorithmic or high frequency trading for a firm to make many trades quickly; for example, pension funds often need to liquidate lots of stock over a brief time frame. Posting trades for a small amount of time (say one second) has less obvious downsides, and could prevent crashes of the type that happened in 2007.

Because of the secrecy surrounding the algorithms, it has not been possible to measure the systemic risk posed by many automated traders acting at the same time. However, it seems that if limitations on trading are not imposed, regulators should attempt to determine the total risk and whether the rewards are worth it.

Labor:

The type of jobs that computers are able to do has changed significantly over the past couple of years. For example, law firms used to hire associates to do document review for discovery, but now can use computer programs. White collar jobs are becoming increasingly susceptible to automation.

During past revolutions in labor, technologies that improve productivity have not destroyed jobs entirely; they merely move them to different sectors. However, most of the jobs that were replaced were not knowledge workers, but blue collar or manual labor. Because of this difference in the type of jobs replaced, it’s possible that this labor shift won’t result in the same sorts of job movements as past shifts in labor. There is a fundamental disagreement about whether that trend will hold up in the future – whether robots and AI will destroy jobs or whether the jobs will just move to other areas. Some argued that the new jobs created may be “below human dignity,” underpaid or not ideal, but will exist – others saw more of a move towards robots in general, with humans not finding new areas of work.

Another key theme was the question of the appropriateness of computers or robots for jobs that require binding decision-making. So far, most of the advances in labor markets related to computers have been improving productivity, with humans still in control. However, as machines become more sophisticated, it’s possible that they will make fewer errors than similarly situated humans. For example, a parole board in Israel was found to parole 65 percent of prisoners seen at the beginning of the day but the number dropped to near zero by the end of the session when the judges were about to break for lunch. Robots, the claim goes, may be in a better position to make those kinds of decisions– they wouldn’t be swayed by emotional appeals, biases or time of day, and could evaluate based on a specific set of variables. If robots can do better than the equivalent human, should we be prepared to replace parole board members? How do we handle accountability for robot justices?  There was a spirited split within the group on this issue.

There’s a certain discomforting factor about life and death decisions being made by algorithmic processes, even given the foibles of human decision-making. Are there cases where we would want humans to make a decision, even if they are worse at it than an algorithm might be?

Education:

The workshop then had a mini-session about education and the role of the university professor as technology progresses. Examples of teaching affected by technology included Stanford’s AI class (with 54,000 people taking the class via the Internet) and the development of computer simulations of experiments. Using computers to speed up and aid research in some fields is easy; however, technological progress becomes more complicated when it is hard to scale the student-teacher experience.

Technology could help democratize the educational experience; however, some of the spontaneity and personal connections between professors and students might be lost. Allowing for universal access to educational materials may be beneficial, but how do you ensure quality control and preserve the ability of students to interact personally?

The Future:

It’s easy to make doomsday predictions without understanding the science, or to suggest regulation as a kneejerk response, but it’s important to realize that it’s hard to intervene without data.

For the Finance case, an intervention seemed most helpful because there were a clearly defined set of problems and actors. In military and labor, fundamental uncertainty about the next steps along the AI path meant that regulation (or even predictions) seemed unwise. However, all three cases made the participants wish for a better understanding of the systematic risks involved with changes that have already gone on, in order to better prepare for the future.

Summary by Kendra Albert.

Here at Future of the Internet, we’ve already talked a little bit about Apple’s content requirements for both the iOS and Mac App Stores in JZ’s The PC is Dead post. As JZ said,

“Pulitzer Prize-winning editorial cartoonist Mark Fiore found his iPhone app rejected because it contained “content that ridicules public figures.” Fiore was well-known enough that the rejection raised eyebrows, and Apple later reversed its decision. But the fact that apps must routinely face approval masks how extraordinary the situation is: tech companies are in the business of approving, one by one, the text, images, and sounds that we are permitted to find and experience on our most common portals to the networked world. Why would we possibly want this to be how the world of ideas works, and why would we think that merely having competing tech companies—each of which is empowered to censor—solves the problem?”

Apple’s approach is an example of a larger phenomenon. Microsoft recently released its guidelines for its new Windows 8 app store, and it looks like the company wants the same level of editorial control. From the Windows 8 Store Terms of Use:

You [the developer] may not include or submit any content that is untrue, misleading, defamatory, infringing, or harassing, that constitutes hate speech, that is or includes sexual content, that insinuates profanity, or that is otherwise objectionable.

This places Microsoft’s review team in the position of deciding what is untrue, infringing or otherwise objectionable (not to mention what “insinuates profanity” — a rather odd turn of phrase, perhaps meant to cover sanitized profanity like “sh*t”?).

There is at least one major difference between the current Microsoft store and the Apple store: Microsoft has phrased its licensing in such a way to allow free and open source software to be distributed through the store without breaking either license. In the license to customer section of its app developer agreement, Microsoft says:

Your license terms must also not conflict with the Standard Application License Terms, in any way, except if you include FOSS, your license terms may conflict with the limitations set forth in Section 3 of those Terms, but only to the extent required by the FOSS that you use. “FOSS” means any software licensed under an Open Source Initiative Approved License.

It appears that the Open Source Initiative’s licenses (which include the GPL) will then be commensurable with Microsoft’s, meaning that software developers don’t have to give up their free and open source rights (and requirements) to use the Windows 8 Store.

Like Apple’s OS X App Store, the Windows 8 store is only one method of installing applications on machines running Windows 8. Users can still download applications from the Internet or buy software elsewhere to install on their machines: “sideloading.”

Microsoft reserves the right to retroactively delete from users’ machines any apps that it no longer wants to distribute through the store. It brings to mind the 1984 Kindle incident,where Amazon removed versions of Orwell’s 1984 from purchasers’ Kindles.  ”The death of the PC” is not a claim about the withering away of the PC form-factor — but rather, the end of commonly installing software without requiring the assent of an intermediary.  Apple’s products have provided a model for this end, one now replicated in part by Microsoft.

From Technology Review:

The Personal Computer Is Dead

Power is fast shifting from end users and software developers to operating system vendors.

By Jonathan Zittrain

Update on 2/23/2011: Apple has pushed back its deadline for OSX sandboxing to June 1st, 2012. The deadline was originally November, 2011, but was pushed to March 1st  in early November. Although Apple claimed that this change was to give developers time to integrate new permissions from an update,  it does follow the announcement of Gatekeeper, which might be a partial substitute for sandboxing.

During the 1990s, PCs ran whatever software was installed on them. Users bought software (not yet called apps) from physical stores or got a copy from their friends. They stuck the CD in the drive, and went through the installation process, or dragged the application to their application folder.  The code was “signed” by the developer (by being from a box), or not at all.  The operating system didn’t stop and ask “are you sure,” no one typed in a root password, and the applications were limited only by what their programmers had decided when coding. Those were the days of the playground, not the sandbox.

The playground had problems. Software with malicious code or bugs could hijack your PC. Other users could tamper with your files. So sandboxing (although not known yet by that name) was born. Starting as far back as creating separate user directories, to as recent a development as cloud data storage, steps have been taken to make the user’s personal data and computing cycles  safely under lock and key – sometimes with the key held by the user, and other times by the operating system maker. When Windows Vista was released in 2009, it tried to solve this problem with “allow” dialogs, which would pop up when an application tried to take actions without the user’s permission. This solution was mostly considered an annoyance, not a feature, as the parameters that caused a dialog were broad and users just clicked through to allow instinctually. Mac OSX’s requests for passwords before installing applications are just another step down this road – meant to put users in control of their own computing destinies, and less dependent on the good will of developers.

On smartphones, things have gone a different way.  We take for granted that applications haven’t been able to access all the features of the phone. Android users view a permissions screen that tells them exactly what to expect from an application – whether it can take photos with the camera, keep the phone from sleeping or access GPS. Apple screens its applications on the way into the iOS App Store, making clear what parts of the phone they can get to, and limits users to apps from the store. All of the app’s files are required to stay in their own little corner of the file system. This process of requiring developers to encapsulate their applications within one folder – and to clear with someone for access to things outside – restricts the potential for harm. This is one version of the phenomenon known as sandboxing. Sandboxing, on the most basic level, is a security measure used to run code that’s not trusted. Rather than allowing software or applications to play freely across the machine, sandboxes restrict them to very specific resources, mitigating the damage they can do.

For years, it’s been a standard on mobile platforms and web applets. For example, the Bejeweled game that you’re playing in one of your Chrome tabs doesn’t have any way of accessing the Word document you are supposed to be working on. In fact, your Chrome tabs can’t even access each other, preventing badly coded webpages from crashing your entire browser. These apps can play in their own sandboxes – but not all over your phone or PC. The same wouldn’t be true for Bejeweled if it were a regular PC app – it’d be free to rummage through everything on the hard drive, surveilling, modifying or deleting at will.  Knowing that, it’s a marvel that serious viruses didn’t appear sooner and more often.

For phones and web applets, sandboxing is ideal. After all, even you, the user, don’t interact with the underlying file structure of your iPhone or Android device (without jailbreaking), and you certainly wouldn’t want to open an online game and have it be able to make changes to your Word document. Sandboxing has serious security advantages – if programs have to lay out in advance what kind of access they get to a device, and are limited to only specific actions, an app that “goes rogue” or is compromised by malware can no longer cause the same harm to the rest of the user’s data. Similarly, a piece of compromised software can’t run processes in the background that it wouldn’t be able to run anyway – limiting potential damage.

Sandboxing usually also includes signed code, or code that is cryptographically linked to a specific developer license.  Not only do Apple and Google know exactly what parts of your phone the code can access, they also know which developer produced the code. Apple’s signature program is run through its developer program, which is tied to a yearly fee, where as Google’s requires some information from the developer, but not a specific license as such.

Enter the Mac App Store

So as things stood previously, most PCs had some steps towards protection against rogue software, but hadn’t taken the sandboxing route. Browsers and smartphones sandbox processes, but different smartphone platforms have different ways of involving the user. Apple’s iOS doesn’t involve the user, and all permissions are handled at the App Store level. Android apps can either be downloaded from the Android Market or from third parties directly. In both cases, a list of permissions are visible to the user and the applicatiosn are sandobxed where they are still sandboxed.

In early November, Apple announced to developers that it was pushing back its deadline for Mac Store Apps to implement sandboxing to March 1st, 2012. This makes the Mac App Store platform much like the Android Market – users have the option to install applications from elsewhere, but all applications that go through the market must be sandboxed. Although the requirements were supposed to take effect earlier this week, the pushed-back date reflects some of the serious problems MacOS developers were running into in making their applications compatible with Apple’s new rules.

Apple’s change marks a huge departure from the way development and code has operated in the past. As discussed above, software on the PC of the past had access to a playground, controlled only by the user’s discretion. Although Apple’s new OSX Lion has supported sandboxing since its release, Apple is using its distribution platform to require that apps follow their new security rules. The App Store push will certainly increase the amount of applications secured.

Although there may be net gains for users who are concerned about the security of applications downloaded from the Internet, there is the potential to limit the types of applications that companies will bother trying to distribute. Programs as widely used as antivirus software and backup utilities are not distributable through the App Store platform, due to the rule against root access, and in the larger scheme of things, Apple’s concerns about security. This means no Norton Anti-Virus, no Dropbox and no MacZip. At this time, distribution outside the store is not problematic; in fact, most large-scale developers seem to not be early adopters. However, as customers become more comfortable with the App Store process, that might change.

OSX Sandboxing in Depth

The sandboxing requirements make applications downloaded through the Mac App Store limited to a very specific set of actions. Ars Technica offered an in-depth discussion of OSX Lion’s sandboxing procedures in its review of the platform, and here are the basics

To play outside the sandbox, applications need “entitlements” that represent permission to access outside tools. Entitlements can include taking photos with the camera, responding to mouse gestures or creating network connections. Lion has about thirty built-in entitlements for applications to request, created and managed by Apple.

When submitting their software to the app store, developers lay out each process that an application might run, and then explain the privileges each one might have. For example, an application like QuickTime decodes video, runs audio, and accesses closed captions from a folder at the same time. Each of those different tasks is split into a sub-process with a different set of permissions – laid out by the developer before submission to the App Store. So the video decoding sub-process of Quicktime has access to the user’s screen and graphics card, but not audio settings. The audio sub-process doesn’t have access to the video card. These divisions keep the software from using system resources without permission. Unlike platforms like Android, users won’t see these processes or entitlement – they’re just for the purpose of Apple approving the software.

Users do have special powers for sandboxed applications – any action that is specifically initiated by a user doesn’t need to be okayed by Apple in advance. They can initiate special, non-entitled actions, like opening a list of recent files or saving documents elsewhere in the file system. Apps can build in these requests without asking for entitlements for them, knowing that the user is going to activate and oversee the process.

Concerns from Developers

Pushback from the development community has come from many fronts. Some developers, such as Recent Redux creator Tim Schroeder, feel that the entitlements that the apps can have do not represent the full spectrum of developers’ needs. There are two very specific use cases that are no longer possible for App Store apps – using AppleScript and file system management

Most users probably don’t interact with AppleScript on a regular basis, but it allows for many of the processes that make software work. Notification systems like Growl, which standardizes user notifications across multiple applications use AppleScript, as do hundreds of apps that allow for better music management in iTunes. On its most basic level, AppleScript is a tool developers use to exchange information between applications, and can produce Apple Events, which make programs take actions. It can run repetitive tasks, print from one application to another, or open applications. However, it also can automate file transfers or photo editing – and will no longer be usable by sandboxed applications. Instead, AppleScript will theoretically be replaced by APIs (application programming interfaces) that allow developers (and Apple) to have better control over application interaction.

Matthias Gansrigler, a developer responsible for applications including ScreenFloat and Yoink, explains how sandboxing could, without a new API, destroy one of his existing pieces of software. GimmeSomeTune (GST) downloads lyrics and album covers, as well as displaying iTunes info in a customizable window. To do those things, GST depends on a connection to iTunes via AppleScript – it extracts information about songs that are playing and uses it to download lyrics and cover art back to iTunes. Without an API, and with Apple eventually sandboxing iTunes, GimmeSomeTune will no longer be able to access the song titles it needs. Gansrigler notes that Apple has not sandboxed iTunes yet, but with it on the horizon, he’s stopping development on the software now.

File system management is the second big field that developers are concerned about. There are many existing systems used by corporations or developers that support version control or sorting of code – and they work in the background in the file system without the user’s consent. Similarly, SSH clients or FTP apps can no longer show real time file trees – which means that all moving of files or downloads must go through the open dialog. To paraphrase an Apple ad, there’s no entitlement for that.

And to make things worse, some of the entitlements are currently temporary, leading developers to be concerned that their software might break after said entitlements are revoked. Apple has promised to provide APIs to replace the temporary entitlements, but it’s not clear when those will be available. In the mean time, developers are in a terrible state of flux – trying to decide whether to continue to put time into applications that may not be able to exist by March.

Independent of the concerns about APIs and entitlements is the uncomfortable truth of the App Store as a completely new way for developers to interact with customers. Although in previous years an OS upgrade or an update could break software, those were rare to the extreme. If an application worked on a computer, it would continue to work. Now, Apple’s role in the development process means that developers have to actively coordinate with Apple to keep their software from breaking – and a slight change in the entitlement system could destroy all of the tethered software. Apple had made itself a controlling party in the application wars – and the consequences of that are still unknown. This sandbox is under the baleful eye of Apple as playground monitor. Given Apple’s willingness to ban security researchers like Charlie Miller from developer programs for publishing security holes, this increased control might not be the security boon anyone hoped for. As if that wasn’t enough, flaws in the sandboxing system have already been reported.

Trading Generativity for Security

Sandboxing represents a true security/generativity trade off. As Jonathan Zittrain said in Chapter 7 of The Future of the Internet and How to Stop It, “Most fundamentally, many of the benefits of generativity come precisely thanks to an absence of walls. We want our e-mail programs to have access to any document on our hard drive, so that we can attach it to an e-mail and send it to a friend.” Applications downloaded from the Mac App Store, in contrast to ones from the generative Internet, may not have these capabilities.

Sandboxing can prevent some damage from an app bound and determined to wreak havoc, but sandboxing is a phenomenon independent of the App Store: Mac OS could implement it with or without Apple screening the software up front. Certainly, not all software that runs on Mac OSX is downloaded through the app store. But as The Unofficial Apple Weblog (TUAW) put it, “There’s also the fact that any discussion that begins with ‘The Mac App Store isn’t the only way to get apps on a Mac’ inevitably ends with the ominous pronouncement ‘yet.’”

Furthermore, there are issues on the development side of generativity. It seems unlikely that developers who are concerned about the market share will develop two versions of the app – (one that’s sandbox safe for the App Store and one that includes extra features that won’t work in a sandbox). As a result, sandboxing might dumb down the feature set available to even those who choose to grab their applications from the Internet. Once programmers are playing in the sandbox, there’s little reason to develop for playground-level access again. Reactions have been slow but scared – app-makers are uncertain as to what a sandboxed future means for their applications and for their distribution.

Even in the short term, where both the App Store and regular distribution methods co-exist, Apple’s sandboxing requirements are a big deal. The uncertainty about the existence of longstanding Mac programming methods like AppleScript and Apple Events, combined with the fact that no one is sure exactly how much business the App Store will do means that the impact is totally unknown. It seems unlikely that Windows 8 or other OSs will follow suit, given that Microsoft hasn’t been pursuing the same sort of distributional model, but the new tethered nature of these applications is a significant change from the way PCs have previously operated. The only thing that’s sure is that Apple would like the future of the Mac platform to be a sandboxed one, and consumers and developers will have to adapt.

12/7/11: Edited to incorporate corrections from the comments re: Android code signing and permissions.

John Battelle asked me a few Qs about my thinking on the themes in The Future of the Internet in the three years since the book came out (four since it was drafted!).  John’s review is available on his blog, and I’ve reproduce the core of it here:

JBAT:

- You wrote the Future of the Internet three years ago. It warned of a lack of awareness with regard to what we’re building, and the consequences of that lack of attention. it also warned of data silos and early lockdown. Three years later, how are we doing? Are things better, worse, the same?

And a follow up. On a scale of one to ten, where one is “actively helping” and ten is “pretty much evil,” how do the following companies rate in terms of the debate you frame in the book?

- Google (you can break this down into Android, Search, Apps, etc)

- Facebook (which was really not at full scale when you published)

- Apple

- Twitter

- Microsoft (again break it down if you wish)

Thanks!

JONATHAN ZITTRAIN:

Sorry this took me so long! I got a little carried away in answering –

- You wrote the Future of the Internet three years ago. It warned of a lack of awareness with regard to what we’re building, and the consequences of that lack of attention. it also warned of data silos and early lockdown. Three years later, how are we doing? Are things better, worse, the same?

It’s the best of times and the worst of times: the digital world offers us more every day, while we continue to set ourselves up for levels of surveillance and control that will be hard to escape as they gel.

That’s because the plus is also the minus: more and more of our activities are mediated by gatekeepers who make life easier, but who also can watch what we do and set boundaries on it — either for their own purposes, or under pressure from government authorities.

On the book’s specific predictions, Apple’s ethos remains a terrific bellwether. The iPhone — released in ’07 — has proved not only a runaway success, but the principles of its iOS have infused themselves across the spectrum. There’s less reason than ever to need a traditional PC, and by that I mean one that lets you run whatever code you want. OS X Lion points the way to a much more controlled PC zone, anyway, as it more and more funnels its software through a single company’s app store rather than from anywhere. I’d be surprised if Microsoft weren’t thinking along similar lines for Windows.

Google has offered a counterpoint, since the Android platform, while including an app store, allows outside code to be run. In part that’s because Google’s play is through the cloud. Google seeks to make our key apps based somewhere within the google.com archipelago, and to offer infrastructure that outside apps can’t resist, such a easy APIs to geographic mapping or user location. It’s important to realize that a cloud-based setup like Google Docs or APIs, or Facebook’s platform offer control similar to that of a managed device like an iPhone or a Kindle. All represent the movement of technology from product to service. Providers of a product have little to say about it after it changes hands. Providers of services are different: they don’t go away, and a choice of one over another can have lingering implications for months and even years.

At the time of the book’s drafting, the alternatives seemed stark: the “sterile” iPhone that ran only Apple’s software on the one hand, and the chaotic PC that ran anything ending in .exe on the other. The iPhone’s openness to outside code beginning in ’08 changed all that. It became what I call “contingently generative” — it runs outside code after approval (and then until it doesn’t). The upside is that the vast creativity of outside coders has led to a software renaissance on mobile devices, including iPhones, from the sublime to the ridiculous. And Apple’s gatekeeping has seemed to be with a light touch; apps not allowed in the store pale in comparison to the torrents of stuff let through. But that masks entire categories of applications that aren’t allowed — namely anything disruptive to Apple’s business model or that of its partners or regulators. No p2p, no alternate email clients, browsers with limited functionality.

More important, the ability to limit code is what makes for the ability to control content. More and more we see content, whether a book, or a magazine subscription, represented in and through an app. It’s sheer genius for a platform maker to demand a cut of in-app purchases. Can you imagine if, back in the day, the only browser allowed on Windows was IE, and further, all commerce conducted through that browser — say, buying a book through Amazon — constituted an “in-app purchase” for which Microsoft was due 30%?

A natural question is why competition isn’t the answer here — or at least reason to not worry about the question. If people thought the iPhone made for a bad deal, why would they want one? The reason they want one is the same thing that made the Mac so appealing when it first came on the scene: it was elegant and intuitive and it just worked. No blue screen of death. Consistency across apps. And, as viruses and worms naturally were designed for the most common platform, Windows, those 5% with Macs weren’t worth the trouble of corrupting.

We’ve seen a new generation of Mac malware as its numbers grow, and in the meantime a first defense is that of curation: the app store provides a rough filter for bad code, and accountability against its makers if something goes wrong even after it’s been approved. So that’s why the market likes these architectures. I’ll bet few Android users actually go “off-roading” with apps not obtained through the official Android app channels. But the fact that they can provides a key safety valve: if Google were to try the same deal as Apple with content providers for in-app content, the content providers could always offer their wares directly to Android users. I’m worried that a piece of malware could emerge on Android that would cause the safety valve of outside code to be changed, either formally by Google, or in practice as people become unwilling to drive outside the lanes.

So how about competition between platforms? Doesn’t that keep each competitor honest, even if all the platforms are curated? I suppose: the way that Prodigy and CompuServe and AOL competed with one another to offer different services as each chased subscribers. (Remember the day when AOL members couldn’t email CompuServe users and vice versa?) That was competition of a sort, but the Internet and the Web put them all to shame — even as the Internet arose from no business plan at all.

Here’s another way to think about it. Suppose you were going buy a new house. There are lots of choices. It’s just that each house is “curated” by its seller. Once you move in, that seller will get to say what furnishings can go in, and collects 30% of the purchase price of whatever you buy for the house. That seller has every reason to want to have a reputation for being generous about what goes in — but it still doesn’t feel very free when, two years after you’re living in the house, a particular coffee table or paint color is denied. There is competition in this situation — just not the full freedom that we rightly associate with inhabiting our dwellings. A small percentage of people might elect to join gated communities with strict rules about what can go inside and outside each house — but most people don’t want to have to consult their condo association by-laws before making choices that affect only themselves.