Measurement is central to online advertising: it facilitates billing, performance measurement, targeting decisions, spending allocation, and more. In a pair of earlier posts we explained how advertisement frequency capping and behavioral targeting are achievable without compiling a user’s browsing history. This post similarly proposes practical, privacy-improved approaches to advertising measurement.

There are, broadly, three advertising events that might require measurement.

1. Impression. An advertisement is displayed to a user. Measured details might include the webpage the ad was served on, the time the ad was served, the user’s location, and the user’s browser.¹
2. Click. The user clicks the ad.
3. Action. After viewing the ad, the user later does something on a different webpage. For example, the user might buy the product or service that was advertised. An action may occur days or weeks after an impression.

Sometimes advertisers pay per impression (“CPM” billing), sometimes per click (“CPC”), sometimes per action (“CPA”), and sometimes for a combination of these events (“hybrid”).

The following sections explain how to conduct a privacy-improved measurement of each advertising event.

Impression

When a user’s browser loads an advertisement, the company serving the ad ordinarily learns the URL of the current webpage,² as well as the user’s IP address and User-Agent string.³ Practical, privacy-improved impression measurement is a granularity problem: How can a website generalize the impression data it collects without substantially compromising the utility of that data?

Requirements will undoubtedly vary by service. We present here a rough design spectrum of the information that a third-party website might retain.⁴

Click

Click measurement is exactly the same as impression measurement, with just one extra piece of information: whether the user clicked the ad.⁵

Action

Action measurement is a more difficult engineering problem. An ad impression is, for measurement purposes, a one-shot event: it occurs within the context of a single webpage. Measuring an action, on the other hand, requires linking an ad impression on one webpage with a subsequent action on another webpage.⁶

A pairing of client-side storage and selective information disclosure can enable privacy-improved action measurement, much like our previous approaches to frequency capping and behavioral targeting. When an ad is displayed, information about the impression can be stored in the browser. If the user later completes an action, the ad company can query the browser for relevant impression information.

Implementing a prototype of our action measurement algorithm was straightforward using HTML 5 local storage. Source is available on GitHub. Performance is a non-issue, as with our prototypes for frequency capping and behavioral targeting.

1. Many advertising companies also record whether this was a first-time (“unique”) impression. Our algorithm for frequency capping can be trivially modified to provide this functionality.  ↩

2. Third-party websites usually learn the first-party webpage URL from a Referer header or explicit Request-URI parameter. There are some methods for a first-party webpage to hide its URL from third-party content, including iframe sandboxing and the HTML 5 noreferrer link annotation. For the moment, these techniques are not sufficiently simple, comprehensive, or supported to anticipate widespread use.  ↩

3. A semi-trusted intermediary or anonymizing network could conceal or generalize a user’s IP address, User-Agent string, and other information. See Adnostic and Privad for examples. These approaches are, at present, not practical for broad deployment.  ↩

4. Present Do Not Track proposals diverge on impression measurement; some would allow current approaches to continue, while others would require quickly generalizing impression data.  ↩

5. Do Not Track would allow the current approach to click measurement since the user has (somewhat constructively) interacted with content from the advertising company.  ↩

6. Because of this property, some Do Not Track proposals would require a privacy-improved approach to action measurement.  ↩

Do Not Track accomodates these divergent preferences in two ways. First, browsers and other user agents include an option for universally signaling a preference against tracking (“DNT: 1”). Firefox, Internet Explorer, and Safari have all integrated this feature, and Chrome will support it by the end of the year. Second, a user can configure exceptions to the universal signal. Some websites may choose to build a proprietary “out-of-band” exception mechanism, using ordinary web technologies, that trumps the “DNT: 1” signal. The Do Not Track Cookbook includes an example of how a Facebook out-of-band exception mechanism might appear.

The W3C Do Not Track standard will provide another option: a simple JavaScript interface that allows a website to request an exception, paired with a signal that some tracking is allowed (“DNT: 0”).

There are many benefits to managing Do Not Track exceptions through the browser.

• Avoids Duplication of Effort. Browser vendors implement an exception mechanism once. Websites can then take advantage of the mechanism with just a few lines of code.
• Persistence. A user is unlikely to accidentally clear browser-based exceptions, in contrast to cookies and other out-of-band exception storage mechanisms.²
• Centralized Management. Users can adjust all their Do Not Track preferences in one place. Out-of-band exceptions might be scattered across the web.³
• Consistent User Interface. The Do Not Track exception user interface will be the same across sites and integrated into the browser’s privacy controls.
• Design Quality. Browser vendors employ some of the brightest minds in user interface design.
• Usability Incentives. Web browsers compete on usability and frequently iterate with user interface improvements. Browser development teams are, for the most part, incentivized to provide users with adequate information about and control over a Do Not Track exception. A website that seeks a Do Not Track exception, on the other hand, is incentivized to push the boundaries of acceptable notice and choice to get that exception.⁴

In order to better understand the technical challenges associated with browser-based Do Not Track exceptions, I implemented a prototype as a Firefox add-on.

The source is available on GitHub. I want to emphasize that this is a prototype: it does not conform to the current W3C specification draft and it is insecure, buggy, and slow.

I learned several lessons from the project.

• Browser-based exceptions are not very difficult to implement. The Firefox prototype required only a couple days of straightforward development.
• Browser-based exceptions are markedly more difficult to implement than the “DNT: 1” header. As a very rough comparison, my reference “DNT: 1” Chrome extension is 9 lines of code, while my prototype Firefox exceptions extension is already 521 lines. Furthermore, implementing “DNT: 1” is largely a systems engineering project, while an exception mechanism necessitates both systems and user interface effort.
• The Do Not Track exception user interface is hard to get right. After several iterations, the user interface in my prototype still leaves much room for improvement. I expect Do Not Track will, like other browser features, benefit from long-term user interface evolution.

The prototype add-on validates what Do Not Track proponents have long recognized: Do Not Track is not a crude on/off switch. Rather, it begins a conversation between websites and users about privacy preferences. Browsers will play a central role in facilitating that conversation.

Thanks to Arvind Narayanan for comments on a draft.

1. A number of surveys have reflected mixed consumer preferences on web tracking. See, e.g., Pew Internet 2012, TRUSTe/Harris Interactive 2011, USA Today/Gallup 2010, McDonald and Cranor 2010, and Turow et al. 2009.

2. If a website maintains user accounts, associating an out-of-band exception with an account can greatly reduce the risk of accidental clearing.

3. This problem could be somewhat mitigated with a browser scripting interface for registering an out-of-band exception mechanism. I proposed the approach last year, and there was renewed interest at a recent W3C working group meeting.

4. See Leon et al. 2011 for a usability analysis of current online advertising user control mechanisms.

Co-authored by Arvind Narayanan and Subodh Iyengar.

In the first installment of the Tracking Not Required series, we discussed a relatively straightforward case: frequency capping. Now let’s get to the 800-pound gorilla, behaviorally targeted advertising, putatively the main driver of online tracking. We will show how to swap a little functionality for a lot of privacy.

Admittedly, implementing behavioral targeting on the client is hard and will require some technical wizardry. It doesn’t come for “free” in that it requires a trade-off in terms of various privacy and deployability desiderata. Fortunately, this has been a fertile topic of research over the past several years, and there are papers describing solutions at a variety of points on the privacy-deployability spectrum. This post will survey these papers, and propose a simplification of the Adnostic approach — along with prototype code — that offers significant privacy and is straightforward to implement.

Goals. Carrying out behavioral advertising without tracking requires several things. First, the user needs to be profiled and categorized based on their browsing history. In nearly all proposed solutions, this happens in the user’s browser. Second, we need an algorithm for selecting targeted ads to display each time the user visits a page. If the profile is stored locally and not shared with the advertising company, this is quite nontrivial. The final component is for reporting of ad impressions and clicks. This component must also deal with click fraud, impression fraud and other threats.

Existing approaches

The chart presents an overview of existing and proposed architectures.

“Cookies” refers to the status quo of server-side tracking; all other architectures are presented in research papers summarized in the Do Not Track bibliography page. CoP stands for “Client-only Profiles,” the architecture proposed by Bilenko and Richardson.

Several points of note. First, everything except PrivAd — which uses an anonymizing proxy — reveals the IP address, and typically the User Agent and Referer to the ad company as part of normal HTTP requests. Second, everything except CoP (and the status quo of tracking cookies) requires software installation. Opinions vary on just how much of a barrier this is. Third, we don’t take a stance on whether PrivAd is more deployable than ObliviAd or vice-versa; they both face significant hurdles. Finally, Adnostic can be used in one of two modes, hence it is listed twice.

There is an interesting technological approach, not listed above, that works by exposing more limited referer information. Without the referer header (or an equivalent), the ad server may identify the user but will not learn the first-party URL, and thus will not be able to track. This will be explored in more depth in a future article.

New approach. In the solution we propose here, the server is recruited for profiling, but doesn’t store the profile. This avoids the need for software installation and allows easy deployability. In addition, non-tracking is externally verifiable, to the extent that IP address + User-Agent is not nearly as effective for tracking as cookie-based unique identifiers.[1] Like CoP, and unlike Adnostic, each ad company can only profile users during visits to pages that it has a third-party presence on, rather than all pages.

Profiling algorithm.

1. The user visits a page that has embedded content from the ad company.

2. JavaScript in the ad company’s content sends the top-level URL to a special classifier service run by the ad company. (The classifier is run on a separate domain. It does not have any cookies or other information specific to the user.)

3. The classifier returns a topic classification of the page.

4. The ad company’s JavaScript receives the page classification and uses it to update the user’s behavioral profile in HTML5 storage. The JavaScript may also consider other factors, such as how long the user stayed on the page.

There is a fair degree of flexibility in steps 3 and 4 — essentially any profiling algorithm can be implemented by appropriately splitting it into a server-side component that classifies individual web pages and a client-side component that analyzes the user’s interaction with these pages.

Ad serving and accounting.

The ad serving process in our proposal is the same as in Adnostic — the server sends a list of ads along with metadata describing each ad, and the client-side component picks the ad that best matches the locally stored profile. To avoid revealing which ad was displayed, the client can either download all (say, 10) ads in the list while displaying only one, or the client downloads only one ad, but ads are served from a different domain which does not share cookies with the tracking domain. Note the similarity to our frequency capping approach, both in terms of the algorithm and its privacy properties.

Accounting, i.e., billing the right advertiser is also identical to Adnostic for the cost-per-click and cost-per-impression models; we refer the reader there. Discussing the cost-per-action model is deferred to a future post.

Implementation. We implemented our behavioral targeting algorithm using HTML 5 local storage. As with our frequency capping implementation, we found performance was exceptionally fast in modern desktop and mobile browsers. For simplicity, our implementation uses a static local database mapping websites to interest segments and a binary threshold for determining interests. In practice, we expect implementers would maintain the mapping server-side and apply more sophisticated logic client-side.

We also present a different work-in-progress implementation that’s broader in scope, encompassing retargeting, behavioral targeting and frequency capping.

Conclusion. Certainly there are costs to our approach — a “thick-client” model will always be slightly more inconvenient to deploy and maintain than a server-based model, and will probably have a lower targeting accuracy. However, we view these costs as minimal compared to the benefits. Some compromise is necessary to get past the current stalemate in web tracking.

Technological feasibility is necessary, but not sufficient, to change the status quo in online tracking. The other key component is incentives. That is why Do Not Track, standards and advocacy are crucial to the online privacy equation.

[1] The engineering and business reasons for this difference in effectiveness will be discussed in a future post.

Debates over web tracking and Do Not Track tend to be framed as a clash between consumer privacy and business need. That’s not quite right. There is, in fact, a spectrum of possible tradeoffs between business interests and consumer privacy.

Our aim with the Tracking Not Required series is to show how those tradeoffs are not at all linear; it is possible to swap a little functionality for a lot of privacy. We only use technologies that are already deployed in browsers, and the solutions we propose are externally verifiable.¹

We focus on issues at the center of Do Not Track negotiations in the World Wide Web Consortium. Advertising companies have pledged to stop forms of ad targeting once a user enables Do Not Track, but many maintain that tracking is essential for a litany of “operational uses.” The Tracking Not Required series demonstrates how business functionality can be implemented without exposing users to the risks of tracking.

This first post addresses frequency capping in online advertising, the most frequently cited “operational use” necessitating tracking.

Background

When an online advertiser places a bid, it often sets a “frequency cap” on how many times a user may see a particular ad or ad campaign. Many advertising companies implement frequency capping with a unique ID cookie; when a user loads an ad, the ad company looks up past ad views using the ID and imposes frequency caps. The ID cookie approach is understandable: frequency capping becomes a straightforward database lookup. But ID cookies come at a significant privacy cost: they enable effective tracking of a user’s browsing activities.

Algorithm

When the browser loads a page, for each ad slot:

1. The advertising company sends a list of ads it is considering for display, including a frequency cap for each ad. The list is ordered by preference.
2. A script iterates through the list. For each ad in the list, the script checks a local ad viewing history to determine whether the ad is frequency capped. It selects the first ad that is uncapped.
3. The script sends the uncapped ad back to the ad company for display or entry into an ad exchange auction.

When an ad is displayed:

1. The script records the impression in the local ad viewing history.
2. The advertising company bills the ad as usual (per impression, per click, per action,² or a hybrid).

Explanation

Our algorithm leverages several design features to improve the privacy properties of frequency capping.

1. Client-side storage. The ad company does not store the user’s ad viewing history.
2. Query-response. Our algorithm shares information only about ads that might be shown, not all ads the user has seen.
3. Client-side logic. The ad company learns whether certain ads are capped or not; it does not learn their view counts.
4. Server-side preprocessing. Our algorithm shares only the first uncapped ad, not the capping state of each ad in the list.

Our algorithm protects user privacy by limiting the number of states the browser can be in. When $latex m$ ads are under consideration, our algorithm only allows the browser to be in one of $latex m$ states–each of the ads might be selected as the first uncapped ad.³ The maximum information entropy contributed is $latex log_2 m$ bits.^{4, 5}

The discussion has thus far centered on choosing a single ad. An ad company will often select multiple ads of multiple formats when populating a page. The associated maximum information entropy is $latex sum_{i=0}^{k}{log_2{m_i choose n_i}}$ for a page with $latex k$ ad formats, where $latex m_i$ is the number of ads considered in the $latex i$th format and $latex n_i$ is the number of ads selected in the $latex i$th format.

For a concrete example, consider a New York Times article page, which often features a banner graphical ad, a sidebar graphical ad, and two footer text ads. If there are three ad formats and five possible ads for each format, an ad network that populates all four slots would gain at most 7.97 bits of information entropy from frequency capping. In other words, the ad network cannot learn more about the average user than if it had set a one-character cookie!

Implementation

Several advertising companies have objected that privacy-preserving frequency capping is not feasible in implementation. We respectfully disagree.

Performance is a non-issue. Our algorithm imposes negligible requirements on browser storage⁶ and computation.⁷ As for network latency, the approach would at most require an additional roundtrip–and for the many ad companies that already load an ad over multiple roundtrips, it wouldn’t necessitate any extra roundtrip.

There would, of course, be some software engineering required for implementation. The necessary scripting is straightforward; we developed a prototype implementation of our algorithm in hours. (Source is available on GitHub.) Backend changes may be more demanding; ad companies would have to shift from selecting particular ads for display to generating preference-ordered lists of possible ads. While assuredly not trivial, we do not see how the engineering would be unusually complicated.

1. A website’s compliance with our proposals could be externally verified in a number of ways, such as with an automated auditing tool (e.g. FourthParty), through use of a trusted implementation, or by an independent auditing firm.

2. Privacy-preserving per-action and hybrid billing can be accomplished by querying the impression history in local storage when a billable action occurs.

3. We assume here that at least one ad is uncapped, and we assume in the later discussion that at least $latex n_i$ ads are uncapped. One way to guarantee those assumptions is to designate certain ads as fallbacks. If those assumptions do not hold, the maximum information entropy is $latex log_2 left(m+1right)$ bits for a single format, single ad selection and $latex sum_{i=0}^{k}{log_2{sum_{j=0}^{n_i}{m_i choose j}}}$ bits for multiple format, multiple ad selection.

4. This property follows from the Gibbs inequality. The information entropy is $latex log_2 m$ bits only if users are evenly distributed across the state space. Where the state space is dynamic and the distribution of users among states is skewed, both of which are very likely to occur with the approach we propose, the information entropy will be significantly less than $latex log_2 m$ bits.

5. We analyze our algorithm for marginal privacy impact, that is, the extent to which it makes user privacy worse off. IP address and User-Agent already contribute substantial information entropy and allow tracking many users.

6. For example, if an ad is represented by a 4 byte identifier and frequency count is stored in 1 byte, frequency caps for 100,000 ads consume merely 500 KB. HTML 5 local storage can hold at least 5 MB in the major browsers; Indexed Database and Web SQL can store even more.

7. In our testing, modern browsers can perform dozens of lookups in an HTML 5 local storage instance with over 100,000 keys in mere milliseconds.

Abstract

In the early days of the web, content was designed and hosted by a single person, group, or organization. No longer. Webpages are increasingly composed of content from myriad unrelated “third-party” websites in the business of advertising, analytics, social networking, and more. Third-party services have tremendous value: they support free content and facilitate web innovation. But third-party services come at a privacy cost: researchers, civil society organizations, and policymakers have increasingly called attention to how third parties can track a user’s browsing activities across websites.

This paper surveys the current policy debate surrounding third-party web tracking and explains the relevant technology. It also presents the FourthParty web measurement platform and studies we have conducted with it. Our aim is to inform researchers with essential background and tools for contributing to public understanding and policy debates about web tracking.

There has been scant focus on Federal Trade Commission Chairman Jon Leibowitz’s brief remarks on Do Not Track. That’s a mistake.

The FTC was an early, vocal proponent of Do Not Track with its December 2010 preliminary staff report on online privacy. FTC staff have frequently cajoled companies and trade groups to implement Do Not Track, and they have participated in every meeting of the World Wide Web Consortium’s Do Not Track working group. Chairman Leibowitz himself attended a recent meeting in Brussels.

The FTC’s thinking matters. The agency can—and does—bring enforcement actions against web trackers (e.g. Chitika, ScanScout) and the websites that facilitate them (e.g. Facebook). The FTC can call for Do Not Track legislation. Further, under the new White House proposal, the agency would be vested with both veto power over self-regulatory codes and enforcement authority for baseline privacy requirements.

FTC commissioners tend to shy from staking out their individual policy views. The FTC is a law enforcement agency, and it only “speaks” through a vote of the commission. Chairman Leibowitz in particular has relied on subtlety and nuance in his policy addresses.

Last Thursday’s speech was unusually direct. Chairman Leibowitz gave the clearest articulation yet of his thinking on Do Not Track—and he got it completely right. Here’s a summary.

• The FTC does policy, not just enforcement. And it’s the agency that will continue to lead on Do Not Track, not the White House or the Commerce Department.
  

  Since our founding in 1914, the FTC also has had a policy function, which has focused recently on privacy.

  With the encouragement of this Administration – which has so keenly recognized the link between protecting consumer privacy online and engendering consumer trust in Internet commerce – an impressive public-private partnership has made a beginning, coming together around one small agency’s Do Not Track initiative.

• A consumer’s Do Not Track preference must do more than stop behavioral ad targeting.

  We envisioned a [Do Not Track] mechanism that would . . . allow consumers to limit how much data is gathered about them online (and not just how many targeted ads they see) . . . .

• At present, online advertising self-regulation only stops behavioral ad targeting. (Some stakeholders have termed the program “Do Not Target.”)
  

  For the past several years, the online advertising industry has been working to develop an icon that consumers could click to opt out of receiving targeted ads.

• A consumer’s Do Not Track preference must affect all third-party websites, including advertising companies, analytics services, and social networks.
  

  While these developments are encouraging, we still need to ensure that all companies that track users – not just advertisers – are at the table.

• The World Wide Web Consortium is the multi-stakeholder forum for establishing Do Not Track technology and policy.
  

  To that end, the World Wide Web Consortium (W3C), an Internet standards-setting body, gathered engineers, consumer groups, and participants across the broad technology industry to create a universal standard for Do Not Track. We look forward to their deliberations also bearing fruit over the coming year.

• The FTC will continue to enforce on web tracking issues, especially when a company violates a consent order with the agency.
  

  Today, although it is still a work in progress, the ad industry has obtained buy-in from companies that deliver 90 percent of online behavioral advertisements; and, with the Better Business Bureau, it has established a mechanism with teeth to address non-compliance, backed up with FTC enforcement. Said differently, if they don’t enforce it, we will.

  Most notably, last year, two of the largest Internet companies entered into consent orders with the FTC that require both to honor their privacy commitments to hundreds of millions of consumers worldwide and to hire outside auditors to monitor their privacy practices.

• The Do Not Track technology is the “DNT: 1” preference signaling mechanism, and consumers are already using it. Microsoft’s Tracking Protection List technology has merit, but it’s a different proposal.
  

  In a related effort, very early on the companies that make web browsers stepped up to our challenge to give consumers choice about how they are tracked online, sometimes known as the browser header approach. Just after the FTC’s call for Do Not Track, Microsoft developed a system to let users of Internet Explorer prevent tracking by different companies and sites. Mozilla introduced a Do Not Track privacy control for its Firefox browser that an impressive number of consumers have adopted; Apple included a similar Do Not Track control in Safari.

Implications

European policymakers have already articulated their positions on advertising self-regulation and Do Not Track. In December the European Union’s Data Protection Working Party issued a formal opinion that found current advertising self-regulation inadequate under EU privacy law and that signaled support for the W3C’s Do Not Track standards process. European Commission Vice-President Neelie Kroes indicated in January that she shares those views; she reaffirmed her position in response to last week’s event.

Now the head of the chief U.S. consumer protection agency has chimed in, and he agrees. The FTC’s upcoming report on consumer privacy online will clarify where the other commissioners stand.

There is an increasingly clear transatlantic consensus: online advertising self-regulation is not enough. The W3C’s Do Not Track standards process is the way forward for providing meaningful consumer choice about third-party web tracking.

Thanks to Ashkan Soltani, Chris Soghoian, and ★★★★★ for conversations that informed this post. Thanks also to Lee Tien for input on a draft. All views, especially wrong ones, are my own.

To begin, I’d like to lend some structure to ongoing policy discussions by unpacking the four business practices that are at issue.

1. Social advertising. Google is leveraging user account information to personalize its advertising on non-Google websites. To do that, Google now identifies its users when they view ads on non-Google websites.
2. Social advertising circumvention. Google intentionally bypassed Safari’s cookie blocking feature to place an identifying cookie that it uses for social advertising.
3. Ordinary advertising circumvention. Google’s social circumvention had a collateral effect: it enabled Google to place its ordinary advertising tracking cookie.
4. Representation. A Google instructional webpage claimed that Safari’s cookie blocking feature “effectively accomplishes the same thing” as opting out of Google’s advertising cookies.

   

   Safari Advertising Cookie Opt-Out Instructions

I’d next like to clarify some key points about our findings.

• No account, login, or user preference was required for circumvention. The circumvention behaviors affected all users, independent of whether they had a Google account, were logged into a Google account, or had made a choice about social advertising.
• Identifying and identifiable information was collected. Google’s social advertising technology is designed to identify the user—that’s how it shows your friends’ pictures! Google’s design document provides additional detail on the feature. For discussion of how third-party web tracking is in general not anonymous, see Arvind Narayanan‘s explanation “There is no such thing as anonymous web tracking” and our research on identifying information leakage.
• Circumvention is not a commonly accepted business practice. We only identified four advertising companies that deployed technology for circumventing Safari’s cookie blocking, and all have since stopped the practice. Furthermore, a self-regulatory organization for the online advertising industry cites Safari’s cookie blocking feature as a way to stop cookies from advertising companies: “[Safari's] default setting will block all third-party cookies, including those of our member ad networks and those of other, non-member ad networks.”
• Apple’s intent was to block advertising-related tracking. The language in Safari’s preferences menu, Apple’s promotional materials, and developer discussions all indicate that advertising-related tracking was a central motivation for the cookie blocking feature.
• Apple’s purpose was not messing with Google. The default cookie blocking feature that Google circumvented was implemented in Safari 1.0, which shipped in 2003—long before Google was in the third-party display advertising business, and long before relations between the companies soured over smartphones. Furthermore, Safari has repeatedly been a pioneer in browser privacy. Safari 1.0 included a simple “privacy reset” choice for clearing browser settings; the other major browsers followed with similar features. Safari 2.0, released in 2005, was the first browser to provide a “private browsing” mode; again, all the other major browsers followed.
• No +1 button was visible on circumvention ads. We never saw an ad with the +1 button in our testing. The circumvention behaviors occurred in ordinary-looking ads. In the special case of YouTube’s homepage, there was no visible ad at all.
• Circumvention was not needed for social sharing. Google’s circumvention was not necessary to make the +1 button clickable. (For the geeks in the audience: Google could have trivially routed clicks through google.com.) The circumvention was only needed¹ to personalize ads—for example, to show friends’ pictures near the +1 button, or in future to target ads based on Google+ social networking data.
• Users likely did not understand their social advertising setting. New users are by default opted into social advertising on signup.

  

  Social Advertising Default on Account Signup

  My understanding is that users with accounts predating the +1 button have social advertising disabled, but are eventually prompted about the setting with “Enable” selected by default. Disabling the feature requires going to Accounts → Google+, locating the buried “+1 on non-Google sites” setting, then toggling it to “Disable”. Google’s description of the feature does not clearly communicate that it allows Google to identify the user on non-Google websites. The description also does not indicate that the feature would override a browser privacy setting.

  

  Social Advertising Opt-Out Location

  

  Social Advertising Opt-Out Page
• Google’s circumvention only affected Google services. It did not allow other advertising companies to track the user.

Finally, I’d like to note a couple questions that remain open for Google.

• Users impacted. Our measurement data suggests a great number of Safari users may have been affected by Google’s circumvention. Google has not yet indicated how many users were impacted.
• Profit. Google held an advantage over its advertising competitors that did not track Safari browsers. That advantage may have resulted in profit. Google has not yet publicized an estimate of its income from tracking Safari browsers.

Google circulated the following statement to media outlets and policymakers on Friday. The company did not post the statement on its website, and my understanding is that Google representatives declined to answer questions about the statement.

The Journal mischaracterizes what happened and why. We used known Safari functionality to provide features that signed-in Google users had enabled. It’s important to stress that these advertising cookies do not collect personal information.

Unlike other major browsers, Apple’s Safari browser blocks third-party cookies by default. However, Safari enables many web features for its users that rely on third parties and third-party cookies, such as “Like” buttons. Last year, we began using this functionality to enable features for signed-in Google users on Safari who had opted to see personalized ads and other content–such as the ability to “+1” things that interest them.

To enable these features, we created a temporary communication link between Safari browsers and Google’s servers, so that we could ascertain whether Safari users were also signed into Google, and had opted for this type of personalization. But we designed this so that the information passing between the user’s Safari browser and Google’s servers was anonymous–effectively creating a barrier between their personal information and the web content they browse.

However, the Safari browser contained functionality that then enabled other Google advertising cookies to be set on the browser. We didn’t anticipate that this would happen, and we have now started removing these advertising cookies from Safari browsers. It’s important to stress that, just as on other browsers, these advertising cookies do not collect personal information.

Users of Internet Explorer, Firefox and Chrome were not affected. Nor were users of any browser (including Safari) who have opted out of our interest-based advertising program using Google’s Ads Preferences Manager.

Thanks to Arvind Narayanan, Ashkan Soltani, Lee Tien, and ★★★★★ for valuable input.

1. This discussion presumes Google would host its social advertising from doubleclick.net instead of google.com. If Google hosted social advertising from google.com there would have been no need to circumvent Safari’s cookie blocking.

In the interest of clearly establishing facts on the ground, this post provides technical analysis of Safari’s cookie blocking feature and the four companies’ practices. It does not address policy or legal issues. (More on that soon.)

Before proceeding further, I want to thank the countless friends and colleagues who provided invaluable feedback on this project. In particular: ★★★★★, whose insights have been vital at every step, and Ashkan Soltani, whose crawling data was instrumental in uncovering PointRoll’s practices and understanding the prevalence of cookie blocking circumvention.

Third-Party Cookie Blocking in Safari

Every popular web browser, save Opera Mini and the Android built-in browser, includes a “third-party cookie blocking” privacy feature. (The remainder of this post uses the term “cookie blocking” for brevity.) These options share a common high-level purpose: impose limits on cookies from “third-party domains,” that is, domains that differ from the “first-party domain” in the browser’s URL bar. In practice, however, implementations vary substantially; for (slightly out-of-date) specifics, see the Center for Democracy and Technology’s 2010 Browser Privacy Features report and Google’s Browser Security Handbook.

Safari’s cookie blocking feature is unique in two ways: its default and its substantive policy.

Unlike every other browser vendor, Apple enables cookie blocking by default. Every iPhone, iPad, iPod Touch, and Mac ships with the privacy feature turned on.

Apple advertises cookie blocking by default as a benefit of choosing Safari.

Some companies track the cookies generated by the websites you visit, so they can gather and sell information about your web activity. Safari is the first browser that blocks these tracking cookies by default, better protecting your privacy. Safari accepts cookies only from the current domain.

Apple’s cookie blocking policy is less restrictive than many competing browser vendors’.¹

• Reading Cookies Safari allows third-party domains to read cookies.
• Modifying Cookies If an HTTP request to a third-party domain includes a cookie, Safari allows the response to write cookies.
• Form Submission If an HTTP request to a third-party domain is caused by the submission of an HTML form, Safari allows the response to write cookies. This component of the policy was removed from WebKit, the open source browser behind Safari, seven months ago by Google engineers. Their rationale is not public; the bug is marked as a security problem. The change has not yet landed in Safari.

These allowances in the Safari cookie blocking policy enable three potentially undesirable behaviors by advertising networks, analytics services, social widgets, and other “third-party websites.”

• If a company operates both a first-party website and a third-party website from the same domain, visitors to the first-party website will be open to cookie-based tracking by the third-party service. Yahoo! is an example: it hosts both first-party websites and third-party advertising services on the yahoo.com domain.
• If a third-party website’s content ever manages to load in a full browser window, the website can set cookies on its domain. An advertising company, for example, could set tracking cookies on its domain with a pop-up, pop-under, or temporary redirect (e.g. when a user clicks an ad).
• A third-party website can use JavaScript to submit a form in an iframe without user interaction.

This post focuses on the last behavior. We discovered four advertising companies that surreptitiously submit a form in an invisible iframe and place trackable cookies in Safari: Google, Vibrant Media, Media Innovation Group, and PointRoll. The balance of the post details each company’s business practices.

Google

Google has, historically, operated most of its first-party websites on the google.com domain and most of its third-party services on other domains. For example: Google Analytics is served from google-analytics.com, Google software libraries are hosted at googleapis.com, Google static content is at gstatic.com, and Google’s advertising services are on doubleclick.net.

Separating first-party websites from third-party services improves security: interactions between google.com content and other websites could introduce vulnerabilities. The domain separation also benefits user privacy: Google associates user account information with google.com cookies. By serving its third-party services from other domains, Google ensures it will not receive google.com cookies, and therefore will not be able to trivially identify user activities on other websites.

But what about when Google does want to identify the user on a non-Google website? Social personalization requires² just that! Google has two design options.

First, Google could embed google.com content on non-Google websites. This is the approach it has taken with its social sharing widgets; both the (defunct) Buzz button and the +1 button load resources from google.com.

Second, Google could synchronize information from its google.com domain to another domain, a process called “cookie syncing” in online advertising lingo. This is the approach Google took with youtube.com after it acquired YouTube. And this is the approach Google settled on for social personalization of doubleclick.net display advertising. Google announced the +1 button for display ads last September; here are the steps in the underlying cookie syncing technology, based on conversations with Googlers and an explanatory document that Google provided.

1. A display advertisement includes the cookie syncing mechanism’s iframe, located at http://googleads.g.doubleclick.net/pagead/drt/s. We observed the iframe load in both desktop and mobile display ads. Here are example ads that included the mechanism, from the Washington Post (Safari on Mac OS X) and MSNBC (Safari on iPhone).

   

   Google Ads Including the google.com → doubleclick.net Cookie Syncing Mechanism

   We also saw what appeared to be a special use of the mechanism on youtube.com, where Google placed an invisible advertisement in the footer that included the cookie syncing iframe.

   In a FourthParty crawl of the homepages of the Alexa U.S. top 500 websites, we detected the cookie syncing mechanism on 39 pages. This figure likely underestimates the prevalence of Google’s cookie syncing code since many websites show less advertising on their homepage. We observed the mechanism on New York Times and MSNBC article pages, for example, but not on their homepages.

2. The iframe loads a page that contains a meta refresh to http://google.com/pagead/drt/ui.
   

   
   <!DOCTYPE HTML PUBLIC>
   
   <html>
   
     <head>
   
       <meta http-equiv="refresh" content="0;url=http://google.com/pagead/drt/ui" />
   
     </head>
   
   </html>
   
   

   Apologies for any overflow; here and throughout the post I err on the side of preserving original formatting.

3. The response at http://google.com/pagead/drt/ui depends on whether the user is logged into Google. If the user is not logged in, the response includes a Location header that directs the browser back to googleads.g.doubleclick.net with some information in the p and ut parameters of the Request-URI.
   

   
   Location: https://googleads.g.doubleclick.net/pagead/drt/si?p=CAA&ut=AFAKxlQAAAAATzuSTM-wZva6TmRV_FF7YdF2nggZfnlI
   
   

   If the user is logged in, the response directs the user to Google’s authentication service.

   
   Location: https://accounts.google.com/ServiceLogin?service=doritos&passive=true&go=true&continue=https://googleads.g.doubleclick.net/pagead/drt/si?p=CAEY9cLA-gQ&ut=AFAKxlQAAAAATz2v-fyl5V0PcBdEsvg95beKTozmJSql
   
   

   The authentication service then directs the user back to googleads.g.doubleclick.net.

   (A quick explanation of the “Doritos” reference—as I understand it, that’s Google’s internal codename for social personalization of third-party display advertising.)

   
   location:https://googleads.g.doubleclick.net/pagead/drt/si?p=CAEY9cLA-gQ&ut=AFAKxlQAAAAATz2v-fyl5V0PcBdEsvg95beKTozmJSql&pli=1&auth=DQAAAIUAAAAPNIlph4K8ZDuUPlslr38CnSgqvc7E26I5RwkOrDDU7r81Q8H6iVYltrf4TEcE1haR9gSXQuARTXXHSWIW6EnmOyb2inWlPm28lprT6Hmkhn_PzhpuYlNUrSFZ9RdOAdro-hHVwMHQojKjOSSkQxQIIvetbMiMIOTcK88Ltq7Td9rQHLHJ_QrNb7EDz727XUM
   
   

   Google’s documentation suggests that the p and ut parameters include an encryption of the user’s login state and, if logged in, account ID. (The Google design document makes a number of technical claims about how the syncing mechanism preserves user privacy. This post does not address those claims.)

4. In a browser other than Safari, the response sets a “_drt_” social personalization cookie on .doubleclick.net. If the user is not logged into Google, the cookie’s value is “NO_DATA”, and the cookie is set to expire in 12 hours.
   

   
   set-cookie:_drt_=NO_DATA; expires=Fri, 17-Feb-2012 13:37:41 GMT; path=/; domain=.doubleclick.net; HttpOnly
   
   

   If the user is logged into Google, the “_drt_” cookie contains an encryption of the user’s Google account ID, set to expire in 24 hours.

   
   set-cookie:_drt_=AFkicjesF-jVECSOLRa1a-hf14FYVKIPEu4goDlxZZdVaxh1D4gDfJ6dvZg7Evnr2C8MluBSk6Nkr8TfL1ksojSb8qsjYHSNMQ; expires=Sat, 18-Feb-2012 01:25:10 GMT; path=/; domain=.doubleclick.net; HttpOnly
   
   

   My understanding is that the cookie expirations are set to limit syncing frequency. If the user was not logged in at last sync, a sync will not be attempted for at least 12 hours; if the user was logged in, a sync will not be attempted for at least 24 hours.

   In early testing, we a noticed a “PREF” cookie was sometimes set at the same time as the “_drt_” cookie.

   
   set-cookie:PREF=ID=5a7be344032983bc:TM=1325643281:LM=1325643281:S=-BWpqDzbE7gq8rg-; expires=Fri, 03-Jan-2014 02:14:41 GMT; path=/; domain=googleads.g.doubleclick.net
   
   

   The behavior appeared to stop before we contacted Google about our findings. We have not received information from Google explaining the “PREF” cookie on googleads.g.doubleclick.net. It appears to have the same format as the “PREF” cookie on google.com and the same two-year expiration.

So far, a (relatively) straightforward cookie syncing mechanism. But we noticed a special response at the last step for Safari browsers. We tested 400 User-Agent strings to verify that this is a special case; a spreadsheet of results is available.

Instead of responding with the “_drt_” cookie, the server sends back a page that includes a form and JavaScript to submit the form (using POST) to its own URL.

<html>

<head></head>

<body>

  <form id="drt_form" method=post action="/pagead/drt/si?p=CAA&amp;ut=AFAKxlQAAAAATzuSTM-wZva6TmRV_FF7YdF2nggZfnlI"></form>

  <script>

    document.getElementById('drt_form').submit();

  </script>

</body>

</html>

The response to the form submission then includes the Set-Cookie header for the “_drt_” cookie.

Recall that if a cookie is sent with an HTTP request, Safari’s blocking policy will allow the response to write cookies. Owing to the “_drt_” cookie, all doubleclick.net content is now immunized from Safari’s cookie blocking policy. The next time Google advertising content attempts to install the “id” tracking cookie for .doubleclick.net, it will successfully set. That next attempt may not even require that the user visit another page: We noticed that many Google ads periodically send requests to doubleclick.net, especially to a URL with the base http://ad.doubleclick.net/activity. A response to one of these requests can include a Set-Cookie header for the “test_cookie” cookie, which Google uses to make sure cookies successfully set (presumably to avoid wasting IDs and associated resources). A response to a subsequent request may then include a Set-Cookie header for the “id” cookie.

We confirmed that Google’s doubleclick.net “id” cookie was functioning in Safari by observing behavioral interest categories appear in Google Ads Preferences. Here is an example set of inferred interests after browsing the New York Times website.

Vibrant Media

Vibrant Media is a contextual advertising network that primarily offers in-text and display advertising. We found conclusive evidence that Vibrant deliberately circumvents Safari’s third-party cookie blocking feature: one of the URLs involved in the circumvention is for the resource /safari.jsp. The following steps describe the circumvention technology as deployed on answers.com. We observed identical behavior at the various region-specific subdomains of cbslocal.com.

1. Vibrant’s main advertising script loads from http://answers.us.intellitxt.com/intellitxt/front.asp?ipid=31690. When the browser has a Safari User-Agent string and no Vibrant cookie, the script includes this code:
   

   
   (function()
   
   {try
   
   {var e=document.createElement('iframe');e.style.display='none';e.src='http://answers.us.intellitxt.com/safari.jsp?t='+(new Date()).getTime();var b=document.getElementsByTagName('body')[0];b.insertBefore(e,b.firstChild);}catch(x){}})();
   
   

2. The Safari-specific code executes, adding an invisible iframe to the page.
   

   
   <iframe style="display: none;" src="http://answers.us.intellitext.com/safari.jsp?t=1328762901524"></iframe>
   
   

   The Request-URI parameter t is the current time in milliseconds, presumably used to prevent caching.

3. The iframe contains a form and a body onload handler that submits the form.
   

   
   <html><head></head><body onload="document.getElementById('myform').submit();"><form method="post" action="http://answers.us.intellitxt.com/safari.jsp?x=1&t=1328762901524" id="myform"></form></body></html>
   
   

4. The response to the form contains no content and an instruction to set a Vibrant ID cookie.
   

   
   Set-Cookie: VM_USR=AG75nlrejUwdiE6n3+naS1YAADwZAAA8VQEAAAE1gRigFAA-; Domain=.intellitxt.com; Expires=[now + 2 months]; Path=/
   
   

   The Request-URI parameter x=1 appears to control whether the response includes the form page or a Set-Cookie header.

   We confirmed that the “VM_USR” cookie is a Vibrant ID by checking the Network Advertising Initiative‘s cookie status page.

   

   Active Vibrant Media Cookie Status Indicator

   We verified that the NAI indicator is based on the presence of a valid “VM_USR” cookie, not the presence of any cookie or any “VM_USR” cookie.

Media Innovation Group

Media Innovation Group (MIG) is an advertising technology provider within the WPP family of companies. MIG’s “Zeus Advertising Platform” (ZAP) is WPP’s “integrated advertising and analytics platform”. According to a report from a vendor, ZAP “is one of the cornerstone products created by MIG” that “provides a holistic view of site analytics and campaign data for a comprehensive understanding of every individual consumer.” ZAP “collects and stores over 13 months of historical user-level data and draws from it to provide complex and robust analysis.” With ZAP, “MIG is currently tracking the effectiveness of every single advertising element within many live campaigns that reach hundreds of millions of unique users per month . . . .”

We found that some MIG advertising content included a script that circumvents Safari’s cookie blocking feature. Here is the relevant part of one such script we discovered at http://b3.mookie1.com/2/B3DM/DLX/1672705484@x71. A few clarifying notes: mookie1.com is a MIG domain (go figure), is_http stores whether the content is loaded over HTTP, and ZAP_id stores the “id” cookie.

if(is_http) {

    if(ZAP_id.indexOf(':') != -1 || ZAP_id == '') {

        var firstTimeSession = 0;



        function submitSessionForm() {

            if (firstTimeSession == 0) {

                firstTimeSession = 1;

                $("#sessionform").submit();

                //setTimeout(processApplication(),2000);

            }

        }



        $("body").append('<iframe id="sessionframe" name="sessionframe" onload="submitSessionForm()" src="http://t.mookie1.com/t/v1/imp?" style="display:none;"></iframe><form id="sessionform" enctype="application/x-www-form-urlencoded" action="http://t.mookie1.com/t/v1/imp?" target="sessionframe" method="POST"></form>');



        function processApplication() {

        }

    }

The script creates an invisible iframe and form, then submits the form into the iframe during the onload handler for the iframe.

In response to the form submission, MIG sets cookies and redirects to a 1×1 GIF.

$ curl -i -L -X POST "http://t.mookie1.com/t/v1/imp?"

HTTP/1.1 302 Found

Date: Fri, 17 Feb 2012 09:48:03 GMT

Server: Apache/2.0.52 (Red Hat)

Cache-Control: no-cache

Pragma: no-cache

P3P: CP="NOI DSP COR NID CUR OUR NOR"

Set-Cookie: id=3025894295853070; path=/; expires=Mon, 18-Mar-13 09:48:03 GMT; path=/; domain=.mookie1.com

Set-Cookie: mdata=1|3025894295853070|1329472083; path=/; expires=Mon, 18-Mar-13 09:48:03 GMT; path=/; domain=.mookie1.com

Set-Cookie: OAX=nVuS508+IlMACEDl; path=/; expires=Mon, 18-Mar-13 09:48:03 GMT; path=/; domain=.mookie1.com

Location: /t/v1/imp/cc?

Content-Length: 277

Connection: close

Content-Type: text/html; charset=iso-8859-1



HTTP/1.1 200 OK

Date: Fri, 17 Feb 2012 09:48:03 GMT

Server: Apache/2.0.52 (Red Hat)

Cache-Control: no-cache

Pragma: no-cache

P3P: CP="NOI DSP COR NID CUR OUR NOR"

Set-Cookie: id=914844815541839; path=/; expires=Mon, 18-Mar-13 09:48:03 GMT; path=/; domain=.mookie1.com

Set-Cookie: mdata=1|914844815541839|1329472083; path=/; expires=Mon, 18-Mar-13 09:48:03 GMT; path=/; domain=.mookie1.com

Set-Cookie: OAX=T6AK5U8+IlMACoIB; path=/; expires=Mon, 18-Mar-13 09:48:03 GMT; path=/; domain=.mookie1.com

Content-Length: 35

Connection: close

Content-Type: image/gif



GIF87a???????,D;

Comments in MIG’s script indicate that “id” is the ZAP ID cookie and “OAX” is the ID cookie for WPP’s B3 advertising optimization and custom marketplace product. We verified that the script sets a tracking cookie with MIG’s NAI status indicator.

MIG’s circumvention code appeared (relatively) infrequently; our crawl of the Alexa U.S. top 500 homepages located it on five sites. It is unclear whether MIG served this script only to Safari users. While we did not see the MIG code in any non-Safari browsers, that may have been due to insufficient sample size; we were not able to reliably cause the MIG code to appear.

That said, we believe it is nevertheless reasonable to infer that MIG’s circumvention was intentional. MIG’s code appears to be based on widely-cited sample code by web developer Anant Garg. Even Facebook’s developer documentation points to the sample.

var isSafari = (/Safari/.test(navigator.userAgent));

var firstTimeSession = 0;



function submitSessionForm() {

	if (firstTimeSession == 0) {

		firstTimeSession = 1;

		$("#sessionform").submit();

		setTimeout(processApplication(),2000);

  	}

}



if (isSafari) {

	$("body").append('<iframe id="sessionframe" name="sessionframe" onload="submitSessionForm()" src="http://www.yourdomain.com/blank.php" style="display:none;"></iframe><form id="sessionform" enctype="application/x-www-form-urlencoded" action="http://www.yourdomain.com/startsession.php" target="sessionframe" action="post"></form>');

} else {

	processApplication();

}



function processApplication() {

	alert('Session has been set. Now you can start your application!');

}

The resemblance is uncanny. The scripts share the exact same variable names, structure, logic, and library dependency (jQuery 1.3.2 on Google Libraries). Even more compelling, MIG commented out a line that it didn’t need!

PointRoll

PointRoll is a rich media advertising company owned by Gannett. PointRoll’s corporate website claims that it “[p]ower[s] 55% of all rich media campaigns online” and “serv[es] over 450 billion impressions for more than two-thirds of the Fortune 500 brands . . . .”

We found that a PointRoll cookie helper script circumvents Safari’s cookie blocking. One instance of the script we studied is at http://ads.pointroll.com/PortalServe/?pid=1574300Y14520120126002933&flash=11&time=4|13:53|-8&redir=http://at.atwola.com/adlink/5113/2209587/0/2392/AdId=2327012;BnId=1;itime=219618215;nodecode=yes;link=$CTURL$&pos=s&postal=94305&r=0.18630846054557448.

Here’s the relevant part of the script. Unlike the other examples, this code was passed through a formatter—it’s otherwise unreadable.

function submitSessionForm(name) {

    var sessionForm = document.getElementById(name);

    if (typeof (sessionForm) != 'undefined') {

        var txtStatus = document.getElementById('txt_' + name);

        if (txtStatus.value == 'UNSUBMITTED') {

            txtStatus.value = 'SUBMITTED';

            console.log("form " + name + " Submitted");

            sessionForm.submit();

        }

    }

}

function prCook(name, value, date, dom) {

    console.log("add form: name=" + name + ": value=" + value + ": date=" + date + ": dom=" + dom);

    var date = (typeof (date) != "undefined") ? date : "Fri, 14-Feb-2014 14:47:07 GMT";

    var dom = (typeof (dom) != "undefined") ? dom : "ads.pointroll.com";

    var sCook = '<iframe id="' + name + '_frame" name="' + name + '_frame" onload="submitSessionForm('';

    sCook += name;

    sCook += '')" src="http://ads.pointroll.com/clients/pointroll/cookie/blank.aspx" style="display:none;"></iframe>';

    sCook += '<form id="';

    sCook += name;

    sCook += '" style="display:none;" enctype="application/x-www-form-urlencoded" action="http://ads.pointroll.com/clients/pointroll/cookie/drop.ashx" target="' + name + '_frame" action="post">';

    sCook += '<input type="text" name="name" value="' + name + '" />';

    sCook += '<input type="text" name="date" value="' + date + '" />';

    sCook += '<input type="text" name="value" value="' + value + '" />';

    sCook += '<input type="text" name="domain" value="' + dom + '" />';

    sCook += '<input type="text" id="txt_';

    sCook += name;

    sCook += '" status" value="UNSUBMITTED" />';

    sCook += '</form>';

    var d = document.createElement('DIV'),

        p = document.getElementsByTagName('BODY');

    d.innerHTML = sCook;

    if (p.length < 1) {

        p = document.getElementsByTagName('HTML');

    }

    p[0].appendChild(d);

The script provides a cookie setting function, prCook. When called, the function creates a new div and places within it an invisible iframe and form with the cookie fields specified by the input parameters. An onload handler on the iframe submits the form into the iframe. For example, the call

prCook('example_cookie_name', 'example_cookie_value', 'Fri, 14-Feb-2014 14:47:07 GMT', 'exampledomain.com')

would result in this code being added in a new div element:

<iframe id="example_cookie_name_frame" name="example_cookie_name_frame" onload="submitSessionForm('example_cookie_name')" src="http://ads.pointroll.com/clients/pointroll/cookie/blank.aspx" style="display:none;"></iframe>

<form id="example_cookie_name" style="display:none" enctype="application/x-www-form-urlencoded" action="http://ads.pointroll.com/clients/pointroll/cookie/drop.ashx" target="example_cookie_name_frame" action="post">

   <input type="text" name="name" value="example_cookie_name" />

   <input type="text" name="date" value="Fri, 14-Feb-2014 14:47:07 GMT" />

   <input type="text" name="value" value="example_cookie_value" />

   <input type="text" name="domain" value="exampledomain.com" />

   <input type="text" id="txt_example_cookie_name" status" value="UNSUBMITTED" >

</form>

Here is the response when the form is submitted.

$ curl -i "http://ads.pointroll.com/clients/pointroll/cookie/drop.ashx?name=example_cookie_name&date=Fri%2C%2014-Feb-2014%2014%3A47%3A07%20GMT&value=example_cookie_value&domain=exampledomain.com"

HTTP/1.1 200 OK

Connection: close

Date: Fri, 17 Feb 2012 07:41:13 GMT

Server: Microsoft-IIS/6.0

P3P: CP="NOI DSP COR PSAo PSDo OUR BUS OTC"

Access-Control-Allow-Origin: *

X-AspNet-Version: 2.0.50727

Pragma: no-cache

p3p: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"

Set-Cookie: example_cookie_name=example_cookie_value; domain=exampledomain.com; expires=Fri, 14-Feb-2014 14:47:07 GMT; path=/

Cache-Control: private

Content-Type: text/html; charset=utf-8

Content-Length: 145



<script>console.log('drop details: cookie name=example_cookie_name; cookie value=example_cookie_value; cookie domain=exampledomain.com')

The response includes a Set-Cookie header with the values from the form. (And introduces a cross-site scripting vulnerability.)

PointRoll’s script includes code for setting 9 cookies.

prCook('PRID', 'EC7B3EB9-DE19-4A32-AC07-83856AB7AE98', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

prCook('PRbu', 'EuQHJKn7z', 'Fri, 14-Feb-2014 14:47:07 GMT', '.pointroll.com');

prCook('PRgo', 'BAA', 'Fri, 14-Feb-2014 14:47:07 GMT', '.pointroll.com');

prCook('PRimp', '01B90400-2AC9-F37E-0209-DAC000190101', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

prCook('PRca', '|AK5C*37611:1|#', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

prCook('PRcp', '|AK5CAJmd:1|#', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

prCook('PRpl', '|Fhtv:1|#', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

prCook('PRcr', '|GTnd:1|#', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

prCook('PRpc', '|FhtvGTnd:1|#', 'Fri, 14-Feb-2014 14:47:07 GMT', 'ads.pointroll.com');

It would seem reasonable to infer that the two-year “PRID” cookie is PointRoll’s unique ID cookie.

As with the MIG code, we did not have a large enough sample size to conclusively determine that this script was sent only to Safari browsers. But, again, we were able to deduce that the cookie blocking circumvention was intentional by comparison to Anant Garg’s example code. There are several telltale signs of copying.

• Structure The script is structured in the same way: an onload handler function followed by the iframe and form.
• Variable Names Both use the variable sessionForm and the handler function submitSessionForm.
• HTML Elements Both use the same attributes, attribute ordering, and coding style in their iframe and form elements. (The PointRoll code includes an additional style attribute in its form element.)
• POST Bug If you’re still unconvinced, here’s the giveaway: the scripts have the same bug. Both say action="post" (which does nothing, apparently) instead of method="post" (which sets the form to use the POST method instead of the GET method). The author of the example code corrected himself in a comment.
  

  “action” is correct for the target script in an HTML form. However, there is a second “action” in the code that should be “method” instead: method=”post” (not action=”post”).

We found PointRoll’s code on 19 homepages in the Alexa U.S. top 500.

Conclusion

When Apple’s developers implemented Safari’s cookie blocking feature, they were balancing several conflicting design priorities. But one decision was clear: it should prevent advertising companies from tracking the user. As a lead developer noted, whatever the implications for other websites, “I do think it would typically stop, say, doubleclick.net from tracking you . . . .”

Four advertising companies circumvented Apple’s protection. Some privacy researchers and advocates have characterized the interplay between third-party web trackers and browser privacy measures as a “cat and mouse game” or “arms race.” This research result regrettably affirms that view as reality—for, quite possibly, millions of users.

1. This post does not address the merits of Safari’s cookie blocking policy.

2. There are some options for privacy-preserving social personalization. But they are far beyond the scope of this post.

Yesterday the Digital Advertising Alliance (DAA) announced a supplementary set of self-regulatory principles for third parties on the web (pdf, press release). This post is a brief — and far from comprehensive — overview of improvements, continued deficiencies, and procedural issues.

Improvements

1. Several sensitive uses of third-party web tracking data are now completely prohibited: adverse terms or ineligibility for employment, credit, medical treatment, and insurance. The principles do not, however, prohibit offering favorable terms or determining eligibility from third-party web tracking data.

2. Transparency and consumer control are now required for many forms of per-device content personalization, not just behaviorally targeted advertising. The principles are ambiguous about whether per-user content personalization, such as Facebook social widgets, also requires transparency and consumer control.

Continuing Substantive Deficiencies

1. Many stakeholders on online privacy, including U.S. and EU regulators, have repeatedly emphasized that effective consumer control necessitates restrictions on the collection of information, not just prohibitions on specific uses of information. The very existence of third-party web tracking data gives rise to numerous privacy risks, including data breach, employee misconduct, government access, and more. The DAA principles nevertheless remain a set of limitations on data use, not data collection. While the supplementary principles begin with broad language about collection limits, they incorporate vast exceptions that wholly swallow the rule. Consider, for example, the exceptions for “market research,” “product development,” and “reporting.”

Market Research means the analysis of: market segmentation or trends; consumer preferences and behaviors; research about consumers, products, or services; or the effectiveness of marketing or advertising.

Product Development means the analysis of: (i) the characteristics of a market or group of consumers; or (ii) the performance of a product, service or feature, in order to improve existing products or services or to develop new products or services.

Reporting is the logging of Multi-Site Data on a Web site(s) . . . for:

• Statistical reporting in connection with the activity on a Web site(s);

. . .

In a plain reading, every third-party web tracking practice would come within these exceptions to mandatory consumer control. (A simple thought experiment: name a third-party web tracking practice that is not encompassed by the provisions above.) Per-device personalization uses of data are not excepted from consumer control only because the principles explicitly add exceptions to the exceptions. Here is the language on “market research” and “product development.”

A key characteristic of market research is that the data is not re-identified to market directly back to, or otherwise re-contact a specific computer or device. Thus, the term “market research” does not include sales, promotional, or marketing activities directed at a specific computer or device.

Like Multi-Site Data used for Market Research, such data used for product development is not re-identified to market directly back to, or otherwise re-contact a specific computer or device.

2. Practices that do not include per-device personalization are not only exempted from consumer control, but are also exempted from any transparency requirement.

3. The DAA has not closed the loophole in its principles for data sharing among corporate affiliates.

4. Despite lack of adoption and widespread criticism, the DAA continues to advocate its opt-out cookie and icon mechanisms.

Procedural Issues

1. The supplementary principles were developed through an opaque process, with limited input from policymakers, researchers, and civil society organizations. Legitimate self-regulation transparently and inclusively addresses consumer concerns; it does not present a fait accomplis.

2. It is unclear why the DAA, as a consortium of organizations in the online advertising space, would have a legitimate claim to regulate third-party web tracking that is not related to advertising. The new principles may, in fact, run contrary to the current policy positions of several companies, including Facebook. It remains to be seen how many non-advertising third parties will accept the DAA’s principles.

In sum: It’s great to see the online advertising industry taking steps in the right direction. There are a few real improvements in the supplementary principles. But they do not address the core privacy issues consistently raised by regulators, legislators, researchers, and advocates, and it is far from clear that the online advertising industry will be able to expand the scope of its program beyond advertising practices.

Thanks to Peter Eckersley at the Electronic Frontier Foundation for reviewing a draft.

Click the local Home Depot ad and your email address gets handed to a dozen companies monitoring you. Your web browsing, past, present, and future, is now associated with your identity. Swap photos with friends on Photobucket and clue a couple dozen more into your username. Keep tabs on your favorite teams with Bleacher Report and you pass your full name to a dozen again. This isn’t a 1984-esque scaremongering hypothetical. This is what’s happening today.

[Update 10/11: Since several readers have asked – this study was funded exclusively by Stanford University and research grants to the Stanford Security Lab. It was not supported by any advocacy organization.]

Background on Third-Party Web Tracking and Anonymity

In a post on the Stanford CIS blog two months ago, Arvind Narayanan explained how third-party web tracking is not at all anonymous.

In the language of computer science, clickstreams – browsing histories that companies collect – are not anonymous at all; rather, they are pseudonymous. The latter term is not only more technically appropriate, it is much more reflective of the fact that at any point after the data has been collected, the tracking company might try to attach an identity to the pseudonym (unique ID) that your data is labeled with. Thus, identification of a user affects not only future tracking, but also retroactively affects the data that’s already been collected. Identification needs to happen only once, ever, per user.

Arvind noted five ways in which a user’s identity may be associated with third-party web tracking data.

• A third party is also a first party, e.g. Facebook, Twitter, or Google+.
• A first party hands off (“leaks”) identifying information to a third party.
• A third party buys identifying information from a “matching service.”
• A third party exploits a security vulnerability to learn a user’s identity.
• A third party “deanonymizes” its data by matching it against identified data.

This post is an empirical study of identifying information leakage from first-party websites to third-party websites.¹

Web Information Leakage

Leakage most often occurs when a first-party website stuffs information into a URL. For example, suppose Example Website sends users after they register to:

http://example.com/register?

username=GoCardinal

&name=Leland%20Stanford

&email=leland%40stanford.edu

&…

Third parties embedded in the page will receive the URL in a referrer header or equivalent² – and therefore Leland Stanford’s username, name, and email.

Another common form of leakage is through the page title. Suppose a website’s landing page includes a title tag of:

Welcome, Leland Stanford!

Embedded third-party scripts often report back with the page title; in this case, they’d include Leland Stanford’s name.

[Update 10/11: The original version of this post conflated the information OkCupid provides to Lotame and BlueKai. In the interest of complete accuracy, and in response to both a deluge of questions on OkCupid's intentional leakage and a note from BlueKai seeking clarification, I have updated this section with per-company intentional leakage. I have also included the results of a leakage test (with the methodology described below) on OkCupid. My apologies to BlueKai for the incorrect implication that it collects the same sensitive profile data that Lotame does. The amibiguous discussion was solely my error.]

Leakage, in common parlance, implies unintentionality. In computer security, leakage is a term of art for an information flow – some instances of leakage are entirely intentional. For example, OkCupid, a free online dating website, appears to sell user profile information to the data providers BlueKai and Lotame. , including gender, age, ZIP code, relationship status, and drug use frequency. To learn which profile information OkCupid leaks, I modified each field of a profile and observed how values sent to the two companies changed. Here’s what the companies appeared to receive:

Age – Both

Cats – Both

Children – Both

Country – Both

Dogs – Both

Drinking Frequency – Lotame

Drug Use Frequency – Lotame

Education – Both

Ethnicity – Lotame

Gender – Both

Income – Both

Job Sector – Both

Language Proficiencies – BlueKai

Relationship Status – Lotame

Religion – Lotame

Smoking Frequency – Lotame

State – Both

ZIP Code – Both

(I also ran the leakage test described below on OkCupid. The username was sent to 27 third-party PS+1s (defined below), including crwdcntrl.net (Lotame) and bluekai.com (BlueKai). Since OkCupid does not limit who can see a profile – a user can only require that visitors be logged in – a username provides access to a user’s entire profile.)

In a series of groundbreaking studies Balachander Krishnamurthy, Craig Wills, and Konstantin Naryshkin have demonstrated that information leakage is a pervasive problem (1, 2, 3). In their most recent paper, the authors examined signup and interaction with 120 popular sites for information leakage to third parties. They found that 56% leaked some form of private information, and 48% leaked a user identifier.

We roughly followed the same methodology as Krishnamurthy, Wills, and Naryshkin, with 1) a focus on identifying information leakage, 2) a greater number of sites, 3) and a public dataset.

Usernames as Identifying Information

Given the sizeable role usernames play in web information leakage, it’s worth taking a moment to note how a username is identifying information. In some cases a username is just a user’s name – for example, @jonathanmayer on Twitter. Even when it isn’t the user’s name, a username is often more than adequate for identifying a user.

First, a username is likely sufficient to link accounts across websites. Users routinely reuse their usernames – after all, who’s going to remember a new login for each site they use? In a paper at PETS 2011, Daniele Perito et al. examined a sample of public data from Google, eBay, and other sites to estimate how linkable usernames are. They found that the vast majority of usernames in their sample had high entropy, and that simple algorithms for linking usernames could achieve pairwise precision and recall of over 70%. (For further discussion of using usernames to link social profiles, see Arvind’s blog posts “The Linkability of Usernames” and “Lendingclub.com: A De-anonymization Walkthrough,” as well as “Modeling Unintended Personal-Information Leakage from Multiple Online Social Networks” and “Large Online Social Footprints – An Emerging Threat” by Danesh Irani et al.) Some companies are already linking usernames in their products, including social matching services (e.g. Infochimps), scraped profiles (e.g. Spokeo), and automated social network linkage (e.g. Google Social Search).³

Second, combining data from multiple accounts often provides a sufficiently comprehensive mosaic to identify an individual.⁴ Arvind, for example, usually goes by the username “randomwalker.” The first page of a Google search turned up his yCombinator Hacker News account, which includes his job and links to his personal website, blog, and Twitter account.

Some websites (e.g. Quantcast) have responsibly recognized that a username is identifying information and have included username in their legal definition of “personally identifiable information” (PII).

Methodology

We examined each website in the Quantcast top 250, checking for whether it

• offered a sign up,
• did not require a purchase or other qualification to sign up, and
• did not include so many features as to be impractical for study.

For each of the 185 websites that met all three criteria, we used the FourthParty web measurement platform to create an account and interact with the site.⁵ We emphasized exploring content that dealt with a user’s identity, such as profile and settings pages. After collecting data, we searched Request-URIs and Referrer headers for known personal information. We treated each public suffix + 1 (PS+1) as an independent entity, and we considered any PS+1 different from a first party’s to be a third party.⁶

Results

A complete spreadsheet of results is available in Excel format. We encourage interested readers to examine the results for themselves. [Update 10/22: Before consulting the spreadsheet, please be sure to read Footnote 6 to understand the limitations of our methodology.] Please email if you would like FourthParty logs for a specific site.

The most frequent type of leakage was a username or user ID.⁷ We identified username or user ID leakage to a third party on 113 websites, 61% of the websites in our sample. The top five PS+1 recipients of username and user ID leakage were:

1. scorecardresearch.com (comScore), on 81 (44%) of the websites in our sample
2. google-analytics.com (Google Analytics), on 78 (42%) of the websites in our sample
3. quantserve.com (Quantcast), on 63 (34%) of the websites in our sample
4. doubleclick.net (Google Advertising), on 62 (34%) of the websites in our sample
5. facebook.com (Facebook), on 45 (24%) of the websites in our sample

Some websites leaked the username or user ID to dozens of third parties. For example, popular photo sharing website Photobucket embeds username in many of its URLs, and includes advertising on most of its pages; we observed the username get sent to 31 third-party PS+1s.

Other identifying information leaked in a number of instances. A sample:

• Viewing a local ad on the Home Depot website sent the user’s first name and email address to 13 companies.
• Entering the wrong password on the Wall Street Journal website sent the user’s email address to 7 companies.

  [Update 10/11: A number of readers have written in noting that the Wall Street Journal leak is not in our spreadsheet. We identified the Wall Street Journal leak in a different browsing session from the one reported in the spreadsheet – and by accident. In the interest of consistency – we did not test logging out and logging back in on other sites, nor logging in with the wrong password – we decided to discuss the leak in our post but not our spreadsheet.]

• Changing user settings on the video sharing site Metacafe sent first name, last name, birthday, email address, physical address, and phone numbers to 2 companies.
• Signing up on the NBC website sent the user’s email address to 7 companies.
• Signing up on Weather Underground sent the user’s email address to 22 companies.
• The mandatory mailing list page during CNBC signup sent the user’s email address to 2 companies.
• Clicking the validation link in the Reuters signup email sent the user’s email address to 5 companies.
• Interacting with Bleacher Report sent the user’s first and last names to 15 companies.
• Interacting with classmates.com sent the user’s first and last names to 22 companies.

Implications

From a legal perspective, identifying information leakage is a debacle. Many first-party websites make what would appear to be incorrect, or at minimum misleading, representations about not sharing PII. Here are some examples.

The Home Depot:

Personal Information Disclosure: The Home Depot will not trade, rent or sell your personal information, without your prior consent, except as otherwise set out herein. [Does not describe sharing with third-parties for advertising or analytics.]

The Wall Street Journal:

We will not sell, rent, or share your Personal Information with these third parties for such parties’ own marketing purposes, unless you choose in advance to have your Personal Information shared for this purpose. Information about your activities on our Online Services and other non-personally identifiable information about you may be used to limit the online ads you encounter to those we believe are consistent with your interests. Third-party advertising networks and advertisers may also use cookies and similar technologies to collect and track non-personally identifiable information such as demographic information, aggregated information, and Internet activity to assist them in delivering advertising on our Online Services that is more relevant to your interests.

Metacafe:

Metacafe’s Privacy Policy is to share personal information only with the owner’s informed consent.

Likewise, a number of third-party trackers disclaim collection of personally identifiable information.⁸

Scorecard Research (comScore):

Does your beacon collect or store any personally identifiable information about me?

The tagging used by ScorecardResearch is unable to identify the user visiting a page.

Quantcast:

We do not tie the information gathered by Quantcast Tags to the personally identifiable information of visitors to a Web site.

. . .

We do not link Log Data to any other Personally Identifiable Information about you or otherwise attempt to discover your identity.

Google Advertising:

We don’t collect or serve ads based on personally identifying information without your permission.

The better practice for all first-party and third-party websites would be to acknowledge that identifying information leakage is a fact of life on the web, and that identifying information may be shared with third parties.

As for policy, some strands of the Do Not Track debate echo a sentiment of “it’s all anonymous,” and so, “where’s the harm?” We believe there is now overwhelming evidence that third-party web tracking is not anonymous. It is a legitimate policy question whether, on balance, Do Not Track should be enforced by law. But the difficult weighing of competing privacy risks and economics can’t be short-circuited by claims of anonymity.

Thanks to Arvind Narayanan for comments on a draft.

[1] For purposes of this post, “identifying information” is information that with moderate probability and moderate effort can be used to identify a user. This post does not use a formulaic legal definition of “personally identifiable information” (PII), an approach that has been discredited by a growing body of computer science research. The Federal Trade Commission staff notably rejected the notion of PII in its draft privacy report last year.

[2] Some third parties encode the referring URL into their Request-URI.

[3] A username isn’t, of course, all a third party has to go on. IP geolocation is another trivial source of information, and can help disambiguate when several individuals use similar usernames. How many Jonathan Mayers are there in Palo Alto, CA? Using the Stanford University network? This is a possible area for future research.

[4] While it is quite clear that in practice a username can often be used to discern a user’s identity, confirmatory empirical research would be valuable.

[5] We used a fictional persona with unique biographical traits to minimize false positives.

[6] For readers who engage in detail with our data, we wish to emphasize several caveats to our methodology.

• We did not study – and cannot study – what companies do when they receive personal information. It is likely that many of the information leaks we identified were logged. Some third parties may take precautions to prevent logging of identifying information, and we certainly laud such efforts. But for policy purposes, there is a tremendous difference between a tracking ecosystem that is anonymous and a tracking ecosystem that is suffused with identity but promises to ignore it.
• Since some websites host content from multiple PS+1s (e.g. amazon.com and amazonaws.com), our definition of a third party introduces some false positives. That said, our findings appear to be quite robust. For example, thresholding for leakage at more than three third parties still leaves 84 websites (45%) leaking a username or user ID.
• We did not examine POST request bodies or cookies, nor did we attempt to identify obfuscated or encrypted personal information.
• Our interaction with websites was neither comprehensive nor representative of what the average user might do. We may have missed information leaks, and some of the information leaks we identified may have affected only a minority of users.
• In the course of a user’s browsing, identifying information for other users might leak. We did not gauge how easily a third party could identify which information was the user’s. In most cases it appeared such a determination would be straightforward.
• The regular expressions we used for matching birth year, birthday, gender, and last name had a not insignificant number of false positives. We recommend against relying solely upon those fields.
• We did not explicitly take note of which stage of signup a leak occurred at.
• We did not use a single sign-on (SSO) provider unless required. Where an SSO was mandatory, we manually labeled PS+1s associated with the SSO provider as first-party. Measuring information leakage when SSOs are used is a promising avenue for future research.
• We did not attempt to discover third parties that have been CNAMEd into a first-party PS+1 (dubbed “hidden third-parties” in some papers).

[7] User IDs were, in our testing, almost always sufficient to locate at least a username, and sometimes additional identifying information. For example, with a Causes.com user ID, anyone can attain a link to a user’s Facebook profile – which in turn provides a name, photo, and possibly more.

[8] Please note: we are not claiming any company has breached its self-regulatory commitments. The Digital Advertising Alliance (DAA) online advertising self-regulation imposes lax restrictions on personally identifiable information. First, personally identifiable information is defined to only include information that is used to identify a user.

Personally Identifiable Information is information about a specific individual including name, address, telephone number, and email address—when used to identify a particular individual.

Second, the DAA principles only require noting the use of PII in a privacy policy and getting consent to retroactively use PII before the privacy policy change.

PII is a term used primarily in two areas in the Principles and Commentary. First, PII is used in the Transparency principle so that consumers are informed specifi- cally about the collection and use of PII for Online Behavioral Advertising purposes. Second, PII is used in this Commentary to describe a specific example of a “material” change that would require Consent from the consumer under Principle V.

A number of technologies have been touted to offer consumers control over third-party web tracking. This post reviews the tools that are available and presents empirical evidence on their effectiveness. Here are the key takeaways:

1. Most desktop browsers currently do not support effective self-help tools. Mobile users are almost completely out of luck.
2. Self-help tools vary substantially in performance.
3. The most effective self-help tools block third-party advertising.

Following the usage model in the FTC staff’s 2010 preliminary online privacy report, this post is oriented towards the user who wants a simple, persistent, comprehensive solution such that with high confidence no third party collects her browsing history. We assume that some third-party trackers will use non-cookie tracking methods including supercookies and fingerprinting (e.g. Microsoft, KISSmetrics, Epic Marketplace, BlueCava, Interclick, Quantcast).

Thanks to Jovanni Hernandez and Akshay Jagadeesh for assisting with data collection, and to Arvind Narayanan and Peter Eckersley for input on drafts.

Self-Regulatory “Opt-Out” Cookies

For over a decade a minority of third-party trackers, most prominently the members of the self-regulatory Network Advertising Initiative (NAI) and Digital Advertising Alliance (DAA), have offered users the ability to set an “opt-out” cookie.

As a technological matter, cookies are a poor mechanism for storing persistent and comprehensive user preferences. Users often delete their cookies, wiping out “opt outs.” Cookies can expire (e.g. Chitika’s “opt-out” cookie). And because each “opt-out” cookie is scoped to specific domain, a user has to periodically install cookies for companies that have newly begun to offer an “opt out.” The user experience for setting “opt-out” cookies is unnecessarily arduous (see the DAA “opt-out” page), and in some cases “opt-out” cookies aren’t even set correctly. That said, there are several browser extensions that significantly improve the usability and persistence of “opt-out” cookies (e.g. Abine Taco, Beef Taco, Google Keep My Opt Outs, NAI Consumer Opt Out Protector, and PrivacyChoice Keep More Opt Outs).

You may have noticed that “opt out” is scrupulously placed in quotes throughout this discussion. That’s because, setting aside technical issues, “opt-out” cookies don’t actually opt users out of tracking. As we explained in an earlier post, “opt-out” cookies only opt users out of seeing ads based on tracking—not tracking itself. And as we showed in a later post on the DAA’s self-regulatory icon initiative, both the NAI and DAA use slippery, deceptive language in describing their “opt-out” programs.¹

Do Not Track

Do Not Track uses an HTTP header to signal a user’s preference to opt out of third-party tracking. Browsers have been quick to adopt the proposal, user adoption is skyrocketing (1, 2), and tools are under development for detecting violations. But, for the moment, most tracking companies steadfastly refuse to comply. We believe Do Not Track is the right way to provide consumer choice on third-party tracking (learn more at DoNotTrack.Us), and we recommend users enable the feature to send a signal to regulators, legislators, and tracking companies. While we are pleased with Do Not Track’s progress, convincing stakeholders to adopt the proposal is a lengthy process. In the interim, users must look elsewhere for effective protection against third-party tracking.

Browser Profile Clearing

Users are often advised to regularly clear their cookies, cache, history, and other browser profile settings to prevent third-party tracking. There are several reasons this approach does not adequately protect users.

First, many third-party tracking methods continue to work. Tracking techniques that do not require storing state in the browser are wholly unaffected. As for stateful tracking, the user must play Whac-A-Mole with third parties. To remove ETag cookies, the user must clear the browser’s cache. To remove Flash cookies, she has to independently clear her Flash plugin data. In short: the user has to scrub anyplace the browser or a plugin can store state.

Second, clearing the browser profile only provides periodic protection. In the intervals between when a user clears his settings, every tracking method works.

Third, clearing the browser profile undermines beneficial functionality. Many of the lost features result in significant annoyances (e.g. stored logins and browsing history). Some even introduce security vulnerabilities (e.g. stored authentication tokens and HTTP Strict Transport Security).

Last, as a practical matter, clearing the browser profile is an unworkable solution. The average user cannot, and should not, reasonably be expected to diligently vacuum her browser on a monthly basis—let alone every week or every day.²

Private Browsing Mode

While implementation specifics vary by browser, private browsing modes share a common goal: eliminate evidence of browsing that resides on the computer. To a first approximation private browsing modes function the same as clearing the browser profile, except the user proactively declares a session to be private (automatically clearing profile changes when the session ends) instead of retroactively clearing the profile. Private browsing has the very same shortcomings as clearing the browser profile: it does not stop all tracking methods, it provides only periodic protection (the user can be tracked within a private browsing session), beneficial web functionality breaks, and as a practical matter a user will not adjust a setting every time his browser launches.

Third-Party Cookie Blocking

All the major web browsers include an option to prevent third-party domains from setting cookies. Because cookies are just one of many ways third parties track users, third-party cookie blocking provides limited protection. And unless a browser blocks third-party cookies from being read,³ clicking a tracker’s ad or visiting a tracker’s website (e.g. Facebook or Google) once is enough to set an indefinite tracking cookie.⁴

Targeted Cookie Blocking

Internet Explorer, Firefox, Chrome, and several browser extensions offer the ability to prevent cookies from certain domains from being read or set. Just like third-party cookie blocking, this approach does not mitigate non-cookie tracking technologies. It also largely eliminates interactive functionality on websites that are both a first party and a third-party tracker (e.g. Facebook or Google).

Execution Blocking

A number of tools are available for preventing the execution of JavaScript (e.g. NoScript), Flash (e.g. Flashblock), and other script content that could be used for tracking. While there are many other reasons to use these tools (including security, speed, and power consumption), they only mitigate a subset of tracking mechanisms.

Content Blocking

[Updated 9/14 to include a note on Request Policy. Thanks to Joe Hall for the suggestion.]

Because of the myriad methods for tracking, many privacy tools focus on preventing the browser from even requesting certain third-party content. While content blocking can effectively prevent third-party tracking, a content blocking tool is only as effective as its list of rules on what to block (often called a “blocklist”). Most content blocking tools consist of nothing more than a regularly updated blocklist (or family of blocklists), in either Adblock Plus or Tracking Protection List format. Request Policy, a Firefox extension, takes the opposite approach: all requests to third-party domains are blocked, save those the user explicitly allows. While Request Policy offers nearly comprehensive protection from third-party tracking, properly configuring it requires substantially greater patience and expertise than the average user can reasonably be expected to possess.

Please note: Chrome, Safari, Mobile Safari, and the Android browser DO NOT presently support content blocking.⁵ Firefox extensions are able to block content, and users can install blocklists in Internet Explorer 9.

Effectiveness Measurement

We conducted a study of the effectiveness of twelve web privacy tools at mitigating third-party web tracking. Please note: several of the blocklists we studied in Adblock Plus format are also available in the less expressive Tracking Protection List format. The change in formats may impact performance.

• Abine Tracking Protection List

Abine’s Tracking Protection List blocks many online advertising and marketing technologies that can track and profile you as you browse the Web. This list is updated weekly to keep you safer and more private.

In our initial testing, the Abine list performed very poorly; manually inspecting the list we identified several typos. We called our findings to Abine’s attention, and the company responded with an updated list. We present below our findings on both the original and updated Abine lists.

• Adversity Ads + Privacy + Antisocial Adblock Plus lists
• EasyList Adblock Plus list

EasyList is the primary subscription that removes adverts from English webpages, including unwanted frames, images and objects. It is the most popular list for Adblock Plus, with over 7 million daily users, and forms the basis of over a dozen combination and supplementary subscriptions.

• EasyPrivacy Adblock Plus list

EasyPrivacy is an optional supplementary subscription that completely removes all forms of tracking from the internet, including web bugs, tracking scripts and information collectors, thereby protecting your personal data.

• EasyList + EasyPrivacy Adblock Plus lists
• Fanboy’s List Ads + Tracking + Annoyance Adblock Plus lists
• Ghostery browser extension (configured to block all trackers, “experimental” cookie blocking not enabled)

Ghostery allows you to block scripts from companies that you don’t trust, delete local shared objects, and even block images and iframes. Ghostery puts your web privacy back in your hands.

• PrivacyChoice 1 Tracking Protection List
• PrivacyChoice 2 Tracking Protection List

PrivacyChoice maintains a comprehensive database of tracking companies, including domains used by nearly 300 ad networks and platforms, tracking methods, summaries of key policies, oversight, and opt-out and opt-in processes. PrivacyChoice has created Tracking Protection Lists based on this data. You have the option of installing two lists. The first list blocks companies that are not subject to oversight by the NAI and the second list blocks all tracking company domains in the PrivacyChoice database. These lists will be automatically updated with new tracking domains discovered through continuous website scanning and user panels.

• PrivacyChoice TrackerBlock browser extension (configured to block all trackers, opt-out cookies not enabled)

Complete control over online tracking using multiple methods, including cookie blocking, persistent opt-out cookies, Flash and HTML5 control, and Do Not Track signals.

• TRUSTe Tracking Protection List

TRUSTe is the leading online privacy certification and services provider. TRUSTe’s TRUSTed Tracking Protection List enables relevant and targeted ads from companies that demonstrate respectful consumer privacy practices and comply with TRUSTe’s high standards and direct oversight. TRUSTe helps users get good ads, without compromising personal privacy.

Effectiveness Measurement – Methodology

For each blocking tool we conducted a crawl of the Alexa U.S. top five hundred websites using the FourthParty web measurement platform. To ensure broad coverage of third parties we crawled the list three times in series, and to provide fresh browser state for each page load we cycled private browsing mode off and on. We also conducted a baseline crawl for comparison. Our crawl data is available on request.

We compiled three measurements with each blocking tool:

HTTP Requests. The number of crawled pages on which each domain (public suffix + 1) receives at least one HTTP request. Almost all third-party web content is served using HTTP, so there likely few if any false negatives. But this measurement includes false positives: some resources are served from a third party that does not track. For example, the Google Libraries API (googleapis.com) serves static content and instructs the browser to cache it for a year.

HTTP Set-Cookie Responses. The number of crawled pages on which each domain (public suffix + 1) sends at least one HTTP response that includes a Set-Cookie header. This metric has some false negatives since it includes neither trackers that do not set cookies over HTTP nor trackers that set their cookies in a first-party context (e.g. Twitter). There are few false positives since in almost all cases cases if a web service wants to preserve state across multiple sites it will just use a unique identifier.

Cookies Added – Cookies Deleted. The number of cookies added less the number of cookies deleted by each domain (fully qualified domain name). Measuring the difference between cookies added and deleted neglects trackers that do not use cookies or set cookies only as a first party, and is overinclusive of first-party sites that set a large number of cookies. Scripts can behave erratically when a browser blocks content, introducing significant noise into this measurement. We include it as a rough benchmark for cookie blocking tools.

Effectiveness Measurement – Results

The following graph reports the average across all tracking domains of the relative difference in each measurement.⁶ We encourage interested users to examine the complete spreadsheet of measurements.

Some observations from inspecting the tools and analyzing the crawl data:

• Self-help tools vary significantly in their effectiveness. Some (especially the Tracking Protection Lists from Abine before 9/6 and from TRUSTe) offer very little protection.⁷ No tool is comprehensive.
• Installing multiple self-help tools can decrease user privacy. Despite receiving widespread negative press for the practice, TRUSTe continues to serve a Tracking Protection List that overrides other lists to allow tracking by BlueKai, comScore, Scorecard Research, and others.
• Some websites depend on the presence of certain third-party scripts (e.g. the Google Analytics ga.js or urchin.js). Ghostery cleverly circumvents this issue by replacing several popular scripts with dummy stand-ins. (See also NoScript surrogate scripts.) It may be worthwhile to add support for dummy scripts to blocklist formats.
• The top performers (EasyList + EasyPrivacy and Fanboy’s List Ads + Tracking + Annoyance) are community-maintained blocklists.
• Both top performers require installing more than one blocklist.
• All the top and near-top performers (EasyList + EasyPrivacy, Fanboy’s List Ads + Tracking + Annoyance, Ghostery, and Adversity Ads + Privacy + Antisocial) block third-party advertising. This result should come as little surprise: third-party tracking is often inextricably commingled with third-party advertising.
• Most self-help tools do a poor job of blocking social plugins, even from the most popular social networks and sharing platforms.

Policy Implications

In the debates surrounding online privacy, many tracking companies have assumed that if they can hold out against Do Not Track, their business practices will continue. That’s not necessarily the case. Some users will turn to the next-best alternative, and we now know what that is: ad blocking. Internet Explorer 9 already supports ad blocking with two clicks. Representatives from Mozilla have repeatedly delivered the ultimatum that if effective regulation or self-regulation does not occur, Firefox will provide users with self-help tools. W3C is working to standardize a blocklist format. The extent to which users adopt ad blocking will, of course, depend on usability, advocacy, and much more. But it likely won’t take much persuading: users dislike advertising, and ad blockers are already the most popular extensions for Firefox, Chrome, and Safari. Third parties should not be so hasty to play Russian roulette with the Internet economy. And publishers should not be so willing to let them.

[1] Sometimes even the NAI and DAA member companies misunderstand what the self-regulatory programs require. Here are two examples from Google’s Keep My Opt Outs tool (1, 2):

Today we’re making available Keep My Opt-Outs, which enables you to opt out permanently from ad tracking cookies.

Will this persistently opt me out of every cookie on the web?

No, this will not opt you out of cookies that are not related to personalized online ads.

[2] Some browsers offer options for clearing components of the browser profile on exit. These options may somewhat mitigate usability issues with regularly cleaning the profile.

[3] Third-party cookie blocking in Internet Explorer, Chrome, and Safari only prevents cookies from being set, not read. Chrome does provide a separate “experimental” option in about:flags that prevents third parties from reading cookies. Firefox’s third-party cookie blocking prevents both setting and reading cookies.

[4] There may also be trivial ways to circumvent third-party cookie blocking. In Safari, for example, a redirect through a domain or a POST to a domain will allow setting cookies.

[5] The blocking API in WebKit (used in Chrome and Safari extensions) has a number of shortcomings. First, it doesn’t prevent network requests, just loading content into the DOM. Second, the API doesn’t allow blocking for all HTTP requests. Last, to support even modestly comprehensive blocking without a significant performance impact, it requires a synchronous message passing feature that Chrome lacks. A more comprehensive blocking API for Chrome is currently under development with no set release date. A previous effort towards a Chrome blocklist feature was cancelled after six months of development.

Android users can block third-party web (but not app) content by running Firefox with Adblock Plus.

[6] We treated a domain as a third-party tracking domain if its metric value was greater than six in the baseline crawl. In other words, we considered a domain to be a third-party tracker if it, to a first approximation, consistently appeared on more than two sites. We found our results quite robust against changing the threshold value for considering a domain a third-party tracker.

To conserve space, the graph above does not show values below zero.

[7] We did not conduct a crawl with the Enhanced Privacy Tracking Protection List, though a cursory inspection of the list revealed that it blocks very few third-party trackers.

Despite all the attention they’ve received in the debates around online privacy, cookies are far from the only way to track a user. Broadly speaking, a website can either stash a unique identifier anyplace in the browser (“tagging”)¹ or explore features of the browser until it becomes unique (“fingerprinting”).² Tracking technologies that do not rely on cookies are often referred to as “supercookies,” and they are widely viewed as unsavory in the computer security community because they continue tracking even when a user clears her cookies to preserve privacy. Sometimes a site will use a supercookie to “respawn” its original identifier cookie, creating a “zombie cookie” — the basis of several lawsuits.

In one of our recent FourthParty web measurement crawls we included a cookie clearing step to emulate a user’s privacy choice. We observed that after clearing the browser’s cookies an identifier cookie (named “MUID” for “machine unique identifier”) respawned on live.com, a Microsoft domain. We dug into Microsoft’s cross-domain cookie syncing code and discovered two independent supercookie mechanisms, one of which was respawning cookies. We contacted Microsoft with our observations, and we have collaborated to assist in rectifying the issues we uncovered. Here is what we know.

Thanks, once again, to Jovanni Hernandez and Akshay Jagadeesh for their indispensable research assistance.

Microsoft’s cookie syncing script would, in some cases, function as a cache cookie and respawn the MUID cookie.

One of the foundational concepts in web security is the cookie same-origin policy: cookies can only be read and modified by the domain that set them. If domains collaborate they can trivially circumvent the same-origin policy and share cookies with each other; this practice is called “cookie syncing.” Cookie syncing often raises privacy concerns. For example, in online advertising real-time bidding, cookie syncing allows a single advertising exchange to notify many advertising networks and data aggregators whenever a user visits a website. That said, there are some unequivocally legitimate use cases for cookie syncing, such as when a company has spread its business across multiple domains (e.g. amazon.com and amazonaws.com).

Microsoft uses cookie syncing to share identifiers across many of its web properties, including bing.com, microsoft.com, msn.com, live.com, and xbox.com. Microsoft also syncs its MUID cookie to atdmt.com, the domain for its Atlas third-party advertising network. We found that one of Microsoft’s cookie syncing scripts (wlHelper.js) included an instruction to set the MUID cookie, and the script would get cached indefinitely.³ If the cached script ran and no MUID cookie was present, the script would set a cookie with its stored MUID. Here is a slightly simplified example snippet of the relevant code:⁴

var id_muid = “5CBC2F2396F14F4EBA255A695D313CD1“;

var muidValue = null;

// the MUID cookie value is read into muidValue

…

if (muidValue == null && id_muid != null) {

   …

   // cookieDomain is set to “; domain=” + the current domain

   var cookieSettings = cookieDomain + “; expires=Fri, 01 Jan 2021 00:00:00 GMT; path=/;”;

   document.cookie = “MUID=” + id_muid + cookieSettings;

}

We identified wlHelper.js scripts on several Microsoft domains:

http://analytics.atdmt.com/Scripts/wlHelper.js?i=MUID

http://analytics.live.com/Scripts/wlHelper.jsi=MUID

http://analytics.microsoft.com/Scripts/wlHelper.js?i=MUID

http://analytics.msn.com/Scripts/wlHelper.js?i=MUID

In our crawling data from the Alexa world top 10,000 sites we found that one or more wlHelper.js scripts were loaded when the browser visited:

http://www.microsoft.com/en-us/default.aspx

http://www.microsoftstore.com/store/msstore/DisplayHomePage

http://www.msn.com/

http://ca.msn.com/

http://es.msn.com/

A user would have her MUID respawned if she: 1) ever visited a site with a wlHelper.js embedded, 2) cleared her cookies but not her cache, and 3) visited a site with the same wlHelper.js embedded and no MUID. It is difficult to estimate the number of users affected by Microsoft’s respawning without knowing more about traffic to Microsoft’s web properties and the conditions under which it would set an MUID. We would note that Microsoft’s web properties are popular destinations with tens of millions of visitors per day.

Once a cookie respawned, we often saw it get sent to other Microsoft domains. In the FourthParty data above, for example, the old MUID was passed to atdmt.com.

http://c.atdmt.com/c.gif?…&MXFR=5CBC2F2396F14F4EBA255A695D313CD1

Microsoft therefore had, in at least this case, sufficient information to trivially associate the user’s interactions with msn.com, live.com, and atdmt.com from before and after cookie clearing.⁶

Microsoft’s cookie syncing script included an ETag cookie.

ETags are a simple cache control mechanism built into HTTP. A website can assign a version number to a resource; when the browser goes to request the resource, and the version hasn’t changed, the website can just tell the browser to use its cached copy. It had long been known that, instead of storing a version number, an ETag could be used to store a user identifier (an “ETag cookie”). Two weeks ago a research team at University of Caliornia, Berkeley discovered the first instance of ETag cookies in use.

We found that, in addition to functioning as a cache cookie, Microsoft’s wlHelper.js script was associated with an ETag cookie containing the MUID.

sqlite> select name, value from cookies where host=’.live.com’ and name=’MUID’ limit 1;

MUID 5CBC2F2396F14F4EBA255A695D313CD1

sqlite> select http_response_headers.name, http_response_headers.value from http_responses, http_response_headers where http_responses.id = http_response_headers.http_response_id and http_responses.url=’http://analytics.atdmt.com/Scripts/wlHelper.js?i=MUID’ and http_response_headers.name=’Etag’ limit 1;

Etag “5CBC2F2396F14F4EBA255A695D313CD13698″

The practical effect was that if a user cleared her cookies but not her cache, subsequent requests for wlHelper.js would be accompanied by both the new MUID value (in a cookie) and the old MUID value (in the ETag). This pairing of old and new MUIDs gave Microsoft sufficient information to associate user interactions with its domains from before and after cookie clearing.⁷

In addition to supercookies, we spotted two other issues with Microsoft’s advertising practices.

Microsoft’s targeted advertising opt-out button was invisible in Chrome and Safari.

Microsoft operates its own advertising choice page. (Note that Microsoft only allows users to opt out of seeing behaviorally targeted advertising; it does not remove its identifier cookies after a user has opted out, nor does it make any promise to stop tracking.) We observed that the opt-out link on Microsoft’s advertising choice page was invisible for Chrome and Safari users.

Microsoft fixed their opt-out button after we called the issue to their attention.

Microsoft’s approach to segregating advertising data does not meaningfully protect user privacy.

In a 2007 report entitled “Privacy Protections in Microsoft’s Ad Serving System and the Process of ‘De-identification,’” Microsoft’s privacy team explains how the company segregates identified Windows Live user information from de-identified third-party advertising data.

One of Microsoft’s goals is to serve targeted ads in a manner that protects user privacy. To avoid using the LiveID cookie to serve per-user ads—because, as described earlier, it is directly associated with information that could personally identify the user—Microsoft has created an “Anonymous” ID, called the ANID, on which its ad serving capabilities are based.

When a user first registers with Windows Live or MSN, a LiveID and an ANID are created simultaneously. The ANID is derived by applying a one-way cryptographic hash function to the LiveID. A one-way cryptographic hash function ensures that there is no practical way of deriving the original value from the resulting hash value—that is, the process cannot be reversed to obtain the original number.

Microsoft makes several expansive claims about its advertising system’s privacy guarantees.

Because all personally and directly identifying information about a user is stored on servers in association with a LiveID rather than an ANID, there is no practical way to link data stored in association with an ANID back to any data on Microsoft servers that could personally and directly identify an individual user.

Furthermore, to associate any of the ANID-based data in the Microsoft ad system with an individual user, an internal or external attacker would not only need access to the ad serving system (to access the data), the Windows Live ID system (to access all LiveIDs ever issued) and the hashing algorithm but would also need a massive computing infrastructure to run the algorithm on each and every LiveID ever created to try to find the ANID in question.

And in 2008 testimony before the Senate Commerce Committee, attached to a 2009 comment to the Federal Trade Commission:

As a result of this “deidentification” process, search query data and data about Web surfing behavior used for ad targeting is associated with an anonymized identifier rather than an account identifier that could be used to personally and directly identify a consumer.

Microsoft’s attempt at “anonymous” advertising data does not achieve nearly so much. First, as Arvind Narayanan noted in a recent blog post, de-identified online tracking data is far from anonymous. Even using completely unassociated identifiers for Windows Live user information and advertising data would not provide much of a privacy guarantee.⁸

Second, Microsoft’s use of a cryptographic hash to generate its ANID cookie contributes little privacy protection. The privacy threat that Microsoft is attempting to mitigate is a comparison between a user’s ANID and LiveID. Cryptographic hashing does not make comparison of two known values a challenge: on the contrary, comparison is a core use case for cryptographic hashing. With knowledge of Microsoft’s hash function, an employee or outsider could trivially compare any LiveID to any ANID.

Closing Thoughts

The online advertising industry is currently locking horns in Washington to prove it can regulate itself. Several trade organizations and private firms have billed themselves as rigorous watchdogs. And yet, in our analysis of one of the most prolific online advertising networks, we found significant privacy shortcomings that even a cursory privacy audit would have uncovered. It is increasingly difficult to accept industry claims that recent negative discoveries reflect “just a few bad apples.” And it is more than a little troubling that a few research groups and occasional press coverage seem to be the only present checks on one of the most privacy-invasive industries in history.

Thanks to Ashkan Soltani for providing feedback on a draft.

[1] See Evercookie, Flash Cookies and Privacy II, and An Analysis of Private Browsing Modes in Modern Browsers.

[2] See How Unique Is Your Web Browser? and Any person… a pamphleteer.

[3] The wlHelper.js script was served with a two-day cache expiry. Subsequent requests after the cache expired received a 304 response to keep the cached version with another two-day expiry.

[4] Microsoft replaced its cookie syncing system, including wlHelper.js, after we alerted them to our findings. An example of the old script is available on Google Code.

[5] We also saw the microsoft.com MUID cookie respawn, but not through wlHelper.js. We are still working to discover the additional supercookie mechanism on microsoft.com.

[6] Web measurement provides limited insight into a website’s backend. In cases where a domain’s cookie respawned, it is quite likely that new tracking information was associated with old tracking information. We cannot say how Microsoft used its data in cases where a domain’s MUID didn’t respawn but it received an old MUID from another domain that did respawn. At minimum it seems unlikely Microsoft discarded this information from all logs.

[7] As above, we cannot say what Microsoft did with its ETag cookie data. It again would be unlikely Microsoft discarded this information from all logs.

[8] Google follows this approach: it serves its third-party advertising content from doubleclick.net, and uses a Doubleclick identifier, instead of serving from google.com and using a Google identifier.

Jovanni Hernandez and Akshay Jagadeesh are the first authors of this study.

Responding to pressure from the Federal Trade Commission, in mid-2009 the largest advertising industry trade groups joined forces to develop a new self-regulatory program for behavioral advertising: the Digital Advertising Alliance (DAA). Like the parallel self-regulatory program for advertising networks, the Network Advertising Initiative (NAI), the DAA makes no promises about providing privacy choices: DAA members must only provide an opt out of seeing advertising that is based on tracking, not an opt out of tracking itself.¹ As Chris Hoofnagle at Berkeley Law has noted on several occasions, the word “privacy” scarcely even appears in the DAA’s documents.

The centerpiece of the DAA self-regulatory program is “enhanced notice,” a common text and icon for linking a consumer to information about behavioral advertising and an opt out of behavioral targeting (again, not tracking). The initial proposal for “enhanced notice” was a large insert for third-party behavioral ads consisting of a “Power I” icon alongside descriptive phrases such as “Interest based ads” and “Why did I get this ad?”

In its final consensus form, “enhanced notice” consists of a small “Forward I” icon (aka the “Advertising Option Icon”) in or around a third-party behaviorally targeted ad and, in some cases, the text “AdChoices.”

“Enhanced notice” can also appear in the footer of a page.

Usability of an advertising choice mechanism, like any software feature, will be a key driver of adoption. We note that at no point did the DAA conduct testing to determine how “enhanced notice” compares in usability to alternative choice mechanism designs, such as an in-browser option.² In fact, web browser Do Not Track features — which aren’t yet fully backed by industry or regulation — are already seeing significantly greater usage than the DAA’s “enhanced notice” (1, 2).

In recent press materials and congressional testimony members of the online advertising industry have touted that trillions of “enhanced notice” icons are being served. We conducted this study to empirically examine the DAA’s “enhanced notice” icons.³ For simplicity, and following convention in the privacy community, we refer to any iteration of the DAA’s icon-based program as “AdChoices.”

Methodology

We began with the Alexa list of top 500 U.S. websites as of August 4th, 2011. For each site on the list we manually inspected⁴ the homepage for third-party advertisements. We labeled content as a third-party ad if it appeared in a standard ad box size and was served by a third-party advertising network. For each third-party ad, we noted whether an AdChoices icon or text link was present. We also manually examined page footers for AdChoices icons and text. Please email if you would like to examine the screenshots from our study.⁵

Results

We identified 627 third-party ads. Only 62 (9.9%) included an AdChoices icon in or around the ad. And only 32 (5.1%) had an “AdChoices” text link. We found an AdChoices icon and text link in the footer of only 13 (2.6%) of the pages we examined.

Restricting our dataset to the 449 non-explicit, domestic websites in the Alexa U.S. top 500, we spotted 512 third-party ads. 58 (11.3%) had an AdChoices icon. 28 (5.5%) had an “AdChoices” text link. We identified the AdChoices icon and text in the footer of 13 pages (2.9%).

A full spreadsheet of results is available in Excel format.

Concluding Thoughts

We, and many other researchers, already had grave doubts about the AdChoices program’s efficacy. The icon is only approximately 13×13 pixels⁶ and nondescript. The accompanying text is in small font and reads, ambiguously, “AdChoices.” Now we learn that the icon rarely shows up and, half the time, doesn’t even include any text.

Beyond demonstrating shortcomings in the DAA’s AdChoices program, our findings also run contrary to two common claims from members of the online advertising industry: that the vast majority of third-party ads are behaviorally targeted⁷ and that the largest players in behavioral targeting have embraced the AdChoices icon (1, 2). Given our results, both claims cannot be true. We call upon the online advertising industry to share its statistics on what proportion of third-party ads are behaviorally targeted and what proportion of third-party ads bear an AdChoices icon.

[1] The NAI and DAA use slightly different language to describe their opt-out commitments. An attorney for Venable, DAA’s counsel, confirmed that there is no practical effect to the difference at a recent symposium at Yale Law School. The NAI is now a participant in the DAA consortium.

NAI: “Opt out of OBA [online behavioral advertising] means that a consumer is provided an opportunity to exercise a choice to disallow OBA . . . . [C]ollection of non-PII data regarding that consumer’s browser may only continue for non-OBA purposes . . . .” (emphasis added).

DAA: “A Third Party should provide consumers with the ability to exercise choice with respect to the collection and use of data for Online Behavioral Advertising purposes . . . .” (emphasis added).

[2] Research by McDonald and Cranor found that the NAI’s choice mechanism, which is built around an information page and an opt-out page much like the DAA’s, created substantial consumer confusion. A paper by the Future of Privacy Forum, an industry-funded advocacy group, found that the icon approach fell flat in conveying information about behavioral targeting (“substantial repetition and consumer education may be needed to improve [the icon's] communication effectiveness over time”) and that the “Power I” icon and “AdChoice” text performed worse than an alternative icon and several alternative texts.

[3] This study is not an examination of legal compliance with the DAA self-regulatory program. The DAA’s formal third-party “enhanced notice” requirement is very lax, and can be satisfied by as little as a link to the DAA in the website’s privacy policy. For a study examining compliance with the DAA’s self-regulatory program, see “AdChoices? Compliance with Online Behavioral Advertising Notice and Choice Requirements” by Komanduri et al.

[4] As with any study that relies on manual labeling, there is a possibility we made labeling errors. At least two different researchers examined each page to minimize the possibility of mislabeling. Moreover, the proportions we are reporting are very robust against individual labeling errors owing to the size of the dataset.

[5] Because a number of the Alexa top websites contain explicit content, we are not able to publicly post our screenshot dataset.

[6] Compare to the smallest standard display ad size at 88×31 pixels.

[7] Industry metrics imply only a very small proportion of third-party advertising is behaviorally targeted. See “Do Not Track Is No Threat to Ad-Supported Businesses.”

Last week marked the twentieth anniversary of the public World Wide Web, and there is much to celebrate. The early web consisted of a few text pages linked together; the modern web supports audio, video, interactivity, complex storage, and even native applications. Both Microsoft and Google are now developing entire operating systems around web technologies.

Tools for measuring the web have not kept pace. Many studies still rely on HTTP header logging and static analysis of HTML, CSS, and JavaScript. Researchers who want to go beyond these simple tools are often forced to develop purpose-built software from scratch.

Today we’re releasing FourthParty, an open-source platform for web measurement. FourthParty is built on Mozilla Firefox and the Add-on SDK, making it fast, modular, easy to use, multi-platform, and up-to-date with the latest web technologies. And FourthParty is already generating research results: it’s the tool we’ve been using in our Tracking the Trackers studies (1, 2). To learn more and get started, visit fourthparty.info.

Last week we reported some early results from the Stanford Security Lab‘s new web measurement platform on how advertising networks respond to opt outs and Do Not Track. This week we’re back with a new discovery in the online advertising ecosystem: Epic Marketplace,¹ a member of the self-regulatory Network Advertising Initiative (NAI), is history stealing.

Many thanks once again to research assistants Akshay Jagadeesh and Jovanni Hernandez.

Background

A link can be styled differently based on whether you’ve been to the page it points to. You may recall, for example, that in the early days of the web links you hadn’t visited were blue and links you had visited were purple. History stealing is a practice that exploits link styling to learn a user’s web browsing history. The approach is simple: to test whether the user has visited a link, add it to a page and check how it’s styled.²

Members of the computer security community have long considered history stealing a serious privacy vulnerability. The risk goes beyond leaking individual tidbits about past browsing; history stealing can be used to track or even identify a user. Mozilla finally implemented a fix in Firefox 4, and the other major browser vendors quickly followed. According to browser usage statistics roughly half of users remain vulnerable to history stealing.

About a year ago researchers at UCSD conducted the first comprehensive study of history stealing in practice. They found that a few popular adult sites were history stealing to learn whether users had visited their competitors. The UCSD team also discovered history stealing by several advertising networks, including Interclick (another NAI member). Class action litigation is ongoing.

Technical Findings – History Stealing

While testing the JavaScript instrumentation in our new web measurement platform we stumbled across Epic Marketplace history stealing on Flixster and Charter.net. We reverse engineered the Epic Marketplace history stealing script and found a number of features:

• The script is fast. Thousands of links are tested per second.
• Links are added in an invisible iframe; there is no apparent effect on the page layout.
• The script dynamically loads lists of URLs and associated interest segments using JSONP.
• Progress is stored in a cookie so the script can resume where it left off.
• The script sets a cookie indicating when it was last run; it will not history steal more than once every twenty-four hours.
• If history stealing is still in progress when the window is closed (e.g. the user navigates to another page) the script sends its findings before ending execution.
• The script slows down if a URL list takes over two seconds to process.
• To prevent multiple history stealing attempts in parallel, the script uses a mutex cookie.
• The script does not directly report the URLs that it detects the user has visited; it sends a deduplicated list of the interest segments associated with the visited URLs.

(For the technically inclined reader, here are an example iframe, script, and URL list.)

We also examined a series of URL lists (spreadsheet) that contain 15,511 entries. The URLs and interest segments range greatly. Some URLs are for a landing page; others are for a specific page. Some interest segments are broad; others are fine-grained. A few example segments:

• Segment 758: discount sites including Groupon and eBay Daily Deals
• Segment 876: sites about coffee, including Dunkin’ Donuts, Folgers, and Starbucks
• Segments 984-989: home improvement sites including Home Depot and Grainger
• Segment 2701: pages about the Ford Fiesta

Several interest segments are highly sensitive:

• Segment 760: pages about getting pregnant and fertility, including at the Mayo Clinic
• Segment 2640: pages about menopause, including at the NIH and the University of Maryland
• Segment 2014: pages about repairing bad credit, including at the FTC
• Segment 2265: pages about debt relief, including at the FTC and the IRS

Technical Findings – Opt Out

We applied the methodology from last week’s study to examine Epic Marketplace’s opt-out practices. (Epic Marketplace was one of the eleven NAI members not included in that study.) We found that Epic Marketplace leaves its tracking cookies in place after both opting out with the NAI mechanism and enabling Do Not Track. We also found that history stealing continues after using either choice mechanism.

Privacy Representations

The 2008 NAI Code of Conduct requires member companies to receive express consent from a user before collecting “Sensitive Consumer Information,” defined as:

• Social Security Numbers or other Government-issued identifiers
• Insurance plan numbers
• Financial account numbers
• Information that describes the precise real-time geographic

  location of an individual derived through location-based services

  such as through GPS-enabled devices

• Precise information about past, present, or potential future health

  or medical conditions or treatments, including genetic, genomic,

  and family medical history

(The Code of Conduct includes the unhelpful footnote, “[t]his provision is to be further developed in a distinct implementation guideline.”)

The Epic Marketplace privacy policy contains the following paragraph under the headings “Information We Collect” and “Non-Personally Identifiable Information”:

Epic Marketplace also automatically receives and records anonymous information that your browser sends whenever you visit a website which is part of the Epic Marketplace Network. We use log files to collect Internet protocol (IP) addresses, browser type, Internet service provider (ISP), referring/exit pages, platform type, date/time stamp, one or more cookies that may uniquely identify your browser, and responses by a web surfer to an advertisement delivered by us. This information may be stored on our systems for about one year.

The privacy policy also claims that:

Web surfers may elect not to provide non-personally identifiable information by following the cookie opt-out procedures set forth below.

As with our prior work, we leave it to the reader to assess whether Epic Marketplace is complying with its privacy representations.

Thanks to Gordon Franken for reviewing this post.

1. Epic Marketplace was, until recently, named Traffic Marketplace. It hosts its third-party content on trafficmp.com.

2. Other forms of history stealing, beyond the scope of this post, rely on page layout, background images, and user interaction.

Over the past several months researchers at the Stanford Security Lab have been developing a platform for measuring dynamic web content. One of our chief applications is a system for automated enforcement of Do Not Track by detecting the myriad forms of third-party tracking, including cookies, HTML5 storage, fingerprinting, and much more. While the software isn’t quite polished enough for public release, we’re eager to share some unexpected early results on the advertising ecosystem. Please bear in mind that these are preliminary findings from experimental software; our primary aims at this stage are developing the platform and validating the approach to third-party tracking detection. Many thanks to Jovanni Hernandez and Akshay Jagadeesh for their invaluable research assistance.

Methodology

We began with a list of advertising companies that participate in the self-regulatory Network Advertising Initiative (NAI). By navigating popular websites we identified a piece of tracking content (primarily ads and beacons) from 64 of the 75 NAI member companies. We performed the following tests on each company’s content:

1) Load the content.

2) Load the content, opt out of the company on the NAI website, and then reload the content.

3) Load the content, enable Do Not Track, and then reload the content.

We manually identified tracking cookies (cookies that appeared to contain a unique identifier or substantially unique information) and how they were altered throughout each test. A spreadsheet of results is available. Please email if you would like a copy of the data we logged while testing a particular company’s content.

1. At least two NAI members are taking overt steps to respect Do Not Track.

Media6Degrees, an advertising data provider, deletes its tracking cookies and sets an opt-out cookie upon receiving a Do Not Track request.

BlueKai, a data provider and management platform, does not set tracking cookies in response to a Do Not Track request, but it does not delete any existing tracking cookies.

2. Over half Half of the NAI members we tested did not remove their tracking cookies after opting out.

NAI member companies pledge only to allow opting out of behavioral ad targeting, not tracking. Of the 64 companies we studied, 33 32 left tracking cookies in place after opting out.

3. At least eight NAI members promise to stop tracking after opting out, but nonetheless leave tracking cookies in place.

We compared our results to a survey of NAI member privacy and opt-out policies recently conducted by Carnegie Mellon’s CyLab. We identified seven companies that (in the study’s reading) promise to stop tracking when a user opts out, but nonetheless leave their tracking cookies in place.

The 24/7 Real Media privacy policy claims that a user may “opt out of receiving our ad delivery, audience management and behavioral targeting cookies.” We found that opting out deleted the company’s tracking cookies, but reloading the content reinstalled the tracking cookies.

Adconion‘s privacy policy states that a user is “free to opt out of the Adconion Cookie.” Opting out deleted one of three tracking cookies but left the other two in place. Reloading the content did not update the remaining tracking cookies.

In its privacy policy, AudienceScience describes its opt-out option as follows: “Should you choose to opt-out, we delete all previously collected information from the cookies, and put new information in the cookie which tells us to stop collecting information from that device.” We found that opting out of AudienceScience removes its unique tracking cookie but does not remove a highly unique cookie that represents the user’s interests. Subsequent loads of the content updated the interest cookie.

[See below for an update from AudienceScience.]

Netmining‘s privacy policy states that upon opting out “we will delete your existing ntmng.com or netmining.com cookie(s) and try to place a new cookie that instructs us not to track your future activities when we detect that cookie.” Opting out deleted the Netmining tracking cookie but did not delete a tracking cookie served from a retailer-specific subdomain of netmng.com (and presumably only used on that retailer’s site). Reloading the content refreshed the retailer-specific cookie.

The Undertone privacy policy notifies users: “If you would like to opt out of OBA, then we offer ‘opt-out cookies’ to block the tracking and placement of future Undertone cookies for OBA purposes on your system for five (5) years.” Opting out removed a highly unique cookie that stores the user’s interests but did not remove a unique cookie. Subsequent loads of the content updated the unique cookie.

Vibrant Media‘s privacy policy provides: “If you’d like to opt-out from having Vibrant Media collect your Non-PII in connection with our Technology, please click here. When you opt out, we will place an opt-out cookie on your computer. The opt-out cookie tells us not to collect your Non-PII to tailor our online advertisement campaigns.” Opting out of Vibrant Media does not remove the network’s unique tracking cookie; the cookie remains in place and is updated with subsequent loads of the content.

The privacy policy on Wall Street on Demand‘s advertising platform claims: “By clicking here, the unique cookie used by this system/domain and stored locally by your browser will be changed to ‘OPT_OUT’. By creating a generic cookie id instead of a unique cookie id – it is even more impossible to track your history.” Opting out deleted Wall Street on Demand’s unique cookie, but left in place a seemingly highly unique cookie that appears to store user interests. Refreshing the content renewed the interests cookie.

We identified one additional company with a privacy policy that may be interpreted to prohibit its current business practices. The TARGUSinfo AdAdvisor opt-out page explains that “[t]he AdAdvisor opt-out works by replacing the existing AdAdvisor cookie with a new cookie that clearly indicates that the user has elected to opt-out of the Services.” Opting out left TARGUSinfo’s unique tracking cookie in place. Refreshing the content did not update the tracking cookie.

4. At least ten NAI members go beyond their privacy policies and remove their tracking cookies.

In comparing our results to the Carnegie Mellon study of privacy policies we found that ten NAI members remove their tracking cookies upon opting out, even though they promise to only stop behavioral targeting of ads. The companies are: BlueKai (retains city-level geolocation), Dapper (bought by Yahoo!), FetchBack, Google, Invite Media, Media6Degrees, Mediaplex, Quantcast, TidalTV, and YuMe.

Concluding Thoughts

These early results scarcely scratch the surface of what we aim to learn with our new web measurement platform. We look forward to sharing new insights in the coming weeks and opening the software in the coming months. If you have experience in the web measurement field and would like to participate in testing the platform, please reach out. And please send web measurement questions — we’re looking for new ways to put the system through its paces!

Updates

[If you would like us to add a statement from your company, please reach out.]

24/7 Real Media has updated its privacy policy.

You may also simply opt out of receiving interest-based advertising by clicking here.

AddThis contacted us about our findings. After a reevaluation, we discovered we had mislabeled a unique session cookie associated with AddThis’s opt-out process as a tracking cookie. The post and spreadsheet have been updated. Our apologies to AddThis for the error.

AudienceScience reached out to clarify its practices. Its cookies store a compressed and encrypted data structure. When a user opts out, AudienceScience removes all interest segments and the unique ID from the data structure, but it continues to update the last time the browser contacted its servers. We have confirmed that AudienceScience now entirely removes its data structure after opting out.

BlueKai confirmed it is taking steps to honor Do Not Track.

Media6Degrees confirmed it is taking steps to honor Do Not Track.

Netmining has updated its privacy policy.

If you select the “opt out” button there for Netmining, we will delete your existing netmng.com or netmining.com online behavioral advertising cookie(s) and try to place a new cookie that instructs us not to track your future activities for the purposes of serving online behavioral advertising when we detect that cookie.

The Network Advertising Initiative has posted a response to the study.

TARGUSinfo submitted the following statement.

Immediately upon the publication of this study, we verified that our Opt-Out was fully functional both through our own www.adadvisor.net/optout.html site as well as through the NAI site. At no time was our opt-out not functioning, meaning that any consumer who had elected to opt out either through us or NAI or aboutads.info was indeed opted out, and no further activity was conducted on that user’s browser. We did identify a minor inconsistency between the opt-out running on our own site and that which was running on the NAI site. Specifically, a second cookie was deleted when the opt-out was set from our own site, but that cookie was left on the browser if the user opted out through the NAI. Despite this cookie remaining on the browser, it was rendered dormant because our opt-out prevents us from reading or accessing any other cookie. We updated the code running on NAI to ensure that this second cookie also gets deleted when a user opts-out through NAI, to ensure that there is no confusion with our actual opt-out functionality and what was stated in our privacy policy.

Undertone has posted a statement responding to the study.

Vibrant Media submitted the following statement.

We drop a user ID cookie when a user initiates engagement with one of our ad units. This collects non-personally identifiable information on keywords a user has engaged with. If the user doesn’t visit a site in our network for 10 days, we delete this data. If someone opts out, we add a do-not-track cookie.

We had been deleting any data associated with the user ID, but had not been deleting the cookie itself (this is acceptable for NAI compliance). When we encounter someone with a do-not-track cookie, we completely ignore the user ID and therefore don’t use their information to serve ads. Although the cookie was remaining, we do not reference or use the ID in any way and we completely delete all data, be it in logs or storage devices for that particular user ID. Going forward, in order to prevent any misunderstanding we will also be deleting that cookie.

We have always been vigilant about adhering to industry best practices and NAI compliance policies.

Wall Street on Demand has updated its privacy policy.

Online Behavioral Advertising (OBA) is the process of targeting specific advertisements to each individual user, based on browsing history. If you opt out of OBA from our service by clicking the link below, the OBA cookie we use to contain this information will be emptied and changed to a placeholder signaling that you have done so. . . . Opting out does not necessarily delete or replace all cookies from our domain; others may remain which are used for aggregate reporting on the performance of the advertisements we serve.

Joint post with Arvind Narayanan.

Earlier today Mozilla announced support for Do Not Fool, a proposed mechanism for opting out of April Fools’ pranks. We cannot support this misguided effort.

First, Do Not Fool would require fundamentally reengineering the Internet, the HTTP protocol, and countless websites. Many of your favorite web destinations like The Onion rely on fooling.

Second, fooling is integral to the American competitive landscape and to innovation. In fact, Do Not Fool would demolish the web’s revenue channels. Don’t just take our word for it—industry-funded, non-peer reviewed, quasi-relevant research proves that fooling accounts for over 99.9% of online revenues.

Third, self-regulation is working. Every time you get fooled today, you have the opportunity to click a tiny icon—on sites that support it—to learn more about how you’ve been fooled. And over fifty major pranksters already allow you to set a cookie to opt out of getting fooled by them, once you figure out who they are. (Though roughly half are just fooling you with that opt out.)

Don’t enable this dangerous new feature. Don’t be fooled by Do Not Fool.

Late last week FTC Commissioner Rosch penned a column in which he repeated a number of hackneyed criticisms of Do Not Track. Senators McCaskill and Pryor articulated similar concerns at a recent hearing. This piece sequentially deconstructs Rosch’s column and replies to each of his substantive critiques.

I also have serious questions about the various do-not-track proposals. In my concurring statement to the preliminary staff report, I said I would support a do-not-track mechanism if it were “technically feasible.” By that I meant that it needed to have a number of attributes that had not yet been demonstrated. That is still true, in my judgment.

Do Not Track raises issues of both technology and policy; it is essential to draw a sharp dividing line between the two. The concerns that Commissioner Rosch expresses relate to business impact. There is now widespread consensus that Do Not Track, implemented as an HTTP header, is “technically feasible.”

First, there are a number of consequences if a consumer adopts a do-not-track mechanism. To begin with, a consumer may sacrifice being served relevant advertising.

Users who enable Do Not Track will still see relevant advertising—just not based on their browsing history on other sites. Contextual advertising and non-tracking forms of interest-targeted advertising are unaffected.

On a related note, there is academic research suggesting that in order to compensate for the loss of the ability to track consumer behavior and the associated ability to serve relevant advertising, advertisers may need to turn to advertising that is more “obtrusive” in order to attract consumers’ attention.

While a number of pundits have claimed Do Not Track will lead to more obtrusive advertising, I am unaware of any academic research on this point. Given layout constraints and the limited consumer tolerance for advertising, it is difficult to believe many sites will add additional advertising for Do Not Track users. And if sites could add more advertising, why wouldn’t they already?

Consumers may also lose the free content they have taken for granted. Not only could consumers potentially lose access to free content on specific websites, I fear that the aggregate effect of widespread adoption by consumers of overly broad do-not-track mechanisms might be the reduction of free content, free applications and innovation across the entire internet economy.

On the contrary, there is substantial reason to believe Do Not Track is no threat to ad-supported businesses. This conclusion is bolstered by the news that thirty online advertising firms are willing to implement Do Not Track.

Beyond that, consumers may forgo the reported ability to earn commissions from “selling” the right to track their behavior or allow the use of their personal information.

Do Not Track allows users to veto third-party tracking; third parties are welcome to respond by offering cash-for-data deals. One of the advantages of Do Not Track is that it creates a market mechanism for negotiating over privacy preferences.

I also wonder whether an overly broad do-not-track mechanism would deprive consumers of some beneficial tracking, such as tracking performed to prevent fraud, to avoid being served the same advertising, or to conduct analytics that foster innovation.

The Do Not Track Internet-Draft accommodates fraud prevention, advertising frequency capping, and aggregate analytics.

Concerns have been raised that do-not-track mechanisms also may have the unintended consequence of blocking tailored content, in addition to advertising.

Do Not Track would not affect first-party personalization (e.g. New York Times recommended reading). It would disallow third-party tailored content, but of course a user could opt back into tracking and personalization by services she trusts.

Second, another issue is potential consumer confusion about the terminology “do not track.” As some have pointed out, there is no consensus on what “tracking” means. In fact, I don’t know precisely what it means.

Like any technical standard, Do Not Track will go through a number of draft iterations. The standards process provides a clear path to achieving a final, consensus definition of Do Not Track.

Some tracking, for purposes unrelated to behavioral advertising, may always occur. When consumers are offered a do-not-track option, they may misunderstand the limited scope of that choice; and, in some instances, calling a mechanism “do not track” could arguably be deceptive.

Just like Do Not Call, Do Not Track will necessarily have certain exceptions that do not completely align with consumer expectations. First, this is no basis for rebuffing the approach in its entirety. Do Not Call remains immensely popular despite not covering political and non-profit entities. Second, browser vendors are already taking steps to improve their privacy user interfaces. Mozilla has retained Aleecia McDonald, a leading scholar on user-friendly privacy, to consult on its Do Not Track implementation.

Third, I am concerned that the recent rush to adopt untested do-not-track mechanisms might be based, in part, on a reluctance to take on the harder task of examining more-nuanced methods of providing consumers with choice. It is always easier to just say “no” with a blunt instrument, rather than to take the time and effort to consider all of the ramifications of the different alternatives.

A common misconception of Do Not Track is that it is a one-size-fits-all choice. This isn’t the case: as I noted in a thought piece for Yale Law’s recent symposium on online advertising, nuanced privacy mechanisms can and should be built on top of Do Not Track.

Finally, the implementation of do-not-track mechanisms must not jeopardize competition by injuring potential competitors. I am concerned that some firms with a monopoly or near-monopoly on a relevant market may use do-not-track mechanisms to cripple competitors from constraining their power.

More specifically, the browser market is heavily concentrated. Most — though not all — firms in the browser market operate for profit and those firms monetize some of their other businesses by advertising. There is nothing wrong with that as such. But we need to know: 1.) whether any of those firms enjoy monopoly or near-monopoly power in any online advertising market; 2.) whether there is any difference between the advertising in which those firms are invested (including the various kinds and combinations) and the advertising portfolio of competitors that may make the latter more vulnerable in the event do-not-track mechanisms are installed; and 3.) whether there is any other way that a firm that dominates the market may be able to disadvantage a rival if do-not-track mechanisms are adopted.

Here’s the concern I believe Commissioner Rosch is attempting to convey: Google, one of the major browser vendors, earns most of its revenue from search advertising and other first-party advertising. If Google were to adopt Do Not Track, it would harm competitors supported by third-party advertising more than it would harm itself. And so Google has an incentive to push Do Not Track in Chrome.

I believe this concern is unfounded. First, as noted above, Do Not Track will not significantly impact advertising revenues for websites. Second, third-party advertising comprises an increasing share of Google’s revenue. Even if Do Not Track would significantly impact third-party advertising revenue, Google would not have an incentive to press its adoption unless it could gain a significant business advantage over its competitors that offset those losses—and there’s no reason to believe it could. Third, the concern is largely moot: Google has yet to implement Do Not Track in Chrome, let alone encourage the feature’s adoption.

Do Not Track is on its way to becoming an Internet standard. In collaboration with Sid Stamm at Mozilla we’ve submitted an Internet-Draft to the IETF, specifying both the HTTP header syntax and the requirements for compliance.

This is just the beginning of the IETF’s process and the evolution of the draft. But it’s a transformative moment for web privacy: Do Not Track is now a formal standards proposal. Every browser, advertising network, analytics service, and social plug-in provider has a clear instruction manual on how to implement Do Not Track.

We owe a tremendous debt of gratitude to the colleagues and friends whose efforts have made Do Not Track a reality: Alissa Cooper, Peter Eckersley, Alex Fowler, John Mitchell, Ashkan Soltani, Lee Tien, and Harlan Yu. And we particularly thank Chris Soghoian, Do Not Track’s unflagging champion for nearly two years.

Last Friday we submitted a comment to the FTC articulating our vision for Do Not Track. We expanded on a number of views already expressed on this blog: Do Not Track is about much more than behavioral advertising, an HTTP header is the right implementation, and Do Not Track is no threat to ad-supported businesses. Here are the new highlights. (For a fuller exposition of each, please see our comment.)

Defining Do Not Track

Beginning with process, there is a strong temptation to leap into precise definitions of Do Not Track. But any set of bright-line rules will incorporate distinctions that are controversial. The FTC should be vested with guided rulemaking authority to, in collaboration with stakeholders, develop bright-line rules. This is the standard model for regulation; Do Not Track should be no different.

Turning to substance, Do Not Track should be coextensive with the privacy concern it addresses: third-party tracking. Providing rulemaking guidance for Do Not Track thus requires establishing standards for “third party,” “tracking,” and exceptions.

Defining Third Party

In our view, the privacy distinction between first parties and third parties is shorthand for user expectations. An entity acts in a first-party capacity if a user reasonably expects to interact with it; it acts in a third-party capacity if a user does not. Relevant factors for user expectations include domain names, branding, and business relationships. In most cases resolving the standard is straightforward: The website the user visits is a first party; an advertising network or analytics provider is a third party.

Defining Tracking

The user’s remedy against third parties should, in the first instance, be coextensive with the privacy violation: Do Not Track should prohibit all data collection, retention, and use. Of course, the online ecosystem is quite complex, and we recognize that there will be a need for well-delineated exceptions where privacy concerns must reasonably give way to greater interests.

Exceptions

Exceptions to Do Not Track may be warranted when there is significant commercial need and privacy concerns and enforcement impact are minimal. We believe a two-step standard best captures this policy: First, legitimate commercial interests must substantially outweigh privacy and enforcement interests, and second, the means of achieving the commercial interests must have no greater privacy and enforcement impact than necessary. A number of tools are available for minimizing the privacy and enforcement effects of an exception, including client-side storage, dropping parts of data, secure hashing, retention periods, internal business controls, limited sharing agreements, trusted intermediaries, and audits.

Do Not Track Can Be Enforced

We envision two technical approaches to verifying Do Not Track compliance. First, most tracking at the application layer can be detected by modifying a browser to report tracking-related activity. For example, if after receiving a Do Not Track header third-party embedded content sets a unique cookie or lists the browser’s plug-ins, the third party may be violating Do Not Track. Second, behavioral advertising can be identified by monitoring ads for interest targeting. Data should be sourced using both crawling and crowdsourcing to ensure comprehensive coverage of top websites and a real-world sample of observations. We are beginning development of a Do Not Track verification system with colleagues in the Stanford Security Laboratory, and we look forward to sharing our work in the coming months.

Do Not Track Should Be Extended to Mobile Platforms

Third-party tracking is proliferating on mobile platforms; such tracking implicates the same privacy concerns as third-party tracking on the web, and likewise warrants a Do Not Track choice mechanism.

The Do Not Track header can be easily adapted to mobile platforms. Instead of a universal browser setting, Do Not Track should be a platform-wide preference that adds a Do Not Track header to all HTTP requests and provides a Do Not Track signal to apps. Much as embedded third-party web trackers would check for the Do Not Track header, embedded third-party mobile app trackers would check for the Do Not Track platform preference. Paralleling the granularity of the header, apps should be able to interact with the platform to request an exception from Do Not Track. As for verification, since third-party mobile tracking is heavily concentrated the problem is much simpler than in the desktop browser context.

Turning to policy, the first vs. third party distinction also seamlessly transitions to the mobile context. An app is a first party; a behavioral advertising network embedded in the app would be a third-party since its presence violates reasonable privacy expectations.

FTC Involvement is Necessary

When we initially articulated our vision for Do Not Track, we noted it could be implemented voluntarily or through industry self-regulation. We now believe legislation and FTC involvement are necessary.

In our view, third-party opposition to Do Not Track at a technological level is largely a façade. The HTTP standard is designed to allow flexible signaling with headers; Internet Explorer alone uses at least eight proprietary headers.

The substantive disagreements about Do Not Track arise from policy. A number of third parties oppose a stringent definition of third-party tracking. Given the diversity of online business models and businesses Do Not Track would affect, and given the consensus-based nature of the relevant trade associations, we believe voluntary comprehensive adoption will not occur. Moreover, as an empirical matter, the online advertising industry in particular has a track record of unsuccessful self-regulation.

The Federal Trade Commission is the right agency to define and enforce Do Not Track. The Commission’s growing technical staff lends it unique domain expertise for defining Do Not Track, and its capacity for and experience with consumer protection actions prime it to enforce Do Not Track.

The FTC received 439 comments on its draft privacy report. We strongly encourage reading the comments by our colleagues at the Electronic Frontier Foundation and the Center for Information Technology Policy.

We thank John Mitchell for his helpful feedback.

We’re pleased to announce we’re beginning work on an IETF Internet-Draft for the Do Not Track header. We look forward to incorporating broad feedback.

In anticipation of the first version of the Internet-Draft, we’re making a few minor updates to the header. The reference implementations at DoNotTrack.Us will be revised shortly.

Dropping “X-”

Since Do Not Track is entering a standardization process, convention dictates dropping the prefix “X-”.

Abbreviating “DNT”

In keeping with header naming best practices, and to conserve network resources, we’re shortening the name.

Adding a “0″ Value

There’s an important policy distinction between users who consent to third-party tracking and users who haven’t expressed a preference. To clarify this difference, the header now has three states:

“DNT: 1″ – The user opts out of third-party tracking.

“DNT: 0″ – The user consents to third-party tracking.

[No Header] – The user has not expressed a preference about third-party tracking.

“If you remove tracking, you remove advertisers.” “Stop [data] sharing and you put a stop to the Internet as we know it.” “Thousands of small websites may disappear.” “Would you like to pay $20 a month for Facebook?” A spate of such recent commentaries have speculated that Do Not Track could hobble advertising-supported businesses. Here’s why it won’t.

Do Not Track would only affect a sliver of the online advertising market.

First, a brief overview of online advertising. Suppose you operate a high-end Napa winery and decide to run an ad. You might place your ad on a specific website (“first-party advertising”), or you might arrange your ad with an advertising network that spans thousands of sites (“third-party advertising”). Here’s a sample of how you might target your ad:

• Contextual Advertising: Your ad appears on pages about wine.
• Demographic Advertising: Your ad appears on pages whose visitors tend to be wealthy.
• Behavioral Advertising: Your ad appears to users who have viewed a number of pages about wine.¹
• Search Advertising: Your ad appears on search result pages for the query “wine.”
• Placement Advertising: Your ad appears on particular pages.
• Social Network Advertising: Your ad appears to social network users who have listed “wine” as an interest.

Of these myriad modes of advertising, Do Not Track would only affect one: third-party behavioral advertising, because it incorporates third-party tracking. And that accounted for, at most, just 4% (less than $1B) of 2009 U.S. online advertising expenditures. While the use of third-party behavioral advertising is rapidly growing, so is the online advertising market; projections place behavioral advertising at only 7% of the U.S. online advertising market in 2014 (1, 2).²

Do Not Track would only affect a new segment of the online advertising market.

Not only is third-party behavioral advertising a small piece of the online advertising market, it’s also a new piece. Behavioral advertising accounted for a negligible share of online advertising until roughly 2007 (1, 2). Countless ad-supported online businesses launched and thrived before then.

Do Not Track would cap—not eliminate—third-party behavioral advertising.

Do Not Track is an opt-out mechanism; uptake is likely to be far from complete. Two helpful benchmarks: After seven years of a permanent opt out, fewer than half of U.S. phone numbers are on the Do Not Call registry (1, 2). And after four years of availability, fewer than 3% of Firefox users have installed its most popular add-on (1, 2).

Advertisers might not reallocate their ad dollars.

Websites that host third-party ads usually receive a fixed share of revenue; they earn more only if advertisers spend more (e.g. AdSense, DoubleClick). Do Not Track would thus impact advertising revenue only if it caused advertisers to reallocate online ad dollars. But that would happen only if advertisers have a strong preference for third-party behavioral advertising. There’s some evidence that advertisers don’t: despite the growing availability of third-party behavioral advertising over the past several years, advertisers haven’t rushed to adopt it. In fact, U.S. online advertising revenues grew at an average annual rate of only 3.4% between 2007 (when behavioral advertising first caught on) and 2009.³

There’s a technology fix: interest-targeted advertising without tracking.

Third-party behavioral advertising incorporates tracking to discover a user’s interests. But interest-targeted advertising can be achieved without tracking. Under one alternative model, the web browser learns a user’s interests, and then passes those interests to an advertising network. A number of research and commercial efforts do just this, including AdNostic, RePriv, and Google Ads Preferences.

Ad-supported businesses could ask—or possibly require—Do Not Track users to allow third-party behavioral advertising.

Do Not Track is not all-or-nothing; users who have opted out can opt back into third-party tracking on specific sites or with specific trackers. So even if third-party behavioral advertising were an important revenue source for ad-supported businesses, even if enough users opted out to have an impact, even if advertisers were inclined to pull their ad dollars, and even if alternative technologies for interest-targeted advertising weren’t available, a business would still have an easy remedy: ask—or possibly require⁴—visitors to disable Do Not Track on the site. The proposal is about increasing privacy choice and transparency, not restricting online business practices.

The reports of advertising’s death are greatly exaggerated.

[1] For simplicity this post glosses over behavioral retargeting, a small subset of behavioral advertising.

[2] These figures reflect both first- and third-party behavioral advertising. They should be taken as an upper limit on the market size for third-party behavioral advertising.

[3] One possible reason: behavioral ads may be only a marginally better deal for advertisers. In Q4 2009, a behavioral ad was 2.1x as effective as the average online ad—but it cost 2x as much (1).

[4] Some regulatory proposals have called for disallowing such “tiered” access.

Many thanks to Arvind Narayanan for endless patience in reviewing drafts. All views are my own.

Since our introduction of DoNotTrack.Us last week we’ve received a deluge of questions. This post answers some of the most common inquiries. If we haven’t covered an issue you’d like a response on, shoot us an email and stay tuned – more Q & A posts are in the pipeline.

Q: Do Not Track does not block third-party tracking. Wouldn’t that be a better solution?

Some privacy-conscious users block third-party tracking, most commonly through browser add-ons. This type of self-help is completely compatible with and complementary to Do Not Track; many Do Not Track users may elect to use blocking software. But blocking alone is not a complete solution to web tracking. Here are our chief concerns:

• Universal blocking is infeasible. Web security research (1, 2, 3) has uncovered dozens of means of tracking users; technical barriers to all these approaches are not practical. And a recent informal study of popular Firefox blocking add-ons suggests that blocking is, in practice, far from a universal opt out. Users should not be left guessing as to whether they’ve actually opted out of tracking.
• Blocking software requires perpetual development and user vigilance. There is frequent turnover of tracking services and tracking technologies. If a developer takes a break, its blocking tool will diminish in effectiveness. Users must, consequently, periodically ensure their blocking software is still maintained and up-to-date.
• Blocking inhibits third-party tools. A number of popular website tools and plug-ins are hosted by a third party that also tracks users. Blocking would disable these tools, while Do Not Track accommodates them.

Q: Would Do Not Track require users to opt out of all third-party tracking, on all sites?

No. Do Not Track users would have the ability to whitelist sites and tracking services by simply not sending the Do Not Track header (or otherwise signaling that tracking is permissible). Whitelist management is primarily a user interface problem; we look forward to creative solutions in this space.

Q: Ads provide important context and notice for managing tracking preferences. How would Do Not Track be linked to ads?

Linking ads, privacy notices, and other visible content to a user’s Do Not Track settings is certainly possible. A software interface (i.e. a protocol handler or JavaScript API) could facilitate querying the user’s Do Not Track settings and prompting the user to make changes.

Q: Would Do Not Track be enabled by default?

Do Not Track could be a default; this is up to browser vendors, legislators, and regulators. Owing to conflicting business interests and engineering demands, we find it unlikely the major browser vendors will voluntarily enable Do Not Track by default. We caution in general, however, against regulation of client software, which has historically proven challenging.

Q: How difficult would it be for a tracking service to implement support for Do Not Track?

We did not have difficulty implementing Do Not Track for popular web servers and web application frameworks. We believe tracking services will have a similar experience, though the business logic required may involve more effort.

Q: Does Do Not Track require the cooperation of browser vendors?

To support Do Not Track, a browser must at minimum provide an add-on API that allows header modification (we review browser APIs here). Browser vendors may also choose to implement Do Not Track themselves.

Q: Will older browsers be left out of Do Not Track?

Backwards compatibility is a challenge for any new web technology. We are aiming for as broad support for Do Not Track as is possible, but we recognize that a minority of older browsers may be left out.

Q: Is Do Not Track a modification to HTTP?

No. Custom HTTP headers are a component of the HTTP standard (see IETF RFC 2616).

The web privacy debate is stuck. Privacy proponents decry the diffusion of behavioral advertising and tracking services (1, 2, 3); industry coalitions respond by expounding the merits of personalized content and advertising revenue (1, 2). But for the average user, the arguments are academic: there is no viable technology for opting out of web tracking. A registry of tracking services, like privacy advocates proposed years ago, is cumbersome and unmanageable. Fiddling with cookies, as many advertising networks and anti-regulation advocates recommend, is an incomplete and temporary fix; both Google and NAI (an advertising industry association) have already moved away from opt-out cookies.

Do Not Track ends this standoff. It provides a web tracking opt-out that is user-friendly, effective, and completely interoperable with the existing web. The technology is simple: whenever your web browser makes a request, it includes an opt-out preference. It’s then up to advertisers and tracking services to honor that preference – voluntarily, by industry self-regulation, or by law.

Arvind Narayanan and I have been researching Do Not Track for several months, and are pleased to now introduce DoNotTrack.Us, a compilation of what we’ve learned. The resource explains Do Not Track, provides prototype implementations, and answers some common questions. We’ll be updating it in the coming months with new findings and responses to feedback.

Excited as we are about the Do Not Track technology, it is but a first step. Important substantive policy questions remain open: What tracking should be impermissible? When a user visits a site, what constitutes a third party? We look forward to collaborating with advertising networks, NGO’s, regulators, lawmakers, and other stakeholders in answering these crucial questions.

The first post in this series rebutted the purported Russian motive for renewed cybersecurity negotiations and the second advanced more plausible self-interested rationales. This third and final post of the series examines the U.S. negotiating position through both substantive and procedural lenses.

——————————

American interest in a substantive cybersecurity deal appears limited, and the U.S. is rightly skeptical of Russian motives (perhaps for the reasons detailed in the prior two posts). Negotiators have publicly expressed support for institutional cooperation on the closely related issue of cybercrime, but firmly oppose an arms control or cyberterrorism treaty. This tenuous commitment is further implicated by the U.S. delegation’s composition. Representation of the NSA, State, DoD, and DHS suggests only a preliminary willingness to hear the Russians out and minimal consideration of a full-on bilateral negotiation.

While the cybersecurity talks may thus be substantively vacuous, they have great procedural merit when viewed in the context of shifting Russian relations and perceptions of cybersecurity.

The Bush administration’s Russia policy was marked by antagonism; proposed missile defense installations in Poland and the Czech Republic and NATO membership for Georgia and Ukraine particularly rankled the Kremlin. Upon taking office the Obama administration committed to “press[ing] the reset button” on U.S.-Russia relations by recommitting to cooperation in areas of shared interest.

Cybersecurity talks may best be evaluated as a facet of this systemic “reset.” Earnest discussions – including fruitless ones – may contribute towards a collegial relationship and further other more substantively promising negotiations between the two powers. The cybersecurity topic is particularly well suited for this role in that it brings often less-than-friendly defense, intelligence, and law enforcement agencies to the same table.

Inside-the-beltway perceptions of cybersecurity have also experienced a sea change. In the early Bush administration cybersecurity problems were predominantly construed as cybercrime problems, and consequently within the purview of law enforcement. For example, one of the first “major actions” advocated by the White House’s 2003 National Strategy to Secure Cyberspace was, “[e]nhance law enforcement’s capabilities for preventing and prosecuting cyberspace attacks.” But by the Obama administration cybersecurity was perceived as a national security issue; the 2009 Cyberspace Policy Review located primary responsibility for cybersecurity in the National Security Council.

This shift suggests additional procedural causes for renewed U.S.-Russia and UN cybersecurity talks. Not only do the discussions reflect the new perception of cybersecurity as a national security issue, but also they nudge other nations towards that view. And directly engaging defense and intelligence agencies accustoms them to viewing cybersecurity as an international issue within their domain.

The U.S. response of simultaneously substantively balking at and procedurally engaging with Russia on cybersecurity appears well-calibrated. Where meager opportunity exists for concluding a meaningful cybersecurity instrument given the Russian motives discussed earlier, the U.S. is nonetheless generating value.

While this favorable outcome is reassuring, it is by no means guaranteed for future cybersecurity talks. There is already a noxious atmosphere of often unwarranted alarmism about cyberwarfare and free-form parallels drawn between cyberattack and weapons of mass destruction. Admix the recurrently prophesied “Digital Pearl Harbor” and it is easy to imagine how an international compact on cybersecurity could look all-too-appealing. This pitfall can only be avoided by training an informed, critical eye on states’ motives to develop the appropriate – if any – cybersecurity negotiating position.

The first post in this series rebutted the purported Russian motive for negotiations, avoiding a security dilemma. This second post posits two alternative self-interested Russian inducements for rapprochement: legitimizing use of force and strategic advantage.

——————————

An alternative rationale for talks advanced by the Russians is fear of “cyberterror” – not the capacity for offensive cyberwarfare, but its use against civilians. A weapons use treaty of this sort could have value in establishing a norm against civilian cyberattack… but there are already strong international treaties and norms against attacks aimed at civilians. And at any rate the untraceability of most cyberattacks will take the teeth out of any use-banning treaty of this sort.

The U.S. delegation is rightly skeptical of this motive; the Russians may well be raising cyberterror in the interest of legitimating use of conventional force. The Russians have repeatedly likened political dissidence to cyberterror, and a substantive cyberterrorism treaty may be submitted by Russia as license to pursue political vendettas with conventional force. To probe how such a treaty might function, consider first a hypothetical full-blown infrastructure-crippling act of cyberterror where the perpetrator is known – Russia already need not restrain itself in retaliating. On the other hand, consider the inevitable website defacements by Chechen separatists or Georgian sympathizers in the midst of increasing hostilities – acts of cyberterrorism in violation of a treaty will assuredly be added to the list of provocations should Russia elect to engage in armed conflict.

This simple thought experiment reveals the deep faultlines that will emerge in negotiating any cyberterrorism treaty. Where is the boundary between vandalism (and other petty cybercrime) and cyberterror? What if acts are committed, as is often the case, by nationals of a state but not its government? What proof is required to sustain an allegation of cyberterror? Doubtlessly the Russian delegation would advance a broad definition of cyberterror, while the Americans would propose a narrowly circumscribed definition. Even if, improbably, the U.S. and Russia negotiated to a shared definition of cyberterror, I fail to see how it could be articulated in a manner not prone to later manipulation. It is not difficult to imagine, for example, how trivial defacement of a bank’s website might be shoehorned into a narrow definition: “destructive acts targeting critical civilian infrastructure.”

Another compelling motive for the Russians is realist self-interest: the Russians may believe they will gain a strategic advantage with a capacity-limiting cyberwarfare treaty. At first blush this seems an implausible reading – the U.S., with its technologically advanced and integrated armed forces, appears a far richer target for cyberattack than Russia given its reliance on decrepit Soviet equipment. Moreover, anecdotally the U.S. military has proven highly vulnerable: multiple unattributed attacks have penetrated defense-related systems (most prominently in 2007), and late last year the Wall Street Journal reported Iraqi militants trivially intercepted live video from Predator drones. But looking ahead a Russian self-interest motive is more plausible. Russia has made no secret of its attempts to rapidly stand up modern, professional armed forces, and in 2009 alone increased military spending by over 25% (projects include a revamped navy and a satellite positioning system, among many others). To accomplish this end the Russians may rely to a large degree on information technology, and particularly on commercial off-the-shelf hardware and software. Lacking time and finances the Russians may be unable to secure their new military systems against cyberattack. Thus while at present the U.S. is more vulnerable, in future Russia may have greater weaknesses. Locking in a cyberwarfare arms control agreement now, while the U.S. is more likely to sign on, could therefore be in Russia’s long-term strategic self-interest.

The specific offensive capabilities Russia has reportedly sought to ban are strongly corroborative of this self-interest rationale. In prior negotiations the Russian delegation has signaled particular concern of deliberately planted software and hardware that would allow disabling or co-opting military equipment. The U.S. will likely have far greater success in developing assets of this sort given the at times close relationship between intelligence agencies and commercial IT firms (e.g. the NSA warrantless wiretapping scandal) and the prevalence of American IT worldwide in military applications (think Windows). Russia, on the other hand, would likely have to rely on human intelligence to place assets of this sort.

Russia’s renewed interest in bilateral cybersecurity negotiations also belies its purported security dilemma rationale. Russian interest in talks lapsed between 1996 and 2009, suggesting a novel stimulus is at work, not some long-standing fear of a security dilemma. The recent rise of alleged “cyberterror” and attempts to modernize Russian armed forces – especially in the wake of the 2008 South Ossetia War with Georgia – far better correlate with Russia’s eagerness to come to the table.

To put a point on these two posts, I submit legitimization of use of force and strategic self-interest are far more plausible Russian motives for cybersecurity negotiations than the purported rationale of avoiding a security dilemma and consequent arms race or destabilization. In the following post I will explore the U.S. delegation’s position and argue the American response to Russia’s proposals is well-calibrated.

Late last year the Obama administration reopened talks with Russia over the militarization of cyberspace and assented to cybersecurity discussion in the United Nations First Committee (Disarmament and National Security). My intention in this three-part series is to probe Russian and American foreign policy on cyberwarfare and advance the thesis that the Russians are negotiating for specific strategic or diplomatic gains, while the Americans are primarily procedurally invested owing to the “reset” in Russian relations and changing perceptions of cyberwarfare.

This first post rebuts the Russians’ purported rationale for talks: avoiding a security dilemma.

——————————

The Russians seek a cyberwarfare arms control instrument ostensibly to avoid a security dilemma and arms race, in the vein of past arrangements for nuclear weapons (i.e. SALT I/II, START I/II, and SORT) and anti-ballistic missile technology (ABM), among others. This basis for negotiations does not withstand scrutiny.

A security dilemma may arise where a state has the opportunity to develop a game-changing new weapons system, even if for purely defensive purposes. For fear of strategic disadvantage other powers may elect to develop the weapon – an arms race – resulting in none gaining a strategic advantage and all bearing a significant cost. Alternatively, technologically incapable of matching or unable to afford the development, other states may take destabilizing offensive steps. Arms control treaties resolve this form of security dilemma by committing states to not developing certain weapons.

Cyberwarfare lacks necessary elements of a security dilemma. First and foremost, cyberwarfare capabilities defy quantifiability. Consider the Cold War nuclear arms race, for example, and the strategic fixation on differences in the number and type of nuclear warheads and delivery systems (the “missile gap”). In the absence of such a metric the two powers have no means of calibrating their activities, and there is no persistent pressure to match or surpass some specific capability the other side maintains.

Intelligence might give each power a rough indication of the other’s cyberwarfare capabilities, but it will be harder to come by than for other military operations. Unlike with other weapons systems, cyberwarfare does not require special installations or resources. There are no centrifuge sites to inspect or uranium shipments to track – just talented programmers and generic computer hardware.

A related issue is that a successful arms control agreement on cyberwarfare would require monitoring and enforcement provisions (“trust but verify”). But as discussed above intelligence on cyberwar capabilities will be harder to come by than for other weapons systems. The Biological Weapons Convention is illustrative of how ineffective an arms control treaty may be without effective monitoring: until a 1989 defection the West was unaware of the scope of Russia’s secret biological weapons program.

Supposing, arguendo, that cyberwarfare capabilities did form an avoidable security dilemma, the negative results that make a security dilemma worth avoiding – excessive expenditures and destabilization – do not arise.

Cyberwarfare is cheap. Developing the F-22 aircraft, for example, cost roughly $65 billion; the annual Air Force cyberspace budget, on the other hand, appears in the low billions and consists primarily of personnel and basing expenditures (Strategic Command Press Release; FY2010 budget).

As for destabilization, there is minimal marginal strategic gain from cyberwarfare capabilities. In the Cold War nuclear arms race there was a perception that if the other side achieved even a slight advantage the bipolar strategic equilibrium would collapse. Cyberwarfare is neither perceived to be – nor is it, in actuality – so effective on the margin. While specific capabilities are not public, it is difficult to imagine cyberattacks will be consistently more effective than conventional strikes. Moreover, given the United States’ enormous strategic advantages in the whole, even significant marginal strategic gains would do little to tip the balance of power to Russia.

Having deconstructed the alleged Russian rationale for talks, the next post in this series will explore alternate viable Russian rationales.

Sunday’s New York Times featured a provocative op-ed arguing in addition to regulating “net neutrality” the FCC should also effectuate “search neutrality” – requiring search providers rank results without consideration of business entities. The author heaps particular scorn upon Google for promoting its own context-relevant services (i.e. maps and weather) at the fore of search results. Others have already reviewed the proposal, leveled implementation critiques, and criticized the author’s gripes with his own site. My aim here is to rebut the piece’s core argument: the analogy of search neutrality to net neutrality. Clearly both are debates about the promotion of innovation and competition through a level playing field. But beyond this commonality the parallel breaks down.

Net neutrality advocates call for regulation because ISP discrimination could render innovative services either impossible to implement owing to traffic restrictions or too expensive to deploy owing to traffic pricing. Consumers cannot “vote with their dollars” for a nondiscriminatory ISP since most locales have few providers and the market is hard to break into. Violations of net neutrality, the argument goes, threaten to nip entire industries in the bud and rob the economy of growth.

Violations of search neutrality, on the other hand, at most increase marketing costs for an innovative or competitive offering. Consumers are more than clever enough to seek and use an alternative to a weaker Google offering (Yelp vs. Google restaurant reviews, anyone?). The author of the op-ed cites Google Maps’ dethroning of MapQuest as evidence of the power of search non-neutrality; on the contrary, I would contend users flocked to Google’s service because it was, well, better. If Google Maps featured MapQuest’s clunky interface and vice versa, would you use it? A glance at historical map site statistics empirically rebuts the author’s claim. The mid-May 2007 introduction of Google’s context-relevant (“universal”) search does not appear correlated with any irregular shift in map site traffic.

Moreover, unlike with net neutrality search consumers stand ready to “vote with their [ad] dollars.” Should Google consistently favor its own services to the detriment of search result quality, consumers can effortlessly shift to any of its numerous competitors. It is no coincidence Google sinks enormous manpower into improving result quality.

There may also be a benefit to the increase in marketing costs from existing violations of search neutrality, like Google’s map and weather offerings. If a service would have to be extensively marketed to compete with Google’s promoted offering – say, a current weather site vs. searching for “Stanford weather” – the market is sending a signal that consumers don’t care about the marginal quality of the product, and the non-Google provider should quit the market.

There is merit to the observation that violations of search neutrality are, on the margin, slightly anti-competitive. But this issue is dwarfed by the potential economy-scale implications of net neutrality. The FCC should not deviate in its rulemaking.

In a recent interview prominent antivirus developer Eugene Kaspersky decried the role of anonymity in cybercrime. This is not a new claim – it is touched on in the Commission on Cybersecurity for the 44th Presidency Report and Cybersecurity Act of 2009, among others – but it misses the mark. Any Internet design would allow anonymity. What renders our Internet vulnerable is primarily weakness of software security and authentication, not anonymity.

Consider a hypothetical of three Internet users: Alice, Bob, and Charlie. If Alice wants to communicate anonymously with Charlie, she may relay her messages through Bob. While Charlie knows Bob is an intermediary, Charlie does not know with whom he is ultimately communicating. For even greater anonymity Alice can pass her messages through multiple Bobs, and by applying cryptography she can ensure no individual Bob can piece together that she is communicating with Charlie. This basic approach to anonymity is remarkable in its independence of the Internet’s design: it only requires that some Bob(s) can and do run intermediary software. Even on an Internet where users could verify each other’s identity this means of anonymity would remain viable.

The sad state of software security – the latest DHS weekly bulletin alone identified over 40 “high severity” vulnerabilities – is what enables malicious users to exploit the Internet’s indelible capacity for anonymity. Modifying the prior hypothetical, suppose Alice now wants to spam, phish, denial of service (DoS) attack, or hack Charlie. After compromising Bob’s computer with malicious software (malware), Alice can send emails, host websites, and launch DoS attacks from it; Charlie knows Bob is apparently misbehaving, but has no means of discovering Alice’s role. Nearly all spam, phishing, and DoS attacks are now perpetrated with networks of compromised computers like Bob’s (botnets). At the writing of a July 2009 private sector report, just five botnets sourced nearly 75% of spam. Worse yet, botnets are increasingly self-perpetuating: spam and phishing websites propagate malware that compromises new computers for the botnet.

Shortcomings in authentication, the means of proving one’s identity either when necessary or at all times, are a secondary contributor to the Internet’s ills. Most applications rely on passwords, which are easily guessed or divulged through deception – the very mechanisms of most phishing and account hijacking. There are potential technical solutions that would enable a user to authenticate themselves without the risk of compromising accounts. But any approach will be undermined by weaknesses in underlying software security when a malicious party can trivially compromise a user’s computer.

The policy community is already trending towards acceptance of Internet anonymity and refocusing on software security and authentication; the recent White House Cyberspace Policy Review in particular emphasizes both issues. To the remaining unpersuaded, I can only offer at last a truism: There’s anonymity on the Internet. Get over it.