Google DoubleClick Overview
The history of Google and its advertising will be discussed first. Google was founded in 1998 by Larry Page and Sergey Brin while at Stanford, and took VC funding from KP and other partners. Google was founded with the search engine (with the PageRank algorithm) as the first product, but later added products like Gmail. Eric Schmidt was bought in as CEO in 2001 and recently left but are still on the board. Google IPOed in 2004, using dual class stock for example.
The first kind of ads that Google did was AdWords, dating back to 2000. AdWords was based on search keywords, and the text ads were displayed at the top of the search results (labelled as ads) and were relatively simple. Typically the highest bidder was shown, and the advertiser paid Google when the user clicked on the ads. AdWords involved relatively little tracking at least initially and will not be mentioned much here. At this time Google was also taking a stand against popup ads.
Google bought DoubleClick in 2008. DoubleClick was invented in 1995. It made more sophisticated ad tracking via cookies and the like famous (which was often called “retargeting”), and the problems will be described here. DoubleClick themselves called its product “Dynamic Advertising Reporting and Targeting” at one point for example. Initially DoubleClick was mostly banner ads, and many users developed so-called banner-blindness from these ads. Cookies were itself invented in Netscape in 1994, and the IETF group that developed RFC 2109 and 2965 already know that tracking with “third-party cookies” were a problem (and it was mentioned in these RFCs). Those attempts at IETF cookie standards ultimately failed partly because they were incompatible with current browsers, and led to RFC 6265 that is closer to how cookies are implemented in browsers today. It also led to W3C P3P which was famously implemented in IE6, which also of course failed (partly because it was too complex) and was removed from Windows 10 but was an attempt to get the tracking under control.
Some of the problems with the opt-out methods are similar to the problems of a national “do not email” registry proposed in the US CAN-SPAM Act of 2003 for spam messages, and such lists to “opt out” of spam are widely considered to be unacceptable in general. Even “opt-out” or “unsubscribe” links in spam is widely considered untrustworthy for obvious reasons, though legitimate mailing lists will also have them. That idea came from the similar “do not call” registry for telephone marketing (to stop annoying marketing phone calls which were considered more annoying than spam of course), but email and internet advertising ended up being very different from telephone calls making these laws difficult to enforce. It is far easier to send an email than to call someone for example, and email is also more difficult to trace to the origin especially given that the Internet is global. FTC has a report at https://www.ftc.gov/reports/can-spam-act-2003-national-do-not-email-registy-federal-trade-commission-report-congress describing these problems (it was a report to Congress that was required by CAN-SPAM), including the possibility that such a list can be abused by spammers for example. “Closed-loop opt-in” using confirmation emails for mailing lists on the other hand is widely accepted, but it is not mentioned in CAN-SPAM. One example includes the tracking of “opt-out” using cookies in things like AdChoices, which themselves can be used for other purposes obviously.
There are some reasons why these problems were not apparent (for example to Larry/Sergey) when Google bought DoubleClick, or when remarketing lists was shared, or for that matter when Urchin became Google Analytics and the data was merged with ad data.
The difficulty of researching things like the tying of remarketing lists during the writing of this essay shows some of the problems. It seems that no one cared about the privacy implications when remarketing lists in AdSense and DoubleClick was shared for example. In many cases, advertisers managed “remarketing” lists of “anonymous” visitors that was being tracked by cookies from a central console without thinking of the privacy problems, treating visitors almost as numbers. This ties in with the idea of treating people as “consumers” to be extracted from that are also fundamentally flawed. Another example of this is AOL that famously made it difficult to cancel at one point, partly because measuring “customer loyalty” as numbers to be extracted from consumers was part of their culture. To make it worse, they once charged consumers by the time spent on AOL, so the longer they stay the more revenue they made.
The Google-DoubleClick acquisitions was also controversial, with EPIC, CDD and US PIRG for example filing complaints with the FTC in April 2007, a “first supplement” to the complaint in June 2007, and a “second supplement” in September 2007. There was also a Senate hearing on Sept 27, 2007 with testimonies from a variety of sources regarding that issue. One of the concerns back then was aggregation of tracking data and lack of control by users, though other issues unrelated to ads like storage of IP addresses by search engines were also mentioned. Ultimately it took the FTC until the end of 2007 to approve the deals, after a “second request”.
Before the Google-DoubleClick acquisition, DoubleClick was once planned to merge with Abacus. FTC blocked the merger because of the privacy problems and it never happened. Abacus Direct seems to be a market researching company targeting consumer buying behavior. As a result, Abacus had a lot of personal info about consumers, and there were concerns that this data could be merged with DoubleClick data and may be used to deanonymize them.
Mozilla signed the Google search deal in 2004, before Google even IPOed (let alone things like DoubleClick). Mozilla switched to a Yahoo search deal in late 2014 (by then the search engine was based on MS’s Bing I think), which was part of Marissa Mayer’s attempt to fix Yahoo before it was sold to Verizon. Recently Mozilla switched back to Google as the default search engine.
BrendanEich mentioned in https://twitter.com/BrendanEich/status/932747825833680897 that “It's not a simple Newtonian-physics (or fake economics based on same) problem.” This was about the history of the Google search deal with Mozilla and the fact that it was signed before Google IPOed (when it was being funded by VCs). It is worth mentioning here that Google was founded in 1998 when the now famous dot-com bubble was at the peak and VC funding was common (allowing many startups to grow fast which was considered more important than profits). Many other dot-com startups at the time had problems and ended up failing when the bubble collapsed around 2001. It is worth mentioning that the DoubleClick acquisition dates back to 2007 which was just before the housing bubble famously collapsed leading to another recession, and that bubble probably started just after the dot-com bubble.
BrendanEich mentioned in https://twitter.com/BrendanEich/status/932473969625595904 that “A friend said in 2003 that Sergey declared G would not acquire display ads & arb. Search vs. Display as that would be “evil”.”, before Google even IPOed (in 2004). Unfortunately no other source was given.
It was mentioned on Twitter that Firefox OS enabled tracking protection by default unlike desktop Firefox. It was mentioned in https://twitter.com/andreasgal/status/932757853504339968 that “Yup. I was able to sneak that past management”. I then asked “I wonder if you ever talked to Larry/Sergey.” and Brendan then answered that Andreas didn’t of course. I wonder what would have happened if they did.
https://pagefair.com/blog/2017/gdpr_risk_to_the_duopoly/ has some information on the effect of EU GDPR on Google ads. Notice that AdWords comply if all “personalization” features are removed for example. This included things like “remarketing”. I suspect that AdWords when it was first created in 2000 did not have these features. Other features like “remarketing lists for search ads” are also listed as not compliant, which was of course probably added later too. There was also the infamous cookie law that required notification for placing cookies, which was not that effective but a major step in the direction given that most ad tracking (including DoubleClick) were based on cookies. Google’s implementation of GDPR caused some concerns with publishers (http://adage.com/article/digital/tensions-flare-google-publishers-gdpr-looms/313592/), and some publishers blocked EU IP addresses in response to GDPR.
Data breaches are also a problem. The AOL search data breach from 2006 is pretty famous. The data was “anonymized” but the search terms was often enough to deanonymize users. Ad tracking data is likely similar, including browsing history and the like. Anonymizing data is a useful technique to avoid accidental abuse, but some kinds of data are hard to anonymize in a way that prevent all abuse. For example, various techniques for anonymizing IP addresses and MAC addresses has been developed, including hashing and truncation. Of course, the more data that is consolidated and collected, the higher the risk and impact of a breach.
Recently, Google’s ad blocking and “better ads” (including so-called Better Ad Alliance) involves annoying ads, but don’t fix the fundamental issues described here. Apple’s ad blocking targets retargeting by limiting the life of cookies for example (making them less effective for tracking), but does not change the display of ads or make ads less annoying (for example, autoplay video ads are pretty famous as well, especially with Flash).
Now, fixing the problems might be difficult. Obviously it would affect not only shareholders but pretty much everyone else if Google completely got rid of tracking ads. This includes sites depending on Google ads for revenue as well as Google itself. One example here is that both Microsoft and Novell used Client Access Licenses (CALs). CALs (called node licenses by Novell I think) are per user or per computer licenses common in server software like NetWare and Windows Server. Of course, when Novell moved to Linux, it was open source software that didn’t have CALs (Like with Red Hat, the company only paid for support) meaning that Novell could not expect the same level of revenue as in the NetWare days (they moved to Linux by buying SUSE). The story about Sun’s open source projects and Jonathan Schwartz (the former “ponytail” CEO), and how they eventually had to sell to Oracle is probably pretty famous as well (some examples of open source projects from that period included OpenSolaris, OpenOffice, and OpenJDK). The ad bubble will probably not last forever though. Bubbles like this one is part of the problem of the current debt-based economy (the main problem is that it allows almost infinite amounts of “debt” in US dollars since we got off the gold standard in 1971, including most commonly government debt), especially it encourage extracting as much money as possible from so-called “consumers” (another example is Adobe Creative Cloud subscriptions and how Adobe’s stock price rose after it was implemented).
Google in 2015 hired Ruth Porat as CFO to bring financial discipline to Google. This included cutting unprofitable projects, especially “Google X” research projects and failed projects like Google Glass. According to https://www.bloomberg.com/news/features/2016-12-08/google-makes-so-much-money-it-never-had-to-worry-about-financial-discipline, one of the things they did was “to force the Other Bets to begin paying for the shared Google services they used”. It is probably reasonable to suspect that the increase in ad revenue due to DoubleClick etc is part of why they were able to start so many of these projects in the first place. One recent example is the recent changes in pricing of of Google Maps, mentioned in https://www.inderapotheke.de/blog/farewell-google-maps
For Mozilla, a good example to illustrate the problems with funding browser development is the Opera browser. It was founded in 1995 in Norway. First browser was released in 1996. It IPOed in 2004. The browser used its own engine and it had a lot of unique features, like relatively good CSS support early on (unlike Netscape 4 at the time which famously had relatively poor support and was a problem for web developers for years). At first it was officially a paid browser with a trial version (like Netscape was before 1998), but later they used ads (choices included banner ads or text-based Google ads) for non-paying customers. They eventually signed a search deal with Google which removed the ads and instead just used Google as the default search engine (like Mozilla’s). Of course, there wasn’t much profit margin in a web browser, and so they had to cut costs to keep stocks and quarterly earnings going up (so planning for the future was difficult for example). It was strong in the mobile world before WebKit became dominant there though (before things like iPhone and Android and when things like WML was common) and may still be strong in some embedded applications, with products like Opera Mini that was basically remote rendering of web pages (useful when devices had less processing power). Opera never had much market share (though it had plenty of fans back in the day), and in the end Opera had to switch to Chromium (with the Blink engine) instead of their own engine and codebase in the desktop browser (though they did release last updates for the old one that included for example TLS enhancements). Opera was eventually sold to a Chinese consortium, which eventually renamed the company Otello. The founders eventually started the Vivaldi browser, which is also based on Chromium/Blink but has many differences. In contrast, the Mozilla Foundation was created as a non-profit organization in around 2003 as the old Netscape was dying off with AOL’s help (AOL bought Netscape in 1998 BTW). It owns a for-profit Mozilla Corporation for tax reasons (non-profits are not subject to taxes that for-profits have in the US). I think the corporation owns the search deals like Yahoo and Google for example. You can still donate to the Mozilla Foundation today. Mozilla Firefox 1.0 was released in 2004 after the Foundation was created (and after the branded Netscape 6/7 releases) and quickly took market share from the dominant IE6 that was stagnating the web (by being virtually unchanged for a long time without any real development) and was also well known for security problems like the Download.Ject attacks. MS was forced to respond with IE6 in Windows XP SP2 which in addition to security enhancements also added a few features like pop-up blocking and IE7 which finally bought real enhancements to the core engine that help web developers (especially in places like CSS). The old Netscape search deal with Google dates back to 1999 (obviously Netscape.com was Netscape’s home page at the time), and the success of the deal probably inspired the later Google search deal that Mozilla did.
One alternative to the current tracking ads is called Basic Attention Token. Basic Attention Token is based on the Ethereum cryptocurrency and blockchain (this is like Bitcoin but it is GPU minable for example using a different algorithm and it is one of the most popular GPU minable coins). It was created by the Brave browser, which supports it directly. It is intended to “directly measure” attention. “Attention” is measured on the client side (based on local browser history) and tokens are rewarded for them (called “basic attention metrics”), eliminating the privacy issues. This is often called a “zero-knowledge proof”. There are also other benefits like reducing so-called “click fraud” that hurts advertisers that is a common problem with current ads and removing the need for intermediaries that do tracking like DoubleClick and Taboola (so advertisers also gets more of the money too since they don’t have to pay them). Many other kinds of tokens and “smart contracts” has been created on Ethereum, and so-called initial coin offerings (ICOs) has been the most common use of Ethereum (helping the price to rise). Of course, there is little to no regulation for them at the moment which results in many scam ICOs too (they tends to raise money very quickly, partly since it is so easy to give coins to them).
There are also systems for paying authors directly like Patreon, though it is also trivial to use PayPal or cryptocurrencies for this purpose (though also harder to donate). Patreon allow money to be “pledged” to specific authors. There are also many kinds of “paywalls” implemented on websites, many of which has their own problems like relying on cookies to track how many times people visited a site (to limit the number before the user have to pay of course) or making it difficult to post links on Slashdot, Reddit, and Hacker News that often dislike paywalls for obvious reasons (though some are better than others).
Of course, the problems described in the essay as well as other problems of ads (including annoyance and performance cost of ads) led to more use of ad blockers, which also have their own history. Banner ad blindness has also been known for years now, and Google’s ads tends to be simple text-based ads at least initially. One of the first type of blocking was popup blockers, and Google was taking a stand against popups in the early days (they were well known to be annoying). They became common in browsers by the mid-2000s (even IE6 in XP SP2 had them). At one point circa 2002, AOL/Netscape was disabling the popup blocker from Netscape-branded Mozilla releases (at one time there was the Mozilla source code/binaries and the official Netscape-branded builds based on the Mozilla source). Of course after user backlash they backed off from doing so. This was long before Google bought DoubleClick for example. Later more sophisticated ad and cookie blockers like AdBlock Plus and uBlock Origin came out as add-ons to browsers like Firefox, and one is built into Brave of course (along with BAT as a replacement for the lost ad revenue). Many other browsers have also similar tracking protection including Firefox and IE, but they just disable them by default and may require that ad blocking lists (such as EasyList) be manually loaded. Of course, some sites has been attempting to detect ad blockers and ask users to turn them off (even Ars Technica did it at one point though it only lasted one day), which is also ineffective and not a good idea for obvious reasons (including the fact that it reflects badly on the sites that are doing it). Lawsuits against ad blockers was also tried in some countries, which was obviously mostly unsuccessful (like a lawsuit against AdBlock Plus in Germany by publishers there).