Degrading Transparency: Comparing Google, Yahoo and Microsoft

作者：Nart Villeneuve 来源：www.nartv.org

Google, Yahoo! and Microsoft all maintain versions of their search engines for the Chinese market that censor political content. One of the key issues that emerged concerned transparency. In 2006, all three search engines, following Google’s lead, introduced a message that informed user when the results of their searches were censored. The presence of a mechanism of notification is a critical component of transparency. This notification informs users that their search results have been censored and indicates, to a certain degree, the reason (often unspecified “local law”) why based on what the user searched for. The message appeared only when the user’s results were censored and thus it was possible to connect the censorship to specific keywords or websites.

By 2008 the level of transparency has decreased. While Google’s censorship notification has remained essentially the same as it was in 2006, Yahoo! and Microsoft have altered the way in which users are notified of censorship. Yahoo! has put its censorship message at the bottom of every page regardless of whether results are censored or not, in effect de-linking the censorship notification from the results. Microsoft has removed the text completely and buried the censorship notification with a separate “help” page. These developments represent a significant degrading of transparency and accountability.

By removing or hiding the placement of the censorship message, which is vague to begin with, users may be unaware that their results have been censored and by de-linking the censor message from what the user actually searched for the topics and websites that are censored remain hidden from the user. The de-linking of the censorship message from the search results impacts the ability to determine what precise sites and “key words” are being censored.

The presence and placement of a censorship notification, along with the specificity of its content and its connection to the results, is an integral component of transparency. The specificity of the reason why content has been removed is an important component that is lacking in the case of China. In other cases, Google has cited specific laws, such as the DMCA, and other legal documents with which they must comply and reported the information, to some degree, to Chilling Effects.org. Yahoo maintains a list of sites its censored for copyright violations. However, in the case of censored political content in the case China nothing other than a reference to “local law” has been provided.

The presence of a notification that is directly connected to the results (notification appears only when content is actually removed in relation to what the user searches for) positively impacts the ability to accurately identify censored website and restricted keywords. When such notifications are either absent or disconnected from the results (for example, a notification that appears on every page regardless of whether results are censored or not) the ability to determine censored sites with a high degree of confidence diminishes as sites may simply not be indexed by the search engine. Therefore the notification is critical not only for informing users but also for the monitoring process.

June 26, 2006
Engine	Presence	Placement	Specificity	Connection	Screenshot
Google	Yes	High Notification is placed under results	Low Results removed to comply with local law	Yes Notification only appears when results are censored	screenshot
Yahoo	Yes	High* Notification is placed under results	Low Results removed	Yes*	screenshot
Microsoft	Yes	High Notification is placed under results	Low Results removed, link to “help” page that mentions local law	Yes	screenshot

* Yahoo China’s web crawlers operate from within China, behind the GFW, therefore sites that are blocked by China are not indexed by Yahoo (and thus do not need to be censored by Yahoo) leaving only sites that are either not blocked by China or are indexed during periods when there is variation in the capacity of China’s filtering system to actually be censored by Yahoo. The behaviour documented here refers to sites indexed by Yahoo but subsequently censored, not sites that are not indexed by Yahoo at all.

January 25, 2008
Engine	Presence	Placement	Specificity	Connection	Screenshot
Google	Yes	High Notification is placed under results	Low Mentions “local law”	Yes Notification only appears when results are censored	screenshot
Yahoo	Yes	Medium Notification is placed at the bottom of every page	Low Mentions “local law”	No	screenshot
Microsoft	Yes**	Low A link to a separate “help” page which contains a link to section that contains the notification	Low A link to a separate “help” page which contains a link to section that mentions “local law”	No	screenshot

** There is no notification on the actual page that results the search results. The user must click a “help” page and then navigate to yet another section that state that results may be removed in compliance with local law. the notification mentions pornography as a possible reason, no mention is made of political content.

Presence: The presence of a form of notification that informs users that results may be censored.
Placement: The location of the censorship notification message, particularly its placement in relation to the results.
Specificity: The extent to which users are informed about specific laws, orders and/or regulations leading to censored results.
Connection: Notification appears only when content is actually removed in relation to what the user searches for making it possible to determine which specific web sites and keywords have actually been censored.

(The versions of the search engines tested are the specific version for China. Google (www.google.cn) and Microsoft (www.live.com/?mkt=zh-cn) have their servers located outside of China and are tested directly while Yahoo’s (www.yahoo.cn) servers are hosted in China and are tested from inside China. This is necessary to test the search engines without interference from China’s filtering system.)

This is the start of an effort to more systematically monitor transparency over time so I am asking for feedback. Is this information useful? In what ways can it be improved?