5

Design Features

As Chapter 2 explained, social media algorithms influence a user’s experience of social media in ways that are both complicated and highly variable. Young people’s experiences with social media can also be influenced by exposure to violent or otherwise toxic content, by harassment, or through introduction to bad actors, such as adults interested in grooming to sexual predation or incitement to political radicalization. The committee recognizes that perfect controls over what users see is not a realistic or necessarily desirable expectation for social media companies. But there are provisions that can be incorporated into the design of apps, games, and websites that limit the personal information companies collect, the types of content available, and the prompts to extend time on a platform.

There will always be a place for a knowledgeable consumer to make informed decisions about risks faced online. In the same way health literacy can allow patients to have more knowledgeable, better prepared conversations with their health providers, media literacy, a topic discussed in detail in the next chapter, can allow for a more informed understanding about decisions made online. At the same time, the complexity and pace of the online environment far exceed what adolescents—or any layperson—could be reasonably expected to understand. This chapter recommends steps at the level of platform design that would help tip the balance of transparency to the users who support the platforms and the government agencies that monitor the fairness of their operations.

AGE-APPROPRIATE DESIGN CODE

The central mission of age-appropriate design is to make technology safer for young people through “a set of processes for digital services when end users are children [that] aids in the tailoring of the services that are provided” (IEEE, 2021, p.11). Age-appropriate design can extend to the way vendors collect and use information about minors and how schools promote educational technology to students. It includes enhanced privacy protections, whether in products specifically designed for minors (e.g., children’s programming on YouTube) or products they are likely to access (e.g., search engines). The rights and developmental needs of children are central to the determination of the age appropriateness of a product or platform.

Creating or modifying a product to meet child-friendly design standards starts with a full review of how the products’ features may influence children and a plan for how to mitigate those risks and test new changes. The next steps, undertaken simultaneously, involve auditing the features, identifying risks, and mitigating them. Steps to minimize targeted advertising to children, for example, begin with an overview of corporate policies on data privacy and the shareholders’ views on the matter (IEEE, 2021).

A growing interest in designing digital technology for children led the Institute for Electrical and Electronics Engineers (IEEE), an international association for electronic and electrical engineering, and the 5Rights Foundation to release a standard for age-appropriate design in 2021 (Shahriari and Shahriari, 2017). This design puts the burden of establishing users’ age on the producer of the technology. Knowledge of users’ age in turn allows companies to present terms of use that reflect adolescents’ progressively growing capacity for understanding and independent decision making (IEEE, 2021).

Age-appropriate design emphasizes the protection of young people’s online privacy. The code requires platforms to collect only the necessary information and use it in the way that the child (or, in some cases, the responsible adult) had agreed to and not for commerce. It also discourages the persuasive design features (i.e., features intended to extend the time spent on a platform such as push notifications and tones when new content is posted) that extend use, especially at night, and promotes a high standard for content moderation. The standard also stipulates that it is the technology developer’s responsibility to reduce the automated recommendation of violent, offensive, or harmful content and misinformation (IEEE, 2021).

For product developers and vendors, age-appropriate design may seem to impose burdensome restrictions, especially if they work in dif-

ferent jurisdictions with varying levels of relevant legal or regulatory controls. In such a situation, the IEEE guidance encourages full review of all applicable laws and regulations and, when in doubt as to the standard required, to proceed with the service that more conservatively reflects the best interests of the child (IEEE, 2021). For example, if a user declines to enter a birthdate or if age cannot be verified, the technology developer should not offer nudges to stay longer on the platform or push notifications at night.

The California Age-Appropriate Design Code Act,¹ like similar legislation in the UK, is primarily concerned with young people’s data privacy (Information Comissioner’s Office, 2023). The IEEE standard goes slightly further, including guidance on limiting features that encourage extended use of platforms. But even the IEEE standard offers fewer specifics on content moderation. Its guidance encourages companies to make investments in moderation proportionate to risk, that terms of content moderation should be clear, and that parents and children should have a means for redress (IEEE, 2021).

Age-appropriate design guidance can be technically vague and hard to enforce, and assessment of compliance has been described as subjective (Farthing et al., 2021; Franqueria et al., 2022; Goldman, 2022). Whether other states follow the age-appropriate design code lead set by California remains to be seen, and even if they do, critics of age-appropriate design have seen it as infantilizing of children, especially older adolescents, as the emphasis on acting in the best interest of the child presupposes that the child is incapable of discerning what their best interests may be (Collinson and Persson, 2022). What is more, some of the most serious risks to the mental and physical health of young people come from overuse and algorithms that present unhealthy content. These are not necessarily problems that age-appropriate design code aims to solve.

The age-appropriate design movement put concrete parameters on what had been an abstract discussion about children’s privacy. Its emphasis on both the inputs to and outputs of a functional privacy system gives researchers and companies a guideline against which to measure the data collection risks that children encounter online. Yet threats to the mental and physical health of young people are often traced to failures of content moderation, algorithms that promote toxic content, and overuse. Social media platforms would benefit from a similar standard to guide assessment of how their algorithms influence well-being.

___________________

¹ California Civil Code §§ 1798.99.28–1798.99.40.

GREATER TRANSPARENCY AND ACCOUNTABILITY

Social media is an important source of entertainment and connection for many people, especially adolescents. Given the importance these platforms have in people’s lives there is a growing momentum for more openness and oversight of their operation. Much of the public outrage elicited by Frances Haugen’s revelations stemmed from the perception of secrecy, the idea that harms known to executives inside the company were kept from the public (Allyn, 2021). Allowing researchers and civil society watchdogs access to social media data and review of their algorithms would allow for a better understanding of how social media platforms influence young people for better or worse.

It is difficult to determine what effect social media has on well-being or the extent to which companies are doing due diligence to protect young people from the more habit-forming affordances of their platforms, as companies retain extremely tight control on their data and algorithms (Krass, 2022). Publicly available data can support some research. The University of Michigan’s Iffy Quotient, for example, aims to monitor the extent to which Facebook and Twitter amplify disinformation (Center for Social Media Responsibility, 2022). But even this is vulnerable. In 2021 Facebook sued researchers attempting to study political advertising—using publicly available information—because the data scraping tools they used violated the platforms’ terms of service, a topic discussed more in Chapter 8 (Knight First Amendment Institute, 2021; Panditharatne, 2022). The tools Facebook authorizes for researchers, including a searchable advertisement library called Ad Library API and a network data analysis tool called CrowdTangle, provide “tightly circumscribed and spotty information” (Panditharatne, 2022). Civil society groups requesting access to social media data have reported an arbitrary lottery-like process, highly dependent on personal relationships and subject to favoritism (Bradshaw and Barrett, 2022).

In the same way, there can be a seemingly arbitrary approach to the enforcement of content moderation guidelines. Participation in social media has become an important part of modern life. When a platform’s decisions seem unfair, aggrieved users may take the position that they are a victim of corporate overreach, being denied access to a public venue, in a manner similar to being disallowed entry to a movie theater or store (MacCarthy, 2021). While such concerns are reasonable, there is also deep public ambivalence regarding outside interference, especially from the government, in determining which public statements should be amplified and which ones should be silenced. In the balancing of trade-offs, a system for content moderation has emerged that relies on oversight boards,

groups of experts that are neither fully independent of the platform nor fully open about their process (Douek, 2019).

A general lack of transparency regarding social media operations has bred public distrust of the platforms and the companies that run them. Figure 5-1 shows results of a 2021 survey conducted by researchers at George Mason University and The Washington Post indicating widespread mistrust of the platforms. Transparency is a remedy for distrust, as it provides some assurance that the platform is conforming to public values and expectations (MacCarthy, 2021).

Some of the companies’ reluctance to share information is well founded. Platform algorithms are proprietary, which can make obliging companies to share seem unfair and uncompetitive. Social media platforms also hold a great deal of information about ordinary people that could, in the wrong hands, be used for surveillance or blackmail (Bradshaw and Barrett, 2022). Therefore, questions of data access and sharing can be especially fraught. Not all researchers can trust that their work will be free from government interference, nor can civil society organizations always assume that their independence will be respected.

The need for more accountability and openness in social media has attracted the attention of Congress. Dozens of pieces of legislation in the

last two sessions alone have taken aim at advertising transparency, data privacy, protections for minors, oversight of mobile apps, targeted marketing, and other aspects of platform operations.² There is clearly momentum and political will for a system for better oversight. In response to this momentum, the social media companies, including Meta and Alphabet, have come to accept data privacy legislation (Kang, 2018). A prompt coordinated effort to develop technical standards for platform operations, transparency, and data use would be a meaningful step toward a better, global system for platform accountability.

Recommendation 5-1: The International Organization for Standardization should convene an ongoing technical working group including industry representatives, civil society, and academic stakeholders to develop standards for social media platform design, transparency, and data use.

Social media is a widely diverse set of tools, used differently, by different people. The extent to which particular platforms are committed to maximizing the beneficial uses and curtailing the harmful ones is not clear to anyone. The development of uniform standards is an essential precursor to any transparent reporting, benchmarking of progress, or regulation. Without such standards outside auditors cannot judge the effectiveness of content moderation or the role of a platform’s advertising or recommendation algorithms in promoting harmful content. Harmonized standards are also the basis of comprehensible public disclosures, such as those governing terms and conditions of using an online service, or measures taken to counter harassment or hate speech. A standard format for data at the application program interface also greatly eases the study of confidential algorithms (MacCarthy, 2022).

The International Organization for Standardization (known as ISO) is an international, nongovernmental organization with a long history in successfully setting and maintaining international standards. Its members include the national standard-setting bodies of 168 countries and use a process described as “voluntary, consensus-based and market relevant” (ISO, n.d.). Given the worldwide reach of social media platforms and the companies’ need to operate across borders and cultures, such international support and buy-in are crucial. ISO also has long experience and well-defined processes for updating standards to ensure their continued

___________________

² A Congress.gov search for legislation in the 117^th and 118^th Congresses for legislation about online advertising, social networks, social media, online privacy, or online data indicated more than 40 pieces of immediately relevant legislation, with many hundreds more tangentially relevant.

relevance, something that will be necessary given the pace of change in this field (ISO, 2021). ISO has considerable experience in similarly thorny and technical topics. The ISO/IEC 27000 family of standards, for example, provides a model for information security management and data protection (ISO, 2022). The committee envisions a similarly inclusive process guiding the development of platform standards for social media.

The recommended standard setting process would be iterative and dynamic, given the rapid pace of change in social media technology and in society’s perception of threats. The ISO process is also designed to include the full range of stakeholders needed to comment on the management of technical processes, including “people such as manufacturers, sellers, buyers, customers, trade associations, users or regulators” (ISO, 2023).

The Types of Standards Needed

The standards for social media operations and platform design would, like the IEEE standard for age-appropriate design, articulate both inputs and outputs of a functional system. Inputs refer to actions taken by the platform, while outputs are partially driven by the platform but are also shaped by the behaviors of users. Inputs can include processes for content moderation or data use, content of privacy agreements, and mandatory disclosures to users, all reflective of decisions largely within the platform’s control. Outputs could include platform health measures, such as the amount of toxicity on a platform, for example. A platform’s content moderation and take-down policies will influence measures of toxicity, but the platform cannot fully control something driven by the behavior of its users. The distinction is key, as adherence to input standards requires little if any margin for reaction time on the part of the platform.

At the platform level, measures should move beyond simple aggregates and provide informative percentile summaries. The reported percentiles would aim to capture the harm experienced by those most vulnerable, such as the amount of cyberbullying experienced by the most bullied decile of adolescents. Since the association between social media use and health outcomes varies across groups, standards should allow quantification at the group or community level. Finally, aligned with algorithmic transparency standards, on request, platforms should provide summaries at the user level.

To better illustrate how this recommendation would work, Table 5-1 gives examples of key measures for which the ISO working group would develop standards together with examples of input and output measures that could be tracked. As discussed in Chapter 2, platform algorithms cover ranking, ad-targeting, and content moderation. As such, Table 5-1

gives examples of standards for each algorithmic type. As noted earlier, standards should be measured comprehensively, so Table 5-1 gives examples of concrete platform-, community-, and user-level outcomes. When applicable, past work with significant overlap is also presented, although the many empty cells in this column illustrate the importance of building these standards and encouraging their adoption.

One important measure that companies should report is their efforts to remediate youth mental health problems. This information, like certain audit and systemic risk reports, should be available on request to the Federal Trade Commission (FTC). Better clarity on and tracking of the standardized indicators would eventually allow for comparisons across platforms and over time, giving both the public and the FTC better clarity on the risks these platforms pose.

It is important to note that the examples provided in Table 5-1 are proposed for illustrative purposes, not as a definitive list of needs. Part of the value of a reoccurring convening via ISO is that the standards could develop in line with a growing body of scientific consensus on the ways social media influences adolescents. Consider for example, the influence of image sharing platforms on body image discussed in Chapter 4. Given the strength of this association, the recommended standards may do well to include measures related to the amount of such content seen on a given platform.

ADOPTING THE STANDARDS

Critics of the previous recommendation may maintain that such steps are not necessary as the social media industry already has relevant rules in place. Self-regulation has long been relied on in the media: television, movies, videogames, and music all make use of industry standards for content rating (Napoli, 2018). Recent years have seen greater effort at industry self- and third-party regulation of social media, exemplified by Facebook’s Oversight Board (Klonick, 2020). This oversight board could help protect users from unfair treatment (Maroni, 2019). At the same time, there will always be a suspicion that the real goal of such a board, or any effort at self-regulation, is to bolster the platform’s market position or authority (Maroni, 2019). Social media platforms’ success depends on engaging as many users as possible, something controversy and emotion can do (Brown, 2021). Asking companies to moderate the more sensational voices on their platform could be asking them to act against their business interests (Brown, 2021).

Skepticism of self-regulation aside, enacting a regulatory framework across jurisdictions on global companies is not always a legally or logistically viable option (Henderson et al., 2016). An acknowledgement of the

TABLE 5-1 Operationalizing Standards for Social Media Operations, Transparency, and Data Use

NOTE: ^a Brown, 2022; ^b Cranor, 2003.

fact that industry stakeholders are often in the best position to set out operational policies underlies the prior recommendation’s specification that industry should be part of the ISO technical work group. There is also reason to believe that companies will have an interest in monitoring one another against the standards the ISO group develops. For this reason, the social media companies should formally adopt these standards and reference them in their public documents.

The companies would do well to adopt such standards to forestall more sweeping regulatory action (Cusumano et al., 2021). The UK’s proposed Online Safety Bill, for example, put significant demands on platforms, even specifying the type of content moderation technology they must use (Church and Pehlivan, 2023). Such restrictions can be impractical and detract from the time and resources platforms can designate for product improvement or even to developing better tools for content moderation.

Recommendation 5-2: Social media providers should adopt the standards referenced in the previous recommendation as a matter of policy and as specific provisions in their terms of service.

A public statement that platforms will comply with all the measures included in the standard and a commitment to the standard in its terms of service would be a meaningful step toward an enforceable legal structure on social media. Section 5 of the Federal Trade Commission Act gives the FTC the authority to penalize firms that engage in unfair or deceptive business practices, although this provision includes an exception enacted in 1980 prohibiting the FTC from using its unfairness authority to promulgate rules governing children’s advertising.³ Using this authority, the agency has brought enforcement actions against companies that have failed to honor commitments made in their privacy policies and other similar documents (FTC, 2023).

Failure to honor basic cybersecurity standards may also represent an unfair business practice (FTC, 2021). Unlike deception, which is judged against a firm’s affirmative statement, unfairness can be seen as a more general failure to meet society’s expectations, including standards of industry practice (Pertschuk et al., 1980). Though applied more sparingly, unfairness can be the basis for enforcement actions even against egregious conduct by companies that have not actively incorporated those standards into their terms of use (FTC, 2003).

The FTC’s ability to characterize business practices as unfair depends on the agency giving firms sufficient notice of what is necessary to meet

___________________

³ 15 U.S. Code § 57a.

their legal obligations.⁴ The agency’s proposed new rule on commercial surveillance and data security has identified the “extent [to which] commercial surveillance practices or lax data security measures harm children, including teenagers” as an area of particular concern.⁵ An industry standard on data security and advertising could facilitate the agency’s oversight of these practices.

The creation of a standard would also support the FTC’s use of consent decrees as a regulatory tool. The agency will negotiate consent decrees with companies that fail to meet expected standards, as it has done for data protection (Daily Business Review, 2015). Once a company agrees to a consent decree, the terms of the decree determine its obligations to remediate, regardless of whether or not those terms are strictly within the FTC’s authority (Rosch, 2011).

The creation of industry standards for social media would inform the FTC’s governance by consent decree, even for social media providers that do not explicitly adopt the standard into its terms of service. Nevertheless, it is the committee’s hope that the standards development process described in Recommendation 5-1 would trigger a virtuous cycle of compliance. International standards can be a marker of good business practice and even a badge of pride, a dynamic that would be analogous to companies seeking green building certification in the absence of any legal obligation to do so. The normative pressure of industry standards could serve as a signal to the public of a company’s sincere and meaningful steps to mitigate the harms associated with its product.

USING THE STANDARDS

A similar process is underway in artificial intelligence (AI) and machine learning. Ethical AI tool kits are designed to enable more open communication among technology developers, researchers, policy makers, and civil society (Wong et al., 2023). Tools such as Model Cards, which provide short explanations of how and against which machine learning tools are benchmarked, are a step toward transparency in AI (Mitchell et al., 2019). Similarly, public documentation on the provenance of the datasets used to calibrate machine learning models is gaining traction as a way to mitigate the harms a biased model can cause (Gebru et al., 2021).

As part of the ethical AI movement, IEEE has set out standards and guidelines to ensure that the AI systems prioritize human well-being in design (Shahriari and Shahriari, 2017). The standards developed from the

___________________

⁴ Federal Trade Commission vs. Wyndham Worldwide Corp., 799 F.3d 236 (3d Cir. 2015).

⁵ Federal Trade Commission, “16 CFR Chapter I: Trade Regulation Rule on Commercial Surveillance and Data Security,” Federal Register 87, No. 202 (October 20, 2022) 63738.

implementation of Recommendation 5-1 could draw on these principles, evaluating the platform’s transparency about its policies and practices and its accountability for data breaches or violations of user privacy. The standards could evaluate whether the platform has age-verification processes, data encryption, and robust privacy policies in place, along with efforts to educate parents and other stakeholders on cyberbullying and reporting and blocking mechanisms. The standards could shine a light on the extent to which platforms are performing due diligence to enforce their age minimums. In 2021, Common Sense Media found that 38 percent of children between ages 8 and 12 have used social media, for example (Rideout et al., 2021). Standards could also clarify whether a social media platform’s content is suitable for children and teens based on age-appropriate criteria and whether the design of the platforms’ features and affordances for young people are developmentally informed or evidence based.

Practically speaking, such standards could form the basis of a rating system or a checklist assessment of items that enumerate responsible design. Such a checklist could be used to create a library of ranked social media platforms or apps wherein included apps have some level of endorsement for children or teens. The library could even provide clear language information to parents and guardians about the specific purposes and affordances of each app—something particularly valuable given the dynamic and changing landscape of new social media platforms.

The committee recognizes that greater transparency and accountability in the design of social media do not necessarily prevent young people from accessing inappropriate content or taking risks online. Many young people are tech-savvy and can find ways to bypass age restrictions or privacy settings. Nevertheless, an objective quality benchmark could be invaluable to parents trying to determine which platforms could provide the most positive experience for their children. Complying with certain standards would be an important indicator to anyone in a position to authorize a platform or app for personal or institutional use, as in a school system. Some form of benchmarking apps could help school districts better interpret the market for educational technology.

Social media operations are remarkably poorly understood, especially for products so influential and widely used. Accessible and comparable standards would be an aid to consumers who want a valid indicator of various platforms’ commitment to data privacy, content moderation, and other important aspects of the user experience. This important first step toward product benchmarking could introduce greater transparency and ultimately more fair competition into an opaque market.

REFERENCES

This page intentionally left blank.