Requirements and Feasibility of a System for Archiving and Disseminating Data from SHRP 2 Reliability and Related Studies (2011)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

C H A P T E R 4 ConclusionsFinal Recommendations The research team recommends that, when evaluated against the criteria of requirements met, conformity with the concep- tual design, initial and life-cycle costs, benefits to stakeholders, risk mitigation, and schedule, the SHRP 2 program proceed with the L13A Reliability Archive project based on the solu- tion Alternative 3 approach. The following sections justify this recommendation based on each of these criteria. Justification Based on Requirements The research team used a simple 3-point scoring method (2 = meets/exceeds requirement; 1 = minimally meets requirement; 0 = does not meet requirement) to evaluate how well each alter- native met the system’s requirements. The results, summarized in Table 4.1, show that Alternative 3 best meets the system requirements. The detailed scoring worksheet can be found in Appendix C. With the exception of the general “Systemwide” category, the requirements are categorized based on functions of an archival system described by the OAIS model. Because many of the requirements are based on function- ality provided by application software, Alternative 1 scored the lowest. Alternatives 2 and 3 are more comparable because both use the same digital repository management software. Alternative 3 ultimately scored the highest because it meets many technical requirements while obviating the need to man- age technology complexity. In cases where it was difficult to evaluate the alternatives without knowing the exact product being assessed, the alter- natives were scored identically. The team believes these detailed requirements will remain useful when making specific prod- uct selections should the SHRP 2 program decide to move ahead with Reliability Project L13A.Justification Based on Conceptual Design Assessing how well the alternatives conform to the concep- tual design is more subjective than assessing requirements on46a function-by-function basis. OAIS describes the roles of pro- ducers and consumers, along with the functionality users in these roles will expect from an archival information system. Producers will require ways to organize, package, submit, assess, and classify information, whereas consumers will require ways to find and access information to which they had been granted access. Together, the six OAIS core archival information system functions of ingest, data management, archival storage, access, administration, and preservation planning are responsible col- lectively for preserving the collection of digital artifacts, moni- toring and ensuring their integrity through physical migrations and format transformations, maintaining their physical secu- rity, facilitating information discovery, and enforcing access control. The availability of COTS software that is built on OAIS principles and concepts clearly influenced the research team’s thinking about solution alternatives. Because this class of dig- ital repository management software provides broad, out-of- the-box coverage of the needs of producers, consumers, as well as the core archival information system functions, the team judged Alternatives 2 and 3 to be clearly superior to Alternative 1 in terms of conformity to the conceptual design. Because archival storage is a central and indispensable part of an open archival information system, it follows that, so long as the system can meet its data preservation mandate, the ulti- mate value of the system is best measured in terms of the ser- vice delivered to users. In this respect, the research team believes that Alternative 3 has the advantage. Cloud storage is part of a larger and rapidly growing trend toward a style of utility computing where services are pro- vided by and accessed through the Internet. While the tangi- ble benefits manifest themselves most clearly in terms of costs and simplified management, as will be discussed next, the research team believes that there is a significant long-term advantage to a deployment strategy that focuses primarily on service delivery and less on technology management. The spirit of the conceptual design is that preserving and curating

47Table 4.1. Requirements Scoring Summary Alternative Scores Category 1 2 3 Producers 0 6 6 Ingestion 4 10 11 Archival storage 20 23 32 Data management 1 14 14 Preservation planning 26 31 37 Administration 12 15 15 Access 3 7 7 Consumers 4 18 18 Systemwide 16 16 28 Totals: 86 140 168the collection so that users benefit from it is what matters most. This is why the team gives Alternative 3 the ultimate edge in this category. Justification Based on Cost Table 4.2 summarizes the life-cycle cost analysis of the three alternatives, details on costs for which can be found in Appendix B. The table illustrates that the life-cycle cost analy- sis strongly favors Alternative 3. Lowest Initial Cost Alternative 3 is about 37% below the $1.2 million budget threshold that SHRP 2 has stipulated for the development and implementation of the L13A Archive. Both Alternative 1 and Alternative 2 are about 15% above this budget limit. This is primarily because Alternative 3 is designed to use the com- mercial cloud storage services. Because of this advantage, Alternative 3 is able to avoid large, up-front capital invest- ment costs. At the same time, Alternative 3 also incurs lowerTable 4.2. Life-Cycle Costs Summary Cost Description Alternative 1 Alternative 2 Alternative 3 Total initial cost Total life-cycle cost Average annual life-cycle cost Present value: First 2 years, 2010 and 2011 Present value: 23 years from 2012 to 2035 Present value: 23 years from 2012 to 2035 $1,309,868 $7,413,047 $322,306 $1,406,751 $7,865,075 $341,960 $758,188 $5,530,132 $240,441system maintenance cost during the first 2 years. As a result, Alternative 3 poses minimum risk to the SHRP 2 program. Lowest Recurring Cost Not only does it exhibit the lowest initial cost, but Alternative 3 also has the lowest life-cycle cost among the alternatives. Both Alternative 1 and Alternative 2 will incur much higher system maintenance costs and periodic hardware upgrade costs dur- ing the system’s life. The largest portion of Alternative 3’s recurring costs is related to the use of cloud storage services. The research team’s estimate is based on the current pricing structure of Amazon’s S3 cloud storage service. Since cloud computing is a disruptive trend in information technology, the team expects that the cloud storage price will become even more competitive in the future, thus leading to lower life-cycle costs than the current estimate for Alternative 3. In summary, based on its lowest initial and life-cycle costs, Alternative 3 is the most cost-effective solution. Justification Based on Benefits In a benefit analysis, benefits are usually defined as either quan- titative and tangible or qualitative and intangible improve- ments expected or resulting from a system investment. Tangible benefits are defined as benefits that can be expressed in terms of monetary value. They typically represent direct revenue to be received during the life cycle of the investment. The intan- gible benefits are those that are qualitative in nature and can- not be ascribed monetary value directly. Similar to many information technology investments, the Reliability Archive project faces the following typical challenges in assessing benefits: • Difficulty in identifying benefits that do not have an obvious market value or price; and • Difficulty in quantifying the value of benefits that do not directly accrue to the investment in the project. To address these challenges, the research team assessed the relative benefits of the alternatives from the perspective of

48the parties who will benefit from the implementation of the Reliability Archive, as well as how well these alternatives can support the implementation strategies recommended in the recent SHRP 2 implementation report that was prepared for Congress (1). Of course, the relative benefits of each alternative can also be assessed with respect to cost. Benefits to the SHRP 2 Program The entire SHRP 2 program will benefit from the implemen- tation of the L13A Archive. The benefits in this category can be assessed with respect to long-term data preservation, shar- ing of system capabilities across projects and programs, and how the alternatives are best positioned to support the imple- mentation of the SHRP 2 program results. INITIAL COST. From the initial investment perspective, Alterna- tive 3 provides SHRP 2 with a huge benefit. The initial cost of Alternative 3 is significantly below the $1.2 million budget con- straint, while Alternative 1 and Alternative 2 are both above it. Alternative 3 clearly helps SHRP 2 avoid a large investment made early in the project prior to system build-out, and well before the business benefits are realized. SCHEDULE. The research team estimated the duration of devel- opment and implementation for each of the three alternatives in the subsequent section. Overall, Alternative 3 provides the highest likelihood of implementing the L13A Archive within the 18-month period that SHRP 2 specifies because it significantly reduces the amount of time to procure and install the necessary IT environment in order for the L13A Archive to operate. The cloud storage services required for Alternative 3 are readily available and require much less time for configuration. LONG-TERM DATA PRESERVATION. SHRP 2 requires the Reliabil- ity Archive to be available for 20 to 50 years. All of the pro- posed alternatives are capable of meeting this goal, assuming they are properly managed over the life cycle of the system. Because Alternative 3 shifts a major portion of this manage- ment responsibility to a service provider with domain exper- tise in this area, the research team judges it to be superior to the other alternatives in this regard. SUSTAINABILITY. Ensuring that obsolescence is avoided wher- ever possible and that technology transitions are well man- aged are keys to the sustainability of the archival system over time. Alternatives 2 and 3 are superior to Alternative 1 in this respect because the application software suites they use pro- vide automated strategies and tools, including support for multiple versions (formats) of a digital object. Alternative 3 has a further advantage by way of virtualization of the mostcomplex technical aspect of the solution, which is the archival storage tier. Offloading responsibility for this to a service provider eliminates the need to periodically refresh storage hardware and manage physical migrations. POTENTIAL LEVERAGE FOR OTHER SHRP 2 PROGRAMS AND PROJECTS. Quite a few projects from other SHRP 2 programs such as Renewal and Safety also collect extensive amounts of data and may eventually require an archive system to preserve their data. Since there is no practical limit to cloud storage capacity, Alternative 3 can easily meet this need, whereas addi- tional hardware would have to be procured, managed, and periodically upgraded for the other two alternatives. CAPACITY FOR GREATER INFORMATION SHARING. Cloud storage is part of a broader web services platform that is constantly evolv- ing and expanding to offer additional functions, applications, and capacities that enable delivery of a wider array of capabili- ties to users. As a result, compared to the other two alternatives, Alternative 3 will provide SHRP 2 with more agility and flexi- bility to increase its data sharing and collaboration capability with other national and regional programs. SUPPORT FOR POSSIBLE FUTURE INSTITUTIONAL STRUCTURES AND GOVERNANCE MODELS. The recent SHRP 2 implementation report suggests several approaches and ideas for building the long-term implementation agent. However, no decision has been made regarding future institutional structures and gover- nance models. The research team expects that, in the interim, the National Academies or TRB will take the responsibility of maintaining the L13A Archive. Alternatives 1 and 2 require the National Academies or TRB to build an extensive IT environ- ment that may not be easy to transition to a future governance structure. By using cloud storage, Alternative 3 has the small- est in-house “footprint” and is relatively neutral to current and future governance models. PACKAGING, BRANDING, AND ENABLING RESEARCH RESULTS TO PRODUCTS. As suggested in the SHRP 2 implementation report, the benefits of some SHRP 2 projects may be optimized if the project results are combined with those of other related research projects into a unified package with unique branding. From an information structure perspective, Alternatives 2 and 3 provide the most capabilities to enable SHRP 2 to achieve this objective because they have the most extensive metadata management functionality. Alternative 3 has a further advantage because under this option the archived data will reside in cloud storage, which, as noted earlier, is part of a broader services platform. This offers the future possibility of easily adding new function- ality without incurring capital expense. Some of the on-demand capability available today includes processing and analysis of large data sets, and data integration (e.g., mashup) services.

49Benefits to the User Community As described, the Reliability Archive system will serve a broad range of users. The following are the research team’s assess- ments of the relative user benefits of the three alternatives. BUSINESS FUNCTIONALITY. Alternative 2 and Alternative 3 should provide the same or similar business functions to the future users in terms of using the L13A Archive. This is because both approaches are based on the same type of digi- tal object repository management software. Alternative 1 will provide directory browsing as the only means to access the data, thus its functionality is much more limited. FOLLOW-ON RESEARCH, TESTING, AND EVALUATION. The primary purpose of the Reliability Archive is to enable future researchers to test, evaluate, and validate the research results and even to build new research on the existing data. Although the pri- mary function of the L13A Archive is to facilitate access and not to provide tools to conduct this research, it has already been noted how Alternative 3 offers more potential becauseof the possibility to leverage other capabilities that are part of the broader web services platform, of which cloud storage is a part. ADVANCED USER ACCESSIBILITY. For advanced user access such as downloading large amounts of data, Alternative 3 eliminates the need to engineer, procure, and manage infrastructure suf- ficient to meet peak and occasional demands. Benefits to Long-Term Implementation Agent As described, a long-term implementation agent should be established to lead and support the implementation of the Reliability Archive system. Its responsibilities include imple- menting and deploying the archive system to a production environment and providing long-term stewardship of the system. Benefits for the implementation agent can be assessed on the basis of the relative capacity of the three alternatives to enable better system performance and reliability, and the rel- ative complexity to manage these alternatives over time (see Table 4.3).Table 4.3. Summary of Benefits by Stakeholder Benefit Targets Benefit Aspects Alternative 1 Alternative 2 Alternative 3 SHRP 2 Program User Community Long-Term Implementation Agent Initial investment under $1.2 million budget Can be implemented in 18 months Long-term preservation Sustainability (avoiding obsolescence, migration management) Potential leverage for other SHRP 2 programs and projects Capacity for greater information sharing Support for possible future institutional structures and governance models Support of program implementation strategy Basic data access and functionality Follow-on research, testing, and evaluation Advanced user accessibility System administration burden System maintenance burden Recurring cost Internal expertise required Long-term stewardship No Possible Yes Yes, but with highest effort Minimal Minimal Least flexible Minimal Minimal No No Highest Moderate Higher Higher Acceptable use of resources No Possible Yes Yes Good Good More flexible Good Yes Good Good Moderate Moderate Higher Higher Better use of resources Yes Lowest Risk Yes Yes, lowest effort Best Best Most flexible Best Yes Best Best Lowest Lowest Lowest Lowest Best use of resources

50SYSTEM ADMINISTRATION BURDEN. Alternative 3 greatly reduces the burden on the long-term implementation agent to provide system administration support for the L13A Archive because this alternative manages data storage via cloud storage services. SYSTEM MAINTENANCE BURDEN. By the same token, the burden of system maintenance on the long-term implementation agent will be minimized under Alternative 3 because it elim- inates the efforts required for hardware and software upkeep and migration. RECURRING COSTS. As analyzed in the previous section, Alter- native 3 is expected to incur lower recurring costs than the other two alternatives. Cloud storage services used by Alter- native 3 adopt the pay-as-you-go model that requires a low initial investment and additional investments that are usage- based. This will help the long-term implementation agent avoid surge-type costs due to periodic hardware upgrade or replace- ment and data migration, thereby allowing better budgetary planning based on system usage over time. INTERNAL EXPERTISE REQUIRED. Because of minimized system administration and system maintenance efforts associated with Alternative 3, the long-term implementation agent will be able to significantly reduce its dependency on specialized IT resources while gaining more flexibility in structuring its cus- tomer support team. LONG-TERM STEWARDSHIP. Maintaining an IT operational envi- ronment is unlikely to be a part of the core competency of the future long-term implementation agent. By alleviating this IT burden on the agent, Alternative 3 allows the agent to focus on the primary goals of the L13A Archive, which are to preserve SHRP 2 research data and to make them accessible to the users via better program management, innovation, and collabora- tion with other transportation programs. An added benefit is that part or all of the cost savings in technology-based capital expenditure may be redirected to better serve the primary mis- sion of the L13A Archive as a long-term data preservation and dissemination tool. Justification Based on Risk Mitigation Risks for the Reliability Archive exist from both technical and business perspectives. This section enumerates these risks by category and compares the ways the three alternatives mitigate these risks. Technical Risk With any technical solution come a variety of risks and dependencies that must be recognized and managed. The following are among the risks associated with the Reliability Archive. DATA PROTECTION. One of the primary goals of the Reliability Archive is the long-term preservation (20 to 50 years) of project-level research information and the accumulated knowl- edge that accrues from it in the form of research products. All of the alternatives can deliver the necessary level of data pro- tection if managed correctly over the life of the system. Alter- native 3 has the lowest risk, however, because it follows best practices for data protection and does not require the National Academies, TRB, and the implementation agent to acquire and maintain sufficient domain expertise in this area to ensure the same level of risk mitigation. TECHNOLOGY OBSOLESCENCE AND MIGRATION. It is reasonable to assume that technology will continue to evolve at the current rates, necessitating hardware replacement at 3- to 5-year inter- vals over the life of the Reliability Archive. This portends a min- imum of three hardware migrations over a 20-year service life. Again, Alternative 3 has the lowest risk because its virtualized, network-based storage has intrinsic capabilities to migrate data. FORMAT OBSOLESCENCE AND MIGRATION. Obsolescence of file for- mats over time presents a risk for data loss. Alternatives 2 and 3 are superior to Alternative 1 in that their OAIS-influenced application models support (1) the identification of file for- mats in the archive that are at risk, and (2) multiple versions of a digital object, allowing for transformation of a soon-to-be- obsolete format to one that is machine- or human-readable. SECURITY. The requirement to maintain physical security as well as prevent unauthorized electronic access is the same for all three alternatives. The differences among the alternatives are simply where the security controls must be applied and by whom; therefore, the three alternatives present similar risk profiles in this respect. PRIVACY. The requirement to tightly control access to certain sensitive information is also the same for all three alternatives. Alternatives 2 and 3 pose a lower risk because their application environments support user-level or role-based access control. VENDOR VIABILITY. The long-term viability of any technology vendor, regardless of size, cannot be predicted. Several factors mitigate this risk. Technologies and technology products that have reasonable market adoption generally continue to be sold and supported by successor companies long after a merger or acquisition. This applies to all the alternatives. Next, data can be insulated from application-level depen- dencies if they are managed in a self-describing, standards- based packaging format. Alternatives 2 and 3 have lower risk

51because they employ this strategy. Furthermore, data can be insulated from specific storage dependencies if they are network-accessible over standard protocols. Alternative 3 presents the lowest risk in that it makes migrating data from one application environment to another or from one storage provider to another relatively simple, should this become necessary. COST. The cost of replacement technology and of managing it is another technical risk that must be considered. It is safe to assume that the pace of technology innovation will con- tinue at the current rate for the foreseeable future, making technology less costly over time (i.e., the cost per unit of storage, network bandwidth, and so forth will decline). Therefore, in the research team’s judgment, none of the alter- natives poses a significant risk for unacceptable cost escala- tion. Because the cloud computing trend will continue to grow, the economy of scale it leverages makes Alternative 3 the lowest risk. FLEXIBILITY. Alternative 3 also poses the lowest risk should the archive’s scope increase dramatically (e.g., to support other pro- grams and house more data) or if it becomes necessary to dis- continue its operation. In either case, the pay-as-you-go model is an inherent advantage of Alternative 3. Business Risk In addition to understanding the technology risks, it is also important to understand the business implications of the three alternatives as they may greatly impact the future long-term implementation agent and its roles and responsibilities. On the basis of the research team’s experience with similar projects, the following risks are quite real during the Reliability Archive’s life cycle: • Potential loss of institutional support for the continuation of the archive’s critical activities and for the maintenance and operation of the system in the post-L13 era; • Lack of a reliable source of continued funding into the uncertain future; and • Lack of backup, by the implementation agent’s staff or con- tractor personnel, for ongoing functional and technology operations. A recent letter from the TRB Long-Term Pavement Perfor- mance (LTPP) Committee addressed to the executive directors of FHWA and AASHTO is a case in point where such business risks are becoming a real threat to the continued development and implementation of projects such as LTPP (2). For the Reliability Archive, these business risks may occur in the near future or years after the long-term implementationagent takes over the responsibility for maintaining and operat- ing the archive. It is prudent to take such risks into considera- tion during the decision-making stage in determining the optimal solution alternative. Even though Alternative 3 cannot and will not completely eliminate these potential business risks, it has the highest probability of reducing the risk factors because of its significantly lower recurring costs, its minimal dependency on specialized IT expertise, and its capacity to allow the implementation agent to make better use of available resources to enhance program management, communica- tions, and collaborations. Justification Based on Schedule One of the key feasibility requirements specified by SHRP 2 is that the implementation of the Reliability Archive be com- pleted within an 18-month time period. Typically, a project schedule is dictated by the technical approach and other fac- tors such as project management, availability of resources, and quality of work. It is not practical at this stage to develop a prescribed proj- ect timeline for each alternative that would accurately specify the duration of each activity. However, in order to compare the three alternatives and draw conclusions on the likelihood of their being completed within the 18-month time period, it is imperative to provide estimates on implementation timeline. Table 4.4 lists the major implementation steps of the three alter- natives and the estimate of the duration of each step. Note that some steps are not applicable to every alternative. This table is built on the following assumptions: • All three alternatives will be implemented in accordance with standard systems development life cycle phases; • Variations on specific implementation steps may arise because of the unique approaches of each alternative; and • Although some activities could be performed in parallel depending on how the project is planned and managed, it is conservatively assumed that all these activities will be per- formed in a sequential manner. The estimated durations shown in Table 4.4 are based on the research team’s understanding of the nature of these steps within each alternative as well as the team’s knowledge of the standard software development life cycle methodology. In actuality, the duration of a step can be affected by many tech- nical, business, and political factors. Its impact on the overall project schedule could be quite substantial depending on the criticality of that step. Table 4.5 assesses the criticality of these activities to the project schedule under each alternative, as well as the likelihood of schedule overrun. From the comparison shown in Table 4.5, the research team expects that the implementation will start with project

52Table 4.4. Estimated Implementation Duration Average Duration (in months) Major Implementation Steps Alternative 1 Alternative 2 Alternative 3 Project planning 1 1 1 Finalize requirements 1 1 1 Finalize system architecture 1 1 1 Finalize user interface design 1 1 1 Procure hardware 4 4 2 Set up data archive infrastructure 4 4 n/a In-house development 5 n/a n/a COTS installation n/a 1 1 COTS configuration n/a 2 2 Cloud computing service setup n/a n/a 3 Acceptance testing 3 3 3 Estimated total project duration 20 18 15Table 4.5. Assessment of Implementation Schedule Risk Likelihood of Schedule Overrun Major Implementation Steps Critical to Schedule Alternative 1 Alternative 2 Alternative 3 Project planning Average Low Low Low Finalize requirements Average Low Low Low Finalize system architecture Average Low Low Low Finalize user interface design Average Low Low Low Procure hardware High High High Low Set up data archive infrastructure High High High n/a In-house development High High n/a n/a COTS installation High n/a Low Low COTS configuration High n/a Low Low Cloud computing service set up High n/a n/a Standard Acceptance testing Average High Medium Mediumplanning and the finalization of requirements and design. These steps will largely involve reviewing and refining the outcome of this project (L13). The deliverables from L13 should provide a jumpstart to these tasks. Both Alternative 1 and Alternative 2 are highly dependent on hardware procurement and installation as well as on data storage environment setup. These steps are on the critical path of the project schedule. They require adherence to National Academies and TRB computer hardware procurement proce-dures, in addition to extensive logistical coordination with the National Academies. This is expected to be a lengthy process. Both Alternative 2 and Alternative 3 will need to install and configure the selected COTS software. The RFP process is expected to lead to the selection of a mature software product. Thus, while this is a critical step, it is not expected to impose significant risks to the project schedule. Although Alternative 1 is viewed as the bare minimum solu- tion, it requires a great deal of effort in project scoping, in-house

53software development, as well as in extensive system and user testing. Therefore, Alternative 1 is considered to be high risk. The cloud storage services needed for Alternative 3 are crit- ical to the overall project schedule; however, they are readily available and require minimal time for configuration. From a technical perspective, all three alternatives can be reasonably completed within an 18-month time period if they are prop- erly managed (see Table 4.6). However, Alternative 1 and Alternative 2 include certain critical steps that could bring potential risks and impact the project schedule, while Alterna- tive 3 does not have such risk factors. Thus, Alternative 3 pro- vides the highest likelihood of completing the implementation of the L13A Archive on time. Conclusions The research team believes that it has established that it is highly feasible for the SHRP 2 program to cost-effectively deploy a data archival system that meets all of the goals and objectives envisioned by its major stakeholders.Table 4.6. Summary of Schedule Risk Analysis Alternative 1 Alternative 2 Alternative 3 Estimated 20 months 18 months 15 months duration Likelihood High Medium Low of project schedule overrunFurthermore, the team believes that Alternative 3 represents the best path to success because of the following considerations: • COTS digital repository management software offers the requisite functionality for producers, consumers, and the effective long-term management of a digital collection. A commercial solution that meets the functional require- ments for the system should be less costly over its life cycle than a custom solution. • Cloud storage is a cost-effective archival storage solution that obviates the need for long-term management of complex technology. • Its lowest initial and recurring costs make it the most cost- effective approach. • It offers the maximum set of benefits to the SHRP 2 pro- gram, user community, and the implementation agent. • It carries the lowest risk both technically and businesswise. • It has the highest likelihood of being deployed within the desired time frame. For these reasons the research team recommends that the SHRP2programproceedwith the L13A Reliability Archive proj- ect as planned, following the approach described in this report. References 1. Special Report 296: Implementing the Results of the Second Strategic Highway Research Program: Saving Lives, Reducing Congestion, Improv- ing Quality of Life. Transportation Research Board of the National Academies, Washington, D.C., 2009. 2. Letter from William H. Temple to Jeffery F. Paniati and John Hors- ley, 17 June 2009, Transportation Research Board. http://onlinepubs .trb.org/onlinepubs/sp/ltpp_letter_24.pdf. Accessed March 3, 2011.