Inbound Review Checklist

Instructions

Draft Version: January 15, 2025

This is a review process to approve open source software to be used in the agency. If you would like to use a specific software, please complete the following steps.

This document is a draft and is subject to further revisions and updates. The contents may change as additional feedback is incorporated.

Instructions

Request Overview

Existing Products and Alternatives

Software Evaluation [BR-OSS-2]

Total Cost of Ownership [BR-OSS-3]

License Compatibility [BR-OSS-4]

Security Assessment

Additional Notes & Questions

Sign off

Going Forward

Request Overview

Software Information

Please provide the following information on the software:

Project & Team Information

Please provide the following information on the project and team:

Existing Products and Alternatives

According to Section 3.1 of the Source Code Policy:

In meeting their software needs, covered agencies must conduct [a] three-step analysis […] intended to leverage existing solutions – consistent with principles of category management and shared services – and suitable commercial solutions, while mitigating unnecessary spending on custom-developed software solutions.

Conduct Strategic Analysis and Analyze Alternatives

It is important to conduct strategic analysis and alternatives analysis to determine whether to use an existing Federal software solution or to acquire or develop a new software solution.

According to BR-OSS-1, products in use at CMS that provide the required functionality are preferred. CMS prefers mature COTS and OSS products that have demonstrated successful implementation at or outside of CMS. The benefits must outweigh the risks of using new or unproven products. Like custom code, CMS business owners using OSS in a CMS application are responsible for its maintenance.

When evaluating alternatives, consider the following:

Existing CMS-approved software

• Are there other OSS or COTS products in use at CMS that can serve the same purpose as the OSS?

Commercial Solutions

• COTS can be extended through custom code to satisfy your requirements, however this can become difficult and costly to maintain.

Custom Software

• Having determined that no Federal or commercial solutions exist that completely meet their needs, consider building or buying custom software.

Resources

• 18F guide on understanding and choosing a software solution

The project teams at CMS are recommended to conduct market research and analyze commercial and other open-source alternatives that may meet their business needs. For your preferred solution along with each alternative considered, fill out the table below:

Create a table for each solution considered

Preferred Solution: *Insert Name*

Alternative A: *Insert name here*

Software Evaluation [BR-OSS-2]

Finding the right OSS that meets CMS’s needs requires assessing the maturity of the software and the level of support provided by the community involved in the OSS project. Evaluate the software based on the criteria below according to BR-OSS-2:

When assessing each criterion, be sure to flag any potential risks found.

Notes on Productization

CMS is especially interested in the degree to which the OSS is “productized” (in comparison to COTS), which encompasses much of these three criteria. Although most OSS is only partially productized, information is available to support productization. CMS’s primary concern is assessing the burden to overcome the lack of productization and determining whether the benefits outweigh the risks in this regard. Initially, CMS will only consider OSS that has third-party vendor support for productization (Red Hat, for example, is a third-party vendor that offers an “enterprise-ready” distribution of Linux (an open source operating system platform) along with associated support, training, and consulting services).

Considerations

• Does the project directly provide or offer support contracts for this product?
• Is there a 3rd party vendor who provides or offers support contracts for this product?
• Is the product released on a regular cadence or release schedule?
• Does the product use ‘releases’ on GitHub or other distribution platforms?
• Is there training available (paid or free) for this product? If yes, please add URLs.
• Does CMS already pay for or contract with a vendor who supports this product?
• If a subscription lapses, does CMS continue to have access to the product as it is, or do we lose access to the system?
• Do any other agencies or operational divisions use this product in a production system?

Total Cost of Ownership [BR-OSS-3]

For CMS, reducing cost is relevant and a valid driver for using OSS. According to BR-OSS-3, the agency must determine the total cost of ownership and the risks of using open source as a solution before adopting or authorizing adoption of OSS for a CMS project.

It will be CMS’s responsibility to ensure that all subsequent government estimates and/or contractor quotes clearly show a detailed breakout of costs associated with using the OSS.

Cost Types

Evaluation Costs: OSS must be installed and evaluated locally to assess its use within the enterprise.

Maintenance Costs: IT departments must define and implement a strategy to manage the OSS once installed, such as installing patches and upgrades.

Installation and Configuration Costs: Installation and configuration of OSS can be time consuming. Effective installation and configuration depend heavily on the organization’s skill level in using open source development tools, system administration, and operations.

Integration and Customization Costs: Open source projects may not be designed to operate within enterprise IT infrastructure. IT departments may have to integrate and customize OSS to suit their environments.

Operations & Support Costs: Addressed in the License Compatibility section

License Compatibility [BR-OSS-4]

When selecting OSS, CMS must ensure that the associated OSS license is consistent with CMS’s objectives and environment. The TRB is responsible for approval of OSS licenses within the CMS Open Source Policy.

Licensing Model / Compliance

Provide information on the software’s license and terms & conditions associated with its usage

Costs

Provide information on how the software is priced

Provisioning

Provide information on the setup, configuration, and access management process required to enable software usage.

Support

Assess the support services delivered, the tools and processes involved, roles of support teams, and overall operation and support costs [BR-OSS-3].

Security Assessment

To evaluate the risks associated with using the software, our software approval and assurance process incorporates thorough checks focused on security, integrity, dependencies, potential malicious activity, and long-term project support. Leveraging tools like OpenSSF Scorecard and MITRE Hipcheck, we identify and flag potential concerns for further investigation by our product and security teams.

Compliance with Federal & Agency Security Policies

Are there CMS or federal security policies that prohibit the use of the OSS? Does the OSS product allow implementation of CMS or federal security requirements?

Software Composition Analysis

1. Generate a Software Bill of Materials (SBOM) (see: https://www.cisa.gov/sbom for more details)
   1. If project is on GitHub, you can find and export the SBOM in the repository insights page: github.com/<REPO*>*/network/dependencies
   2. If the project is not on GitHub, you can use a tool like pysyft to generate an SBOM too.
2. Use tools in the toolkit below to ingest the SBOM and create reports providing detailed information of the software composition
   1. Check for common and known vulnerabilities using the National Vulnerability Database (NVD)

Toolkit

Below is a list of suggested tools to run for software composition analysis:

Results

**Insert report and review here**

Run and Review OpenSSF Scorecard

The OpenSSF Scorecard assesses security risks and community security practices of the software through a series of automated checks. The scorecard checks for vulnerabilities affecting different parts of the software supply chain including source code, build, dependencies, testing, and project maintenance.

Determining which scorecard minimums to use

• Is it a web-facing project?
• Does it allow for user input?
• Is it written in a memory unsafe language like C or C++?
• Is it packaged and released on a distribution site like pypi, npm, crates, etc…?

Instructions

1. Run the scorecard on the software repository using the CLI.
2. Report the scores in the table below. Check whether they pass the minimum.

Results

Overall Score:

**Insert review here**

Run and Review MITRE Hipcheck

Hipcheck is a command line interface (CLI) tool for analyzing open source software packages and source repositories to understand their software supply chain risk. It analyzes a project's software development practices and detects active supply chain attacks to give you both a long-term and immediate picture of the risk from using a package.

Results

**Insert scorecard report here**

Additional Notes & Questions

*Insert any notes or questions here*

Sign off

After reviewing the materials prepared by the team that put in the request, the business owner signs off on a risk acceptance for usage of the software product.

Requesting sign off from key people on this request.

Going Forward

Contractors using open source frameworks and libraries are responsible for:

• ensuring that the OSS works properly within the CMS environment
• staff is well versed in configuring and subsequently operating the software
• monitoring the community behind the OSS product for new versions and patches, and support for older versions
• monitoring the community to ensure it continues to offer substantial support for the OSS product