7A21--VISN Radiology Artificial Intelligence

STATEMENT OF WORK STATEMENT OF WORK Confidential Page 2 of 2 STATEMENT OF WORK FDA approved Radiology Artificial Intelligence (AI) software platform with multiple imaging AI applications for the Veterans Integrated Service Network (VISN) 1 1. Contracting Officer s Representative (COR) / Point of Contact (POC) 2 2. Contract Title: 2 3. Background 2 4. Scope 2 5. General Requirements 3 a) Additional Equipment or Replacement Equipment (EE number): 3 6. Privacy & Confidentiality statement 3 7. Contractor Responsibilities: 4 a) Project Management / Reporting 4 i. Deliverables: 5 b) AI Platform Functional Requirements 5 i. Deliverables: 5 c) Radiology AI software platform implementation 7 i. Deliverable: 8 d) Training and Support- 8 i. Deliverables: 8 e) Assessment, Authorization, and Continuous Monitoring 8 i. Deliverables: 9 f) Trustworthy Artificial Intelligence 9 g) VA Technical Reference Model 11 h) Federal Identity, Credential, and Access Management (FICAM) 12 i) Internet Protocol Version 6 (IPV6) 13 j) Trusted Internet Connection (TIC) 14 k) Standard Computer Configuration 14 l) Veteran Focused Integration Process (VFIP) 14 m) Process Asset Library (PAL) 15 n) VA Information Custodial Language 15 o) VA Training 17 p) Privacy & Confidentiality statement: 18 q) Government-Furnished Equipment (GFE)/Government-Furnished Information (GFI): 18 r) Other Pertinent Information or Special Considerations. 20 s) Place of Performance 20 t) Period of Performance 20 u) National Holidays 20 v) Badge 21 w) Parking 21 x) Invoicing & Payment 21 Contracting Officer s Representative (COR) / Point of Contact (POC) Name: Keith Thibault Title: VISN 1 Radiology Program Manger Phone Number: 603-657-0645 E-Mail Address: Keith.thibault@va.gov Contract Title: FDA approved Radiology Artificial Intelligence (AI) software platform with multiple imaging AI applications for the Veterans Integrated Service Network (VISN) 1. Background The purpose of this open solicitation is to provide VISN 1 VA New England with a shared AI platform to assist radiology in the detection and reporting of disease, streamlining the workflow, and improve detections sensitivity and specificity. VISN 1 is comprised of the following locations: VA Boston Healthcare System (Jamaica Plain, West Roxbury, Brockton), Bedford VA Healthcare System, Central Western Massachusetts VA Healthcare System (Leeds and Worcester), Connecticut VA Healthcare System (West Haven, Newington), Providence VA Healthcare System, Manchester VA Healthcare System, White River Junction VA Healthcare System and Maine VA Healthcare System (Togus/Augusta/Portland). VISN 1 is currently providing Radiology services utilizing a hub and spoke PACS model spread across numerous states and Veterans Hospitals and Health Centres. VISN 1 completes an average of 320,000 multi-specialty exams per year. The expected growth rate assumed to be 5% on a yearly basis. Scope The scope of this contract is to provide a Commercial-Off-the-Shelf (COTS), Radiology Platform with AI solution, which utilizes AI techniques and technologies, and Software as a Medical Device (SaMD) delivery model. The tool will integrate with the VA Electronic Medical Health Record (EMHR), radiology Picture Archive and Communications Systems (PACS) and workflows such that providers can leverage AI capabilities within the Radiology Service to improve healthcare delivery to the Veteran and increase operational efficiency for VA. General Requirements AI platform will have a Radiology AI Database and Processing node, cloud based is preferred but on-premises at least on one of the VISN 1 sites is acceptable. All sites will have access to the AI database directly from their site's PACS system. This will allow almost immediate results on the reading of images through the Radiologists' normal workflow. The AI platform will then house three or more FDA-cleared AI software applications that may include: Neurology for stroke, intercranial hemorrhage, and/or TBI detection Pulmonary embolus Bone fracture detection Cardiac calcium detection and scoring Breast density and breast cancer detection (with capability for adding additional AI software applications). The selected AI solution will be fully DICOM & HL7 compliant for orchestration and reporting. AI technologies will allow for instantaneous identification of potentially abnormal images through a deep learning algorithm. Additional Equipment or Replacement Equipment (EE number): The AI platform consists of a combination of Window/Linux servers that may be virtualized. Specifications will be tailored to the VISN's exam study volumes, anticipated AI algorithms and expected/required turnaround times for the specific applications. Although the VISN may supply the hardware and operating systems, the selected solution will be able to assist the VISN with tailored specifications and specific BOM for VA approved hardware devices. Privacy & Confidentiality statement In reference to VHA Handbook 6500.6 Appendix A, Block 6, the equipment included in this contractor does involve connection of IT devices to a VA network. Therefore, C&A, SAP, & the following from App C apply. In reference to VHA Handbook 6500.6 Appendix A, Block 7, the equipment does involve storing, generating, transmitting, or exchanging VA sensitive information. Therefore, the following from App C apply. The contractor, their personnel, and their subcontractors shall be subject to the Federal laws, regulations, standards, and VA Directives and Handbooks regarding information protection, patient privacy, and information system security as delineated in this contract. During contract performance, contractor may require access to VA facility, equipment, information systems, or sensitive data; to provide services, install, train, maintain or repair equipment. VA Course No. VA10176 is privacy training required for VA contractors requiring access to VA facilities for more than 2 consecutive days or any access to VA information systems. Training will be completed prior to being granted access to facilities, equipment, or to the data therein. Training is available on VA TMS website at: https://www.tms.va.gov/SecureAuth35/. Contractor will provide evidence of training to the COR, upon request. Contractor Responsibilities: Project Management / Reporting The Contractor shall deliver a Project Management Plan (PMP) that describes the product delivery schedule, milestones, and risks within the deadlines established by the Government in the tasks below. The Contractor shall deliver the initial PMP at the technical kick-off meeting. Work shall not commence until the VA PM approves the final PMP. The Contractor shall provide the COR with Monthly Progress Reports that include product delivery status, milestones achieved during the past month, activities planning in the coming month, and delivery risks with recommended mitigations. The report shall also identify any problems that arose and a description of how the problems were resolved. If problems have not been completely resolved, the Contractor shall provide an explanation including their plan and timeframe for resolving the issue. The Contractor shall monitor performance against the previous month s delivered Monthly Progress Report and report any deviations. The Contractor shall hold a post-award kick-off meeting within 10 days after contract award. The Contractor shall present, for review and approval by the Government, the details of the intended approach, work plan, and project schedule for each effort. The Contractor shall collaborate with the government to set the dates, locations (shall be virtual), agenda (shall be provided to all attendees at least five calendar days prior to the meeting), and meeting minutes (shall be provided to all attendees within three calendar days after the meeting). The Contractor shall invite the CO, CS, COR, and VA PM. The Contractor shall deliver a short report summarizing the Kickoff Meeting and providing minutes from the meeting within five days after the kick-off meeting. The Contractors shall deliver monthly usage reports to facilitate the VA s execution of the pilot. Usage reports shall include but are not limited to the number of times an end user uses the system over the reported time period, reports of license holders who have not logged in by week, reports of any comments on quality submitted to the contractor transcripts of encounters, data on the number of edits and time spent editing notes prior to their approval by the physician, and total number of scribed encounters. At the end of a performance period (base period and option periods), the Contractor shall present a list of additional integrations or steps needed to scale the project beyond the pilot. Example of this include additional data sets to connect to, additional software requiring integration, and additional tools to be created. Deliverables: Project Management Plan Monthly Progress Reports Technical Kickoff Meeting Report and Minutes Monthly Usage Statistics Report Report on additional integrations needed for scalable use. AI Platform Functional Requirements The Contractor shall provide a Commercial Off-the-Shelf (COTS) FDA-approved Radiology AI Software as a Medical Device Platform SaMD solution, on a Contractor provided infrastructure that is compliant with an on-premises solution. The AI platform infrastructure would include computational and graphical hardware needed to run AI applications/module solutions without access to internet/cloud. Only external connections would be for troubleshooting or initial deployment setup. All additional AI software added to the AI platform shall be FDA cleared. See additional detail regarding VA Authority to Operate (ATO) requirements. Alternatively, the solution may be hosted on the VA Enterprise, contingent upon VA provision of an ATO, in which case the Contractor shall coordinate with VA to ensure compliance with all applicable security and operational directives. Specific platform functional requirements include: Deliverables: Platform must have an Authority to Operate (ATO) with VA systems. Platform must provide access to a marketplace of third-party FDA-cleared machine learning algorithms for medical imaging applications. Solutions available on platform must include but are not limited to the following high-priority applications: stroke triage (including intracranial hemorrhage, large-vessel occlusion), traumatic brain injury; musculoskeletal trauma (including detection of fractures, dislocations, joint effusions, and vertebral body compression deformities); oncologic imaging including cancer risk stratification (breast density) and cancer screening (breast cancer mammography). Platform must have the ability to provide a broad AI marketplace of additional AI solutions beyond the initial AI solutions procured in the base contract that can be added on as an option. The options must allow flexibility to procure new additional AI solutions either at specified VAMCs or across the entire VISN. Platform must be capable of running custom (local) medical imaging algorithms through a standard predefined API. APIs for custom algorithm deployment may include but are not limited to containerized applications, REST endpoints, DICOM servers or other packaged software in a portable format. Applications deployed within the platform must be able to run model inference using raw DICOM files as input. A priority will be given to platforms with internal routing and orchestration capabilities including but not limited to correctly routing specific exams and/or imaging series to specific algorithms. Applications deployed within the platform must be able to generate modality independent, Contractor neutral outputs. Outputs must be routed back to all required DICOM archives including VA PACS, Vista Imaging, and VA enterprise. Outputs must be integrated natively into PACS viewing software using conventional interfaces such as DICOM Secondary Capture (SC). Outputs may also be integrated into third-party dictation software such as Microsoft PowerScribe, including measurements and other quantitative model predictions. A priority will be given to platforms that further extend these capabilities with custom viewing interfaces including but not limited to third party web-based viewers and/or mobile applications. Platform must have the capability to perform quality assurance (QA) and quality control (QC) on deployed models. QA/QC functionality may be implemented through mechanisms such as global dashboard aggregate statistics for usage and algorithm results. Priority is given to platforms with capability of archiving user feedback including identifying algorithm errors (false positives, false negatives) or other operational issues (model inference on incorrect exam). Priority is given to platforms with automated or semiautomated methods of characterizing prospective algorithm performance for example through the use of weak labels derived from radiology reports with language models. Platform may be able to support bulk transfer of DICOM data from VA PACS to external node including VA enterprise cloud. Platform may be able to support image-level and header-level deidentification. Platform may be able to support imaging data annotation including classification, object localization, and segmentation. Platform may be able to support extended algorithm functionality provided by third-party applications Operate within the VA Enterprise with an on-premises approved system. Client application must operate on standard VA Government Furnished Equipment (GFE) and be TRM approved internal VA OIT vetting process. or something to that degree versus outlining everything. Retain local encrypted temporary data backup for resuming use when operation is restored. In the event of intermittent network connectivity issues (less than 5 minutes during one request) the software shall seamlessly resume operation when a connection is restored In the event of a fault or connectivity loss greater than 5 minutes, the software shall provide user feedback and the ability to reinitiate a request from backup once software resumes. Be able to be configure to automatically clear backup storage after a set time limit. Contractor shall provide the data in the VA required format in VA PACS. System shall automatically route candidate cases to the various AI algorithms based on configurable parameters using HL7 and DICOM values. All routing shall be optimized to reduce transfer and analysis timelines. Positive AI findings shall trigger a change to a higher priority status. Negative findings shall not decrement the submitted priority level. Workflow and analysis for ICH AI processing shall be no more than 10 minutes from image ingest at the PACS server to completed analysis with priority status change for a positive finding and write back/storage of key images. AI processing shall generate a structured report to VR system and write back to the PACS with a secondary capture series identifying the finding (positive, negative, unable to process) and will identify the specific slice that triggered the finding. Platform shall be capable of testing and validating model performance of VA developed AI models in a development environment prior to deploying into production. Platform shall be able to test VA developed algorithms for improving quality of care efficiently by detecting disease states and prediction of clinical outcomes through AI. Platform must be able to process DICOM raw images and associated metadata for medical radiology images obtained in routine clinical practice (i.e., MRI, CT, X-Ray, Ultrasonography, etc.) in a separate secure container from FDA AI SaMD commercial off the shelf (COTS) workflows. The SaMD solution shall include a web-based and clinician facing component compatible with MS Windows and Apple iOS. The Contractor shall implement the solution, including training and operational support, at eight of VA Medical Center locations located in the continental U.S. These sites are currently proposed and may undergo adjustments as necessary. The solution shall have the capability to test VA developed algorithms for improving quality of care more efficiently by detecting disease states and prediction of clinical outcomes through artificial intelligence. The platform must be able to process DICOM raw images and associated metadata for medical radiology images obtained in routine clinical practice (i.e., MRI, CT, X-Ray, Ultrasonography, etc.) in a separate and secure container from FDA AI SaMD commercial off the shelf (COTS) workflows. Radiology AI software platform implementation The Contractor shall implement the Radiology AI Software as a Medical Device Platform capability at all VISN 1 VA Medical Center (VAMC) locations. The Radiology AI Software Platform capability shall be configured and ready for release within 90 days after Contract award and completion of VA ATO and shall be provided for the remainder of the PoP. The Contractor shall provide a secure connection to enable access, set up, and configuring of the platform at each site. The platform shall be capable of being client supported, however during this pilot, the Contractor shall provide operational support and advice to medical center personnel on the use of the platform. Deliverable: VA ATO SaMD Licensing Training and Support- The Contractor shall: Provide remote (virtual) and/or on-site training sessions for all VA sites within VISN 1. Training dates shall be coordinated with VA PM, COR, and the individual sites Point of Contact (POC). Training shall be recorded for future use and provided to VA and VA shall own the recordings. The Training shall include both an overview of the platform and detailed walkthrough of the capabilities. Additional future training maybe provided as needed at no additional costs. Provide training materials to VA five (5) days prior to any training event. The Training Materials shall provide role-based training information and user guide to address how the users utilize the Radiology AI Software Platform and software features. Material shall consist of, at a minimum, detailed user manual and quick reference guide in digital formats. Provide technical support, online and via phone, during normal duty hours (Monday Friday, 0700-1730 Local Time relative to the facility implementing) to troubleshoot and resolve technical/functional issues with the SaMD solution, to include accessing the capability. The Contractor shall provide this support for the duration of the PoP. Inquiries shall be responded to within twenty-four working hours. Deliverables: Remote (virtual) training session(s) Training Materials Technical support, online and via phone Assessment, Authorization, and Continuous Monitoring Shall the Contractor use an externally hosted solution, the Contractor shall ensure the Radiology AI Software as a Medical Device Platform SaMD solution has an on-premises solution and received approval for the determined impact level prior to activating any software licenses for use. This shall include: The information system solution selected by the Contractor shall comply with the Federal Information Security Management Act (FISMA). The Contractor shall comply with Federal requirements as mandated by Federal laws and policies, including making available any documentation, physical access, and logical access needed to support this requirement. Following guidance from the Federal CIO, VA will utilize existing JAB ATO. VA will be using an on-premises solution baselines as a starting point, since they are specifically tailored for services. The Contractor shall, where applicable, assist with the VA ATO Process to help achieve agency authorization of the service or migrated application. The Contractor shall complete supporting documentation within 30 days after contract award. The Contractor shall complete a Third-Party Assessment Organizatio…