DSRIP Mid-Point Assessment

Overview of the Mid-Point Assessment Process

February 2016            www.health.ny.gov            DRAFT

DSRIP Mid-Point Assessment

The Mid-Point Assessment is a required component of the New York Delivery System Reform Incentive Payment (DSRIP) Program as identified in Section VIII. 11.d. of the Special Terms and Conditions (STCs) of the 1115 Waiver that governs the DSRIP Program. The requirements of the Mid-Point Assessment are further detailed in Section VI.d. in Attachment I to the STCs.

This document is intended to provide an outline of:

  • Mid-Point Assessment Requirements
  • Mid-Point Assessment Timeline
  • Mid-Point Assessment Process, including the process by which the Independent Assessor will conduct the assessment, the data elements that will be used to conduct the assessment, the PPS recommendations process, and the process for formal approval of the recommendations by DOH, the Project Approval and Oversight Panel, the Commissioner of Health, and CMS.
  • Opportunities for Provider Network Modifications
  • Implementation of Mid-Point Assessment Project Plan Modifications

I. Mid-Point Assessment Requirements

The Mid-Point Assessment requirements are outlined in Section VI.d in Attachment I to the STCs of the waiver. The requirements indicate that at a minimum, the following elements will be assessed:

  1. Compliance with the approved DSRIP project plan, including the elements described in the project narrative;
  2. Compliance with the required core components for projects described in the DSRIP Strategies Menu and Metrics, including continuous quality improvement activities;
  3. Non-duplication of Federal funds;
  4. An analysis and summary of relevant data on performance on metrics and indicators to this point in time;
  5. The benefit of the project to the Medicaid and uninsured population and to the health outcomes of all patients served by the project (examples include number of readmissions, potentially preventable admissions, or adverse events that will be prevented by the project);
  6. An assessment of project governance including recommendations for how governance can be improved to ensure success. The composition of the performing provider system network from the start of the project until the midpoint will be reviewed. Adherence to required policies regarding management of lower performing providers in the network, as described in Section X of Attachment I, will be reviewed with a special focus on any action with regard to removing lower performing members prior to DYs 3, 4, and 5. (Note: Modifying coalition members requires a plan modification);
  7. The opportunity to continue to improve the project by applying any lessons learned or best practices that can increase the likelihood of the project advancing the three part aim; and
  8. Assessment of current financial viability of all lead providers participating on the DSRIP project.

II. Timelines for Mid-Point Assessment

The following table outlines all of the major events, deliverables, and due dates for the completion of the Mid-Point Assessment. The Mid-Point Assessment will be initiated with the DY2, Q1 PPS Quarterly Report, due from the PPS to the Independent Assessor on July 31, 2016. The Mid-Point Assessment will conclude with the PPS implementation of the final recommendations by March 31, 2017.

Event Date
PPS Completes DY2, Q1 Quarterly Report July 31, 2016
Independent Assessor begins reviews of DY2, Q1 Quarterly Report and Initiates Mid-Point Assessment August 1, 2016
Independent Assessor completes initial review of DY2, Q1 Quarterly Report; PPS 15 day remediation period begins August 31, 2016
PPS return DY2, Q1 Quarterly Report to Independent Assessor; PPS 15 day remediation period ends September 14, 2016
Independent Assessor finalizes results of DY2, Q1 Quarterly Report September 29, 2016
Independent Assessor finalizes Mid-Point Assessment Recommendations October 28, 2016
Independent Assessor Recommendations released for PPS and Public Comment October 31, 2016
PPS Response and Recommendation period begins; Public Comment period begins October 31, 2016
Release Initial Mid-Point Assessment recommendations to PPS for review November 22, 2016
Release Initial Mid-Point Assessment Recommendations to public for comment November 29, 2016
PPS Response and Recommendation period ends; Public Comment period ends December 1, 2016
Independent Assessor compiles PPS Response and Recommendations with Public Comment December 8, 2016
Independent Assessor submits Mid-Point Assessment Recommendations to DOH December 12, 2016
Mid-Point Assessment Recommendations released for 30 day Public Comment Period December 15, 2016
Public comment period for Initial Mid-Point Assessment recommendations and Primary Care Narratives ends December 21, 2016
Release Final Mid-Point Assessment recommendations for 30-day public comment period January 3, 2016
Mid-Point Assessment 30 day Public Comment Period Closes January 15, 2017
Project Approval and Oversight Panel Convenes to review Mid-Point Assessment Recommendations January 17 – 20, 2017
Project Approval and Oversight Panel Mid-Point Assessment Recommendations Submitted to Commissioner of Health January 24, 2017
Commissioner of Health submits Mid-Point Assessment Recommendations to CMS January 31, 2017
PPS Complete Modifications to DSRIP Project Plans March 31, 2017

III. Mid-Point Assessment Process

Independent Assessor Reviews and Recommendations
The Independent Assessor will conduct the Mid-Point Assessment consistent with the requirements outlined in the STCs and Attachment I. The Mid-Point Assessment will focus on the progress made by the PPS through the end of the first quarter of DSRIP Year 2 towards establishing the necessary organizational foundation and towards the implementation of the project requirements consistent with the approved DSRIP Project Plan. This section will outline the process that will be followed by the Independent Assessor in conducting the Mid-Point Assessment.

Data Sources
The Independent Assessor will conduct the Mid-Point Assessment using data sources that are available at the time of the assessment and will work to minimize the amount of additional reporting required by the PPS. The primary data sources used in the Mid-Point Assessment will be:

  • Approved DSRIP Project Plan Applications
  • PPS Quarterly Report Submissions - DY1, Q1 through DY2, Q1*
  • Claims and non-claims data for P4R/P4P measures

The Independent Assessor will use the available data to determine if the PPS are implementing the projects in accordance with their approved DSRIP Project Plans and to assess PPS progress towards meeting project implementation milestones.

*Please note that the Mid-Point Assessment will be based on the PPS efforts through the DY2, Q1 PPS Quarterly Report to accommodate the timelines necessary to ensure the completion of all Mid-Point Assessment tasks prior to the start of DY3. While the Independent Assessor will not be considering the PPS progress through DY2, Q2, PPS will be able to use the progress that has been demonstrated through the DY2, Q2 PPS Quarterly Report to support their responses to the Independent Assessor recommendations or as the basis for the PPS recommendations submitted to the Independent Assessor.

Mid-Point Assessment Scoring
The Independent Assessor will assign a score to the overall DSRIP Project Plan for each PPS and to the individual projects in the PPS Project Plan.

The scoring for the overall DSRIP Project Plan will be evaluated for compliance with requirements set forth in the STCs and Attachment I as well as the plans defined by the PPS in the DSRIP Project Plan Application. Specifically, the overall DSRIP Project Plan will be evaluated against the following criteria defined in the STCs and Attachment I:

  • Compliance with the approved DSRIP project plan, including the elements described in the project narrative;
  • Compliance with the required core components for projects described in the DSRIP Strategies Menu and Metrics, including continuous quality improvement activities;
  • Non-duplication of Federal funds;
  • An assessment of project governance including recommendations for how governance can be improved to ensure success. The composition of the performing provider system network from the start of the project until the midpoint will be reviewed. Adherence to required policies regarding management of lower performing providers in the network, as described in Section X of Attachment I, will be reviewed with a special focus on any action with regard to removing lower performing members prior to DYs 3, 4, and 5. (Note: Modifying coalition members requires a plan modification)

For each of the four areas identified in the list above, the PPS will be assigned a ranking in accordance with the following scale:

  • 1 = fully satisfies the applicable criteria
  • 2 = partially satisfies the applicable criteria
  • 3 = does not satisfy the applicable criteria

The scoring for the individual projects will focus on the progress the PPS has made towards the completion of project milestones and measures. While it is not expected that PPS will have completed all of the project requirements at the Mid-Point Assessment, it is important that the PPS demonstrate progress towards the completion of the project requirements associated with each project, specifically those with a required completion date in DY2. The Independent Assessor will also focus their review on PPS progress towards meeting those project requirements that cross multiple projects including, but not limited to, milestones associated with the attainment of PCMH Level 3 certification and the implementation and use of EHRs to share data and track patient engagement across the PPS.

Based on the PPS efforts on project implementation activities through the end of DY2, Q1, each project will be assigned a risk score on a scale of 1 - 5 as outlined in the table below.

PPS Project Risk Score
1 = Project on Track: This the lowest risk score indicating the project is more than likely to meet intended goals.
2 = Project is Very Likely to be on Track: This is a low risk score indicating the project is more than likely to meet intended goals but has minor challenges to be overcome.
3 = Project is Likely to be on Track: This is a moderate risk score indicating the project could meet intended goals but requires some performance improvements and overcoming challenges.
4 = Project is at risk to be on Track: This is a high risk score indicating the project may fail to meet intended goals without significant modifications or performance improvements.
5 = Project is Off Track: This is the highest risk score indicating that the project will more than likely fail to meet the intended goals, even with significant modifications or performance improvements.

The Independent Assessor will use the combination of the overall DSRIP Project Plan compliance assessment ranking and the individual PPS Project Risk Scores to inform the recommendations for the Mid-Point Assessment.

Independent Assessor Recommendations
The Independent Assessor will release initial recommendations for a 30 day comment period. The 30 day comment period will allow for public comment as well as for responses from the PPS. The Independent Assessor recommendations may include, but not be limited to,

  • Continuation of PPS Project Plan;
  • Modifications to the approved project plans, such as a change in a project or projects selected for implementation by a PPS;
  • Changes to the PPS Lead entity;
  • Consolidations of multiple PPS in to a single PPS;
  • Discontinuation of a PPS´ Project Plan;
  • Other remediation or improvements to increase the likelihood of PPS project success
  • Adding new providers to PPS network to assist in meeting PPS performance goals; or
  • Replication of best practices of other PPS.

PPS Recommendations
Once the Independent Assessor has completed its Mid-Point Assessment review and released its recommendations for public comment, the PPS will have the opportunity to submit responses to the Independent Assessor´s recommendations. In submitting responses to the Independent Assessor´s recommendations, the PPS will be able to refute or support the recommendations of the Independent Assessor. The PPS will also be able to propose alternative recommendations to those made by the Independent Assessor or new recommendations that were not included in the initial recommendations made by the Independent Assessor.

Recommendations from the PPS will be reviewed by the Independent Assessor and considered in the development of the final recommendations to be submitted to DOH. The PPS will receive a standard format to be used in submitting recommendations for the Independent Assessor´s consideration.

Independent Assessor Recommendations to DOH
The Independent Assessor will submit a set of final recommendations to DOH based on the initial recommendations made by the Independent Assessor, the public comments received through the 30 day comment period, and the recommendations submitted by the PPS. DOH will review all recommendations submitted by the Independent Assessor and put forth a final set of recommendations for review by the Project Approval and Oversight Panel.

Recommendations to the Project Approval and Oversight Panel
The Project Approval and Oversight Panel (Panel) will convene to review the recommendations put for the by the Independent Assessor for the Mid-Point Assessment. The Panel will focus on a review of the recommendations to modify, consolidate, or discontinue DSRIP Project Plans and the Panel will have the ability to Accept, Accept with Modifications, or Reject the recommendations made by the Independent Assessor.

In addition to the review of the Mid-Point Assessment recommendations, the Panel meetings will provide an opportunity to highlight PPS success through the first five quarters of DSRIP, as identified through the Mid-Point Assessment.

Project Approval and Oversight Panel Recommendations to the Commissioner of Health
Following the conclusion of the Panel meetings to review the recommendations set forth through DOH, the final recommendations will be compiled and presented to the Commissioner of Health. PPS will have the opportunity to submit letters in support of the recommendations from the Panel or to request the reconsideration of the Panel recommendations by the Commissioner, consistent with the process for the DSRIP Project Plan Application scoring. As identified in the proposed timelines in Section II of this document, it is anticipated that the Commissioner will be receiving the recommendations from the Panel no later than one week following the conclusion of the Panel meetings. PPS wishing to submit a formal request for reconsideration to the Commissioner must do so no later than five days after the close of the Panel meetings to ensure their request is considered by the Commissioner.

Commissioner of Health Recommendations to CMS
Consistent with the approach implemented for the DSRIP Project Plan Application scores, the final recommendations from the Commissioner of Health will be submitted to CMS for final approval. In submitting the recommendations to CMS for approval, the Commissioner will prioritize those recommendations that are more transformative in nature, i.e. consolidation of PPS or discontinuation of a DSRIP Project Plan, to ensure sufficient time is provided for CMS review and for PPS implementation of the changes.

IV. PPS Lead Financial Stability Test

While the Independent Assessor will be using data from existing PPS reports and other readily available data sources to minimize additional reporting burdens on the PPS, it will be necessary for the PPS to complete the PPS Lead Financial Stability Test as part of the Mid-Point Assessment. The PPS Lead Financial Stability Test will be administered consistent with the approach utilized for the DSRIP Project Plan Application.

  • For PPS that are led by a NewCo, it is expected that in lieu of available financial data for the NewCo, the Financial Stability Test will be completed by the organizations that have come together to form the NewCo The Financial Stability Test will need to be completed by all lead organizations for the NewCo, however only one of those organizations must pass the test for the NewCo to have passed the test.
  • For PPS that are led by multiple entities but not under a NewCo structure, each of the lead entities will be responsible for completing the Financial Stability Test, however only one of the lead entities must pass the test for the lead to continue in its role.
  • For PPS led by a public entity, the lead entity can submit the necessary documentation to complete the Financial Stability Test or the lead entity can submit documentation from the governmental entity of which the lead entity is a part of. The letter of backing from the governmental entity will be sufficient for the PPS Lead to have passed the test and continue in its role as the lead entity.

In the event the PPS Lead is unable to pass the Financial Stability Test, the Independent Assessor can recommend that the PPS identify a new lead entity that will pass the Financial Stability Test. If the PPS Lead received a commitment of financial backing from another entity as part of the Financial Stability Test for the DSRIP project Plan Application, the Independent Assessor can recommend that the entity that committed to financially backing the PPS Lead assume PPS Lead responsibilities.

V. Provider Network Modifications at Mid-Point Assessment

Removal of Network Partners
In addition to the recommendations submitted by the PPS for the DSRIP Project Plan, the PPS can also include requests for modifications to the PPS network, including the removal of network partners. The modifications to the PPS network may include both additions of partners to the network or the removal of network partners.

For the addition of new partners to the PPS network, the PPS will be required to follow a process similar to the process utilized for the networking reopening that occurred in November 2015.

PPS will receive further guidance on the dates for the network reopening for the addition of new partners with the start of the Mid-Point Assessment.

A PPS wishing to remove network partners must do so in accordance with Section X of Attachment I to the STCs. PPS must demonstrate non-performance or failure of the network partner to meet DSRIP program requirements to support the request for the removal of the network partner. Documentation must include a description of the escalation process employed by the PPS for identifying and tracking non-performing network partners as well as the notifications to the under-performing network partners informing them of the performance concerns of the PPS. In requesting the removal of PPS network partners, the PPS will not be able to remove more than 5% of their network partners.

Safety Net Status of Providers
At the Mid-Point Assessment, PPS will have the ability to request a review of the Safety Net status of any network partners that are not currently designed at Safety Net providers. The review process will follow the process previously used during the Safety Net appeals at the outset of DSRIP. PPS will be responsible for submitting the applicable Safety Net Appeals Form for all providers that it wishes to have their Safety Net status reviewed.

Determinations on the Safety Net status of providers will be made consistent with the process utilized in the completion of initial Safety Net status determinations and in the initial round of Safety Net Appeals. The determinations will be based on the most recent data available and are subject to CMS approval. As with the initial Safety Net determinations and the Safety Net Appeals, only Medicaid providers will be considered for Safety Net status

VI. Implementation of Mid-Point Assessment Project Plan Modifications

PPS will have two months from the time the final recommendations are submitted from the Commissioner of Health to implement the recommended changes to the DSRIP Project Plans for the start of DY3. It is expected that all recommended DSRIP Project Plan modifications are implemented for the start of DY3 to position the PPS for success as funding shifts from Domain 1 infrastructure and Domain 2 and 3 pay for reporting (P4R) to pay for performance (P4P).

The Independent Assessor will monitor the PPS implementation efforts for the required modifications to the DSRIP Project Plans to ensure that the PPS has made the required modifications. In the event the Independent Assessor determines that a PPS hasn´t modified the project plan as required, the Independent Assessor may submit further recommendations to the Project Approval and Oversight Panel. These recommendations may include:

  • Suspension of the PPS project and associated funding; or
  • Termination of the PPS project plan and funding