DSRIP Mid-Point Assessment

Overview of the Mid-Point Assessment Process

April 2016            www.health.ny.gov            FINAL

DSRIP Mid-Point Assessment

The Mid-Point Assessment is a required component of the New York Delivery System Reform Incentive Payment (DSRIP) Program as identified in Section VIII. 11.d. of the Special Terms and Conditions (STCs) of the 1115 Waiver that governs the DSRIP Program. The requirements of the Mid-Point Assessment are further detailed in Section VI.d. in Attachment I to the STCs.

This document is intended to provide an outline of:

  • Mid-Point Assessment Requirements
  • Mid-Point Assessment Timeline
  • Mid-Point Assessment Process, including the process by which the Independent Assessor will conduct the assessment, the data elements that will be used to conduct the assessment, the PPS recommendations process, and the process for formal approval of the recommendations by DOH, the Project Approval and Oversight Panel, the Commissioner of Health, and CMS.
  • Opportunities for Provider Network Modifications
  • Implementation of Mid-Point Assessment Project Plan Modifications

I. Mid-Point Assessment Requirements

The Mid-Point Assessment requirements are outlined in Section VI.d in Attachment I to the STCs of the waiver. The requirements indicate that at a minimum, the following elements will be assessed:

  1. Compliance with the approved DSRIP project plan, including the elements described in the project narrative;
  2. Compliance with the required core components for projects described in the DSRIP Strategies Menu and Metrics, including continuous quality improvement activities;
  3. Non-duplication of Federal funds;
  4. An analysis and summary of relevant data on performance on metrics and indicators to this point in time;
  5. The benefit of the project to the Medicaid and uninsured population and to the health outcomes of all patients served by the project (examples include number of readmissions, potentially preventable admissions, or adverse events that will be prevented by the project);
  6. An assessment of project governance including recommendations for how governance can be improved to ensure success. The composition of the performing provider system network from the start of the project until the midpoint will be reviewed. Adherence to required policies regarding management of lower performing providers in the network, as described in Section X of Attachment I, will be reviewed with a special focus on any action with regard to removing lower performing members prior to DYs 3, 4, and 5. (Note: Modifying coalition members requires a plan modification);
  7. The opportunity to continue to improve the project by applying any lessons learned or best practices that can increase the likelihood of the project advancing the three part aim; and
  8. Assessment of current financial viability of all lead providers participating on the DSRIP project.

II. Timelines for Mid-Point Assessment

The following table outlines all of the major events, deliverables, and due dates for the completion of the Mid-Point Assessment. The Mid-Point Assessment will be initiated with the DY2, Q1 PPS Quarterly Report, due from the PPS to the Independent Assessor on July 31, 2016. The Mid-Point Assessment will conclude with the PPS implementation of the final recommendations by March 31, 2017.

Event Date
PPS Completes DY2, Q1 Quarterly Report July 31, 2016
Independent Assessor begins reviews of DY2, Q1 Quarterly Report and Initiates Mid-Point Assessment August 1, 2016
Independent Assessor completes initial review of DY2, Q1 Quarterly Report; PPS 15 day remediation period begins August 31, 2016
PPS return DY2, Q1 Quarterly Report to Independent Assessor; PPS 15 day remediation period ends September 14, 2016
Independent Assessor finalizes results of DY2, Q1 Quarterly Report September 29, 2016
Independent Assessor finalizes Mid-Point Assessment Recommendations October 28, 2016
Independent Assessor Initial Recommendations released to PPS for review. October 31, 2016
Independent Assessor Initial Recommendations released for Public Comment November 7, 2016
PPS Response and Recommendation period ends; Public Comment period ends December 1, 2016
Independent Assessor compiles PPS Response and Recommendations with Public Comment December 8, 2016
Independent Assessor submits Mid-Point Assessment Recommendations to DOH December 12, 2016
Mid-Point Assessment Recommendations released for 30 day Public Comment Period December 15, 2016
Mid-Point Assessment 30 day Public Comment Period Closes January 15, 2017
Project Approval and Oversight Panel Convenes to review Mid-Point Assessment Recommendations January 17 – 20, 2017
Project Approval and Oversight Panel Mid-Point Assessment Recommendations Submitted to Commissioner of Health January 24, 2017
Commissioner of Health submits Mid-Point Assessment Recommendations to CMS January 31, 2017
PPS Develops a Mid-Point Assessment Action Plan and Timeline to address necessary Modifications to DSRIP Project Plans and Submits the Mid-Point Assessment Action Plan to the Independent Assessor for Review and Approval March 2, 2017
Independent Assessor approves Mid-Point Assessment Action Plans for PPS implementation in DY3 March 31, 2017

III. Mid-Point Assessment Process

Independent Assessor Reviews and Recommendations
The Independent Assessor will conduct the Mid-Point Assessment consistent with the requirements outlined in the STCs and Attachment I. The Mid-Point Assessment will focus on the progress made by the PPS through the end of the first quarter of DSRIP Year 2 towards establishing the necessary organizational foundation and towards the implementation of the project requirements consistent with the approved DSRIP Project Plan (inclusive of DSRIP Application and DSRIP Implementation Plan). This section will outline the process that will be followed by the Independent Assessor in conducting the Mid-Point Assessment.

Data Sources
The Independent Assessor will conduct the Mid-Point Assessment using data sources that are available at the time of the assessment and will work to minimize the amount of additional reporting required by the PPS. The primary data sources used in the Mid-Point Assessment will be:

  • Approved DSRIP Project Plan Applications
  • PPS Quarterly Report Submissions - DY1, Q1 through DY2, Q1*
  • Claims and non-claims data for P4R/P4P measures

The Independent Assessor will use the available data to determine if the PPS are implementing the projects in accordance with their approved DSRIP Project Plans and to assess PPS progress towards meeting project implementation milestones. The Independent Assessor recognizes that PPS Quarterly Report submissions through DY2, Q1 will have focused primarily on the Domain 1 Organizational Milestones. The PPS will therefore have an opportunity to expand upon the details of their project implementation efforts through the submission of a narrative for each project the PPS is implementing with their DY2, Q1 PPS Quarterly Report submission. The narrative must, at a minimum, detail any challenges the PPS has encountered in project implementation efforts and how the PPS is working to mitigate those challenges as well as identify any implementation approaches that the PPS would consider a best practice. PPS may also use the narrative to provide additional details on their project implementation efforts to further explain their activities beyond what is documented in the PPS Quarterly Reports. The narrative may also be used to address any changes to the populations that were proposed to be served through the project based on changes identified through ongoing community needs assessments or otherwise.

In addition to the narratives submitted for each project the PPS is pursuing, the PPS will also have the ability to submit a narrative to highlight the overall efforts of the PPS to date. This narrative should focus on organizational efforts in areas such as Governance, Workforce, Cultural Competency and Health Literacy, Value Based Purchasing (VBP), Community Based Organization (CBO) Engagement, and Information Technology, to name a few. For example, if a PPS is aggressively pursuing VBP contracting arrangements with MCOs, the PPS could leverage this narrative to highlight the success of these efforts. This narrative, unlike the project narratives, is not required.

*Please note that the Mid-Point Assessment will be based on the PPS efforts through the DY2, Q1 PPS Quarterly Report to accommodate the timelines necessary to ensure the completion of all Mid-Point Assessment tasks prior to the start of DY3. While the Independent Assessor will not be considering the PPS progress through DY2, Q2, PPS will be able to use the progress that has been demonstrated through the DY2, Q2 PPS Quarterly Report to support their responses to the Independent Assessor recommendations or as the basis for the PPS recommendations submitted to the Independent Assessor.

360 Evaluation Survey

The Independent Assessor will create a 360 Evaluation Survey which is intended to gather information from non-lead PPS network partners about their experience with the PPS. The survey will gather information on the network partner's experience in the following areas:

  • Participation in Governance;
  • Project Development;
  • Project Execution;
  • Communication with and between the lead and other partners;
  • Conflict Resolution;
  • Funds Flow;
  • Overall Satisfaction; and
  • Additional areas may be added to the survey at the discretion of the Independent Assessor.

Additional areas may be added to the survey at the discretion of the Independent Assessor.

The Independent Assessor will select a sample of PPS network partners to participate in the survey. The sample will be done to ensure a cross-section of the PPS networks. The Independent Assessor will distribute the survey to the selected providers. Once all responses are received the Independent Assessor will anonymously summarize the results for each PPS and publish the summary.

Mid-Point Assessment Scoring
The Independent Assessor will assign a score to the overall DSRIP Project Plan for each PPS and to the individual projects in the PPS Project Plan.

The scoring for the overall DSRIP Project Plan will be evaluated for compliance with requirements set forth in the STCs and Attachment I as well as the plans defined by the PPS in the DSRIP Project Plan Application. Specifically, the overall DSRIP Project Plan will be evaluated against the following criteria defined in the STCs and Attachment I:

  • Compliance with the approved DSRIP project plan, including the elements described in the project narrative;
  • Compliance with the required core components for projects described in the DSRIP Strategies Menu and Metrics, including continuous quality improvement activities;
  • Non-duplication of Federal funds;
  • An assessment of project governance including recommendations for how governance can be improved to ensure success. The composition of the performing provider system network from the start of the project until the midpoint will be reviewed. Adherence to required policies regarding management of lower performing providers in the network, as described in Section X of Attachment I, will be reviewed with a special focus on any action with regard to removing lower performing members prior to DYs 3, 4, and 5. (Note: Modifying coalition members requires a plan modification)

For each of the four areas identified in the list above, the PPS will be assigned a ranking in accordance with the following scale:

  • 1 = fully satisfies the applicable criteria
  • 2 = partially satisfies the applicable criteria
  • 3 = does not satisfy the applicable criteria

The scoring for the individual projects will focus on the progress the PPS has made towards the completion of project milestones and measures. While it is not expected that PPS will have completed all of the project requirements at the Mid-Point Assessment, it is important that the PPS demonstrate progress towards the completion of the project requirements associated with each project, specifically those with a required completion date in DY2. The Independent Assessor will also focus their review on PPS progress towards meeting those project requirements that cross multiple projects including, but not limited to, milestones associated with the attainment of PCMH Level 3 certification and the implementation and use of EHRs to share data and track patient engagement across the PPS.

Based on the PPS efforts on project implementation activities through the end of DY2, Q1, each project will be assigned a risk score on a scale of 1 - 5 as outlined in the table below.

PPS Project Risk Score
1 = Project on Track: This the lowest risk score indicating the project is more than likely to meet intended goals.
2 = Project is Very Likely to be on Track: This is a low risk score indicating the project is more than likely to meet intended goals but has minor challenges to be overcome.
3 = Project is Likely to be on Track: This is a moderate risk score indicating the project could meet intended goals but requires some performance improvements and overcoming challenges.
4 = Project is at risk to be on Track: This is a high risk score indicating the project may fail to meet intended goals without significant modifications or performance improvements.
5 = Project is Off Track: This is the highest risk score indicating that the project will more than likely fail to meet the intended goals, even with significant modifications or performance improvements.

The Independent Assessor will use the combination of the overall DSRIP Project Plan compliance assessment ranking and the individual PPS Project Risk Scores to inform the recommendations for the Mid-Point Assessment.

Independent Assessor Recommendations
The Independent Assessor will release initial recommendations to each PPS who will have a one week advanced opportunity to review and respond to recommendations prior to release to the public. Recommendations will subsequently be released to the public for an initial public comment period consistent with the timeline outlined in Section II. The comment period will allow for public comment as well as for responses from the PPS. The Independent Assessor recommendations may include, but not be limited to,

  • Continuation of PPS Project Plan;
  • Modifications to the approved project plans, such as a change in a project or projects selected for implementation by a PPS;
  • Changes to the PPS Lead entity;
  • Consolidations of multiple PPS in to a single PPS;
  • Discontinuation of a PPS´ Project Plan;
  • Other remediation or improvements to increase the likelihood of PPS project success
  • Adding new providers to PPS network to assist in meeting PPS performance goals; or
  • Replication of best practices of other PPS.

PPS Recommendations
Once the Independent Assessor has completed its Mid-Point Assessment review and released its recommendations for comment, the PPS will have the opportunity to submit responses to the Independent Assessor's recommendations. In submitting responses to the Independent Assessor's recommendations, the PPS will be able to refute or support the recommendations of the Independent Assessor. The PPS will also be able to propose alternative recommendations to those made by the Independent Assessor or new recommendations that were not included in the initial recommendations made by the Independent Assessor. The PPS can request a meeting with the Independent Assessor. The request must be made within two weeks of receipt of the recommendations. The meeting will provide a direct dialogue with the Independent Assessor to discuss the recommendations.

Recommendations from the PPS will be reviewed by the Independent Assessor and considered in the development of the final recommendations to be submitted to DOH. The PPS will receive a standard format to be used in submitting recommendations for the Independent Assessor´s consideration.

Independent Assessor Recommendations to DOH
The Independent Assessor will submit a set of final recommendations to DOH based on the initial recommendations made by the Independent Assessor, the public comments received through the 30 day comment period, and the recommendations submitted by the PPS. DOH will review all recommendations submitted by the Independent Assessor and put forth a final set of recommendations for review by the Project Approval and Oversight Panel.

Recommendations submitted to the Department will be made available to the public for a 30 day comment period in advance of review of the recommendations by the Project Approval and Oversight Panel.

Recommendations to the Project Approval and Oversight Panel
The Project Approval and Oversight Panel (Panel) will convene to review the recommendations put for the by the Independent Assessor for the Mid-Point Assessment. The Panel will focus on a review of the recommendations to modify, consolidate, or discontinue DSRIP Project Plans and the Panel will have the ability to Accept, Accept with Modifications, or Reject the recommendations made by the Independent Assessor.

In addition to the review of the Mid-Point Assessment recommendations, the Panel meetings will provide an opportunity to highlight PPS success through the first five quarters of DSRIP, as identified through the Mid-Point Assessment.

Project Approval and Oversight Panel Recommendations to the Commissioner of Health
Following the conclusion of the Panel meetings to review the recommendations set forth through DOH, the final recommendations will be compiled and presented to the Commissioner of Health. PPS will have the opportunity to submit letters in support of the recommendations from the Panel or to request the reconsideration of the Panel recommendations by the Commissioner, consistent with the process for the DSRIP Project Plan Application scoring. As identified in the proposed timelines in Section II of this document, it is anticipated that the Commissioner will be receiving the recommendations from the Panel no later than one week following the conclusion of the Panel meetings. PPS wishing to submit a formal request for reconsideration to the Commissioner must do so no later than five days after the close of the Panel meetings to ensure their request is considered by the Commissioner.

Commissioner of Health Recommendations to CMS
Consistent with the approach implemented for the DSRIP Project Plan Application scores, the final recommendations from the Commissioner of Health will be submitted to CMS for final approval. In submitting the recommendations to CMS for approval, the Commissioner will prioritize those recommendations that are more transformative in nature, i.e. consolidation of PPS or discontinuation of a DSRIP Project Plan, to ensure sufficient time is provided for CMS review and for PPS implementation of the changes.

IV. PPS Lead Financial Stability Test

While the Independent Assessor will be using data from existing PPS reports and other readily available data sources to minimize additional reporting burdens on the PPS, it will be necessary for the PPS to complete the PPS Lead Financial Stability Test as part of the Mid-Point Assessment. The PPS Lead Financial Stability Test will be administered consistent with the approach utilized for the DSRIP Project Plan Application.

  • For PPS that are led by a NewCo, it is expected that in lieu of available financial data for the NewCo, the Financial Stability Test will be completed by the organizations that have come together to form the NewCo The Financial Stability Test will need to be completed by all lead organizations for the NewCo, however only one of those organizations must pass the test for the NewCo to have passed the test.
  • For PPS that are led by multiple entities but not under a NewCo structure, each of the lead entities will be responsible for completing the Financial Stability Test, however only one of the lead entities must pass the test for the lead to continue in its role.
  • For PPS led by a public entity, the lead entity can submit the necessary documentation to complete the Financial Stability Test or the lead entity can submit documentation from the governmental entity of which the lead entity is a part of. The letter of backing from the governmental entity will be sufficient for the PPS Lead to have passed the test and continue in its role as the lead entity.

In the event the PPS Lead is unable to pass the Financial Stability Test, the Independent Assessor can recommend that the PPS identify a new lead entity that will pass the Financial Stability Test. If the PPS Lead received a commitment of financial backing from another entity as part of the Financial Stability Test for the DSRIP project Plan Application, the Independent Assessor can recommend that the entity that committed to financially backing the PPS Lead assume PPS Lead responsibilities.

V. Provider Network Modifications at Mid-Point Assessment

Removal of Network Partners
In addition to the recommendations submitted by the PPS for the DSRIP Project Plan, the PPS can also include requests for modifications to the PPS network, including the removal of network partners. The modifications to the PPS network may include both additions of partners to the network or the removal of network partners.

For the addition of new partners to the PPS network, the PPS will be required to follow a process similar to the process utilized for the networking reopening that occurred in November 2015.

PPS will receive further guidance on the dates for the network reopening for the addition of new partners with the start of the Mid-Point Assessment.

A PPS wishing to remove network partners must do so in accordance with Section X of Attachment I to the STCs. PPS must demonstrate non-performance or failure of the network partner to meet DSRIP program requirements to support the request for the removal of the network partner. Documentation must include a description of the escalation process employed by the PPS for identifying and tracking non-performing network partners as well as the notifications to the under-performing network partners informing them of the performance concerns of the PPS. In requesting the removal of PPS network partners, the PPS will not be able to remove more than 10% of the total number of providers in its network.

PPS may also accept requests from network partners for the 'voluntary removal' of those partners from the PPS network. The PPS will need to receive a formal notification from that network partner detailing their request to be removed from the PPS network. Any network partners that are removed from the PPS network as a result of that network partner's request will be counted towards the cap of 10% for the PPS.

Safety Net Status of Providers
At the Mid-Point Assessment, PPS will have the ability to request a review of the Safety Net status of any network partners that are not currently designed at Safety Net providers. The review process will follow the process previously used during the Safety Net appeals at the outset of DSRIP. PPS will be responsible for submitting the applicable Safety Net Appeals Form for all providers that it wishes to have their Safety Net status reviewed.

Determinations on the Safety Net status of providers will be made consistent with the process utilized in the completion of initial Safety Net status determinations and in the initial round of Safety Net Appeals. The determinations will be based on the most recent data available and are subject to CMS approval. As with the initial Safety Net determinations and the Safety Net Appeals, only Medicaid providers will be considered for Safety Net status

VI. Implementation of Mid-Point Assessment Project Plan Modifications

PPS will have 30 days from the time the final recommendations are submitted from the Commissioner of Health to develop a Mid-Point Assessment Action Plan (Action Plan) and timeline detailing how the PPS will implement the recommended changes to the DSRIP Project Plans. The Action Plan and timeline will be submitted to the Independent Assessor for their review and approval. It is expected that the Action Plan addressing all recommended DSRIP Project Plan modifications is implemented by the start of DY3 to position the PPS for success as funding shifts from Domain 1 infrastructure and Domain 2 and 3 pay for reporting (P4R) to pay for performance (P4P).

The implementation of the Action Plan will become a part of the PPS Quarterly Report submission until all modifications have been completed. The Independent Assessor will monitor the PPS Action Plan efforts for the required modifications to the DSRIP Project Plans to ensure that the PPS is making progress towards the required modifications on a quarterly basis. The expectation is that all DSRIP Project Plan modification items addressed in the Action Plan should be completed by DY3, Q2 (September 30, 2017). However, PPS may propose timelines beyond DY3, Q2 as part of their Action Plan submission for IA review and approval if the PPS believes the modifications requires more time. It is at the discretion of the Independent Assessor to allow an extension of modifications beyond DY3, Q2. If there is a failure to adhere to the Action Plan there will be a loss of an AV for the applicable Organizational or Project section. For example, a failure by to PPS to implement the Action Plan requiring modification of a PPS governance structure by the timelines agreed to in the Action Plan may result in the loss of the AV by the timelines may result in the loss an AV for the governance organization section.

In the event the Independent Assessor determines that a PPS hasn´t modified the project plan as required, the Independent Assessor may submit further recommendations to the Project Approval and Oversight Panel. These recommendations may include:

  • Suspension of the PPS project and associated funding; or
  • Termination of the PPS project plan and funding