You are Here: Home Page > NY 1115 Medicaid Waiver > MRT Plan Current STCs - April 16, 2020

MRT Plan Current STCs - April 16, 2020

Plan is also available in Portable Document Format (PDF)

CENTERS FOR MEDICARE & MEDICAID SERVICES
WAIVER AUTHORITIES

NUMBER: 11-W-00114/2

TITLE: Medicaid Redesign Team

AWARDEE: New York State Department of Health

All requirements of the Medicaid program expressed in law, regulation, and policy statement, not expressly waived in this list, shall apply to the demonstration.

The following waivers shall enable New York to implement the approved Special Terms and Conditions (STC) for the New York Medicaid Redesign Team section 1115 demonstration (formerly the New York Partnership Plan) beginning December 7, 2016 and ending March 31, 2021.

1. Statewideness Section 1902(a)(1)

To permit New York to geographically phase in the Managed Long Term Care (MLTC) program and the Health and Recovery Plans (HARP) and to phase in Behavioral Health (BH) Home and Community Based Services (HCBS) into HIV Special Needs Plans (HIV SNP).

2. Comparability Section 1902(a)(10), section 1902(a)(17)

To enable New York to apply a more liberal income standard for individuals who are deinstitutionalized and receive HCBS through the managed long-term care program than for other individuals receiving community-based long-term care.
To the extent necessary to permit New York to waive cost sharing for non-drug benefit cost sharing imposed under the Medicaid state plan for beneficiaries enrolled in the Mainstream Medicaid Managed Care Plan (MMMC) - including Health and Recovery Plans (HARP) and HIV SNPs - and who are not otherwise exempt from cost sharing in §447.56(a)(1).
Family of One Non-1915 Children, or "Fo1 Children" - To allow the state to target eligibility to, and impose a participation capacity limit on, medically needy children under age 21 who are otherwise described in 42 CFR §435.308 of the regulations who: 1) receive Health Home Comprehensive Care Management under the state plan in replacement of the case management services such individuals formerly received through participation in New York´s NY #.4125 1915(c) waiver and who no longer participate in such waiver due to the elimination of the case management services, but who continue to meet the targeting criteria, risk factors, and clinical eligibility standard for such waiver; and 2) receive HCBS 1915(c) services who meet the risk factors, targeting criteria, and clinical eligibility standard for the above-identified 1915(c) waiver. Individuals who meet either targeting classification will have excluded from their financial eligibility determination the income and resources of third parties whose income and resources could otherwise be deemed available under 42 CFR §435.602(a)(2)(i). Such individuals will also have their income and resources compared to the medically needy income level (MNIL) and resource standard for a single individual, as described in New York´s state Medicaid plan.

3. Amount, Duration & Scope Section 1902(a)(10)(B)

To enable New York to provide behavioral health (BH) HCBS services, whether furnished as a state plan benefit or as a demonstration benefit to targeted populations that may not be consistent with the targeting authorized under the approved state plan, in amount, duration and scope that exceeds those available to eligible individuals not in those targeted populations.

4. Freedom of Choice Section 1902(a)(23)(A)

To the extent necessary to enable New York to require beneficiaries to enroll in managed care plans, including the Mainstream Medicaid Managed Care (MMMC), and MLTC (excluding individuals designated as "Long-Term Nursing Home Stays") and HARPs programs in order to obtain benefits offered by those plans. Beneficiaries shall retain freedom of choice of family planning providers.

5. Reasonable Promptness Section 1902(a)(8)

To enable the state to limit the number of medically needy Fo1 Children not otherwise enrolled in the Children´s 1915(c) waiver.

Title XIX Requirements Not Applicable to Self-Direction Pilot Program
(see Expenditure Authority 8, "Self-Direction Pilot")

6. Direct Payment to Providers Section 1902(a)(32)

To the extent necessary to permit the state to make payments to beneficiaries enrolled in the Self Direction Pilot Program to the extent that such funds are used to obtain self-directed HCBS LTC services and supports.

CENTERS FOR MEDICARE & MEDICAID SERVICES
WAIVER AUTHORITIES

NUMBER: 11-W-00114/2

TITLE: Medicaid Redesign Team

AWARDEE: New York State Department of Health

Under the authority of section 1115(a)(2) of the Social Security Act ("the Act"), expenditures made by New York for the items identified below, which are not otherwise included as expenditures under section 1903 of the Act shall, until the ending date specified for each authority as listed below, be regarded as expenditures under the state´s title XIX plan. These expenditure authorities shall be effective from December 7, 2016 through March 31, 2021, except as otherwise noted.

The following expenditure authorities shall enable New York to implement the approved Special Terms and Conditions (STC) for the New York Medicaid Redesign Team Medicaid Section 1115 demonstration.

Demonstration-Eligible Populations. Expenditures for healthcare related costs for the following populations that are not otherwise eligible under the Medicaid state plan.
1. Demonstration Population 2 (TANF Adult). Temporary Assistance for Needy Families (TANF) Recipients. Expenditures for health care related costs for low- income adults enrolled in TANF. These individuals are exempt from receiving a MAGI determination in accordance with §1902(e)(14)(D)(i)(I) of the Act.
2. Demonstration Population 9 (HCBS Expansion). Individuals who are not otherwise eligible, are receiving HCBS, and who are determined to be medically needy based on New York´s medically needy income level, after application of community spouse and spousal impoverishment eligibility and post-eligibility rules consistent with section 1924 of the Act.
3. Demonstration Population 10 (Institution to Community). Expenditures for health care related costs for individuals moved from institutional nursing facility settings to community settings for long-term services and supports who would not otherwise be eligible based on income, but whose income does not exceed the income standard described in STC 4(c) of section IV, and who receive services through the managed long-term care program under the demonstration.
4. Included in Demonstration Population 12 (Fo1 Children)- Medically needy children Fo1 Demonstration children under age 21 with a waiver of 1902(a)(10)(C)(i)(III) who meet the targeting criteria, risk factors, and clinical eligibility standard for #NY.4125 waiver including ICF, NF, or Hospital Level of Care (LOC) who are not otherwise enrolled in the Children´s 1915(c).
Twelve-Month Continuous Eligibility Period. Expenditures for health care related costs for individuals who have been determined eligible under groups specified in Table 1 of STC 3 in Section IV for continued benefits during any periods within a twelve-month eligibility period when these individuals would be found ineligible if subject to redetermination. This authority includes providing continuous coverage for the Adult Group determined financially eligible using Modified Adjusted Gross Income (MAGI) based eligibility methods. For expenditures related to the Adult Group, specifically, the state shall make a downward adjustment of 2.6 percent in claimed expenditures for federal matching at the enhanced federal matching rate and will instead claim those expenditures at the regular matching rate.
Facilitated Enrollment Services. Expenditures for enrollment assistance services provided by managed care organizations (MCO), the costs for which are included in the claimed MCO capitation rates.
Demonstration Services for Behavioral Health Provided under Mainstream Medicaid Managed Care (MMMC). Expenditures for provision of residential addiction services, crisis intervention and licensed behavioral health practitioner services to MMMC enrollees only and are not provided under the state plan [Demonstration Services 9].
Targeted Behavioral Health (BH) HCBS Services. Expenditures for the provision of BH HCBS services under Health and Recovery Plans (HARP) and HIV Special Needs Plans (SNP) that are not otherwise available under the approved state plan [Demonstration Services 8].
Designated State Health Programs Funding. Expenditures for the designated state health program specified in STC 15 in Section VII of the STCs, not to exceed $2 billion in FFP through March 31, 2020 [Demonstration Services 10]. This authority expires March 31, 2020.
Delivery System Reform Incentive Payment (DSRIP) Program. Expenditures for incentive payments and planning grant payments for the DSRIP program specified in Section VII of the STCs, not to exceed $8 billion of FFP from April 14, 2014 through March 31, 2020 [Demonstration Services 11]. This authority expires March 31, 2020
Self-Direction Pilot. Expenditures to allow the state to make self-direction services available to HARP and HIV/SNP enrollees receiving BH HCBS or children meeting targeting criteria for the Children´s 1915(c) Waiver and in MMMC receiving HCBS under the Children´s Waiver. The program will be in effect from January 1, 2017 through March 31, 2021 [Demonstration Services 8].

CENTERS FOR MEDICARE & MEDICAID SERVICES
WAIVER AUTHORITIES

NUMBER: 11-W-00114/2

TITLE: Medicaid Redesign Team

AWARDEE: New York State Department of Health

I. PREFACE

The following are the Special Terms and Conditions (STCs) for the New York Medicaid Redesign Team section 1115(a) Medicaid demonstration (hereinafter "demonstration" or "MRT") to enable the New York State Department Office of Health (hereinafter "state" or "DOH") to operate this demonstration. The Centers for Medicare & Medicaid Services (CMS) has granted the state waivers of requirements under section 1902(a) of the Social Security Act (hereinafter "the Act") and expenditure authorities authorizing federal matching of demonstration costs that are not otherwise matchable and which are separately enumerated. These STCs set forth in detail the nature, character, and extent of federal involvement in the Demonstration and New York´s obligations to CMS related to this demonstration. The MRT demonstration will be statewide and is approved from December 7, 2016 through March 31, 2021.

The STCs have been arranged into the following sections:

Preface
Program Description and Objectives
General Program Requirements
Populations Affected by and Eligible Under the Demonstration
Demonstration Benefits and Enrollment
Delivery Systems
Delivery System Reform Program Description and Objectives
General Reporting Requirements
General Financial Requirements
Monitoring Budget Neutrality
Evaluation of the Demonstration
Schedule of Deliverables for the Demonstration

The STCs also include the following attachments:

Mainstream Medicaid Managed Care (including HIV SNP and HARP) Benefits
Managed Long-Term Care Benefits
Home and Community-Based Services Expansion Program Benefits
Behavioral Health (BH) Home and Community Based Services (HCBS) in HARPS and HIV SNPs
Quarterly Operational Report Format
Self-Directed Care Pilot
Mandatory Managed Long-Term Care/Care Coordination Model (CCM)
HARP Evaluation Plan
DSRIP Program Funding and Mechanics Protocol
DSRIP Strategies Menu and Metrics
DSRIP Operational Protocol
DSHP Claiming Protocol
Final Evaluation Design and Final Evaluation Plan
Behavioral Health HCBS services offered by HARPs and HIV SNPs and Individual Directed Goods and Services
Design Evaluation Questions

Additionally, attachments have been included to provide supplementary information and guidance for specific STCs.

|top of section| |top of page|

II. PROGRAM DESCRIPTION AND OBJECTIVES

The state´s goal in implementing the Medicaid Redesign Team Section 1115(a) demonstration is to improve access to health services and outcomes for low-income New Yorkers by:

Improving access to health care for the Medicaid population;
Improving the quality of health services delivered; and
Expanding coverage with resources generated through managed care efficiencies to additional low-income New Yorkers.

The demonstration is designed to permit New York to use a managed care delivery system to deliver benefits to Medicaid recipients, create efficiencies in the Medicaid program, and enable the extension of coverage to certain individuals who need long-term care and supports. It was originally approved in 1997 to enroll most Medicaid recipients into managed care organizations (MCO) (Medicaid managed care program). As part of the demonstration´s renewal in 2006, authority to require some disabled and aged populations to enroll in mandatory managed care was transferred to a new demonstration, the Federal-State Health Reform Partnership (F-SHRP). Effective April 1, 2014, this authority was restored to this demonstration as F-SHRP was phased out.

In 2001 the Family Health Plus (FHPlus) program was implemented as an amendment to the demonstration, providing comprehensive health coverage to low-income uninsured adults, with and without dependent children, who have income greater than Medicaid state plan eligibility standards. FHPlus was further amended in 2007 to implement an employer sponsored health insurance (ESHI) component. Individuals eligible for FHPlus who have access to cost-effective ESHI are required to enroll in that coverage, with FHPlus providing any wrap-around services necessary to ensure that enrollees get all FHPlus benefits. FHPlus expired on December 31, 2013 and became a state-only program, but federal matching funding for state expenditures for FHPlus will continue to be available as a designated state health program through December 31, 2014.

In 2002 the demonstration was expanded to incorporate a family planning benefit under which family planning and family planning related services were provided to women losing Medicaid eligibility and to certain other adults of childbearing age (family planning expansion program). The family planning expansion program expired on December 31, 2013 and became a state plan benefit.

In 2010 the Home and Community Based Services Expansion program (HCBS Expansion program) was added to the demonstration. It covers cost-effective home and community-based services to certain adults with significant medical needs as an alternative to institutional care in a nursing facility. The benefits and program structure mirrors those of existing section 1915(c) waiver programs and aims to cover quality services for individuals in the community, ensure the well-being and safety of the participants and increase opportunities for self-advocacy and self- reliance.

As part of the 2011 extension, the state was authorized to develop and implement two new initiatives designed to improve the quality of care rendered to Partnership Plan recipients. The first, the Hospital-Medical Home (H-MH) project, provided funding and performance incentives to hospital teaching programs in order to improve the coordination, continuity and quality of care for individuals receiving primary care in outpatient hospital settings and facilitate certification of such programs by the National Committee for Quality Assurance as patient-centered medical homes. This demonstration initiative ended on December 31, 2014.

Under the second 2011 initiative, the state would have provided funding, on a competitive basis, to hospitals and/or collaborations or hospitals and other providers for the purpose of developing and implementing strategies to reduce the rate of Potentially Preventable Readmissions for the Medicaid population. The demonstration initiative was never implemented.

Finally, in 2011 CMS began providing matching funding for the state´s program to address clinic uncompensated care through its Indigent Care Pool (ICP). This pool expired on December 31, 2014.

In 2012, New York added to the demonstration an initiative to improve service delivery and coordination of long-term care services and supports for individuals through a managed care model. Under the Managed Long Term Care (MLTC) program, eligible individuals in need of more than 120 days of community-based long-term care are enrolled with managed care providers to receive long-term services and supports as well as other ancillary services. Other covered services are available on a fee-for-service basis to the extent that New York has not exercised its option to include the individual in the Mainstream Medicaid Managed Care Program (MMMC). Enrollment in MLTC was phased in geographically and by group.

The state´s goal specific to MLTC are listed below:

Expanding access to managed long-term care for Medicaid enrollees who are in need of long-term services and supports (LTSS)
Improving patient safety and quality of care for enrollees in MLTC plans
Reducing preventable inpatient and nursing home admissions
Improving satisfaction, safety and quality of life

In April 2013, New York had three amendments approved. The first amendment was a continuation of the state´s goal for transitioning more Medicaid beneficiaries into managed care. Under this amendment, the Long Term Home Health Care Program (LTHHCP) participants began transitioning, on a geographic basis, from New York´s 1915(c) waiver into the 1115 demonstration and into managed care. Second, this amendment eliminated the exclusion from MMMC of both foster care children placed by local social service agencies and individuals participating in the Medicaid buy-in program for the working disabled.

Additionally, the April 2013 amendment approved expenditure authority for New York to claim FFP for expenditures made for certain designated state health programs (DSHP) beginning April 1, 2013 through March 31, 2014. These DSHPs were aimed to improve health outcomes for Medicaid and other low-income individuals, and the federal funding was linked to requirements for the state to submit deliverables to demonstrate successful efforts to transform its health system for individuals with developmental disabilities.

A December 2013 amendment was approved to ensure that the demonstration made changes that were necessary in order to coordinate its programs with the Medicaid expansion and other changes made under the Affordable Care Act (ACA) implementation beginning January 1, 2014.

Effective April 1, 2014, CMS approved an amendment to extend several authorities that expired in calendar year 2014. As part of the amendment CMS extended authorities related to the transitioning of parents into state plan coverage and other authorities that provide administrative ease to the state´s programs and continuing to provide services to vulnerable populations, i.e. HCBS Expansion program and individuals moved from institutional settings into community-based settings.

Also, effective April 1, 2014, populations receiving managed care or managed long-term care in the 14 counties that encompassed the Federal-State Health Reform Partnership (F-SHRP) demonstration were moved into this demonstration.

An amendment approved on April 14, 2014 allowed New York to take the first steps toward a major delivery system reform through a Delivery System Reform Incentive Payment (DSRIP) program. This amendment to the Partnership Plan demonstration provided for an Interim Access Assurance Fund (IAAF) to ensure that sufficient numbers and types of providers were available in the community to participate in the transformation activities contemplated by the DSRIP Program. The DSRIP program incentivized providers through additional payments beginning in 2015. The amendment also included expenditure authority for DSHPs to allow the state to concentrate resources on the investments necessary to implement its DSRIP program. Savings from the DSRIP program were anticipated to exceed the cost of the DSHP program.

On December 31, 2014, CMS amended the demonstration to enable New York to extend long-term nursing facility services to enrollees of New York´s MMMC and MLTC populations. Enrollment in MMMC and MLTC was extended to individuals entering residential health care facilities (RHCF) for stays that are classified as permanent. As part of the agreement, the state also instituted an independent long-term services and support (LTSS) assessment process via an enrollment broker and implemented its Independent Consumer Support Program in areas of the state where services and enrollment were being instituted.

In August 2015 CMS approved New York´s request to implement Health and Recovery Plans (HARP) to integrate physical, behavioral health and BH HCBS for Medicaid enrollees with diagnosed severe mental illness (SMI) and/or substance use disorder (SUD) to receive services in their own homes and communities. Under the demonstration, HARPs are a separate coverage product that is targeted to Medicaid enrollees that meet need-based criteria for SMI and/or SUD established by the state. HIV SNP under MMMC will also offer BH HCBS services to eligible individuals meeting targeting, risk, and functional needs criteria. All MMMC plans will offer BH benefits in integrated plans including four new demonstration services.

The demonstration was also amended to effectuate eligibility flexibilities for the Adult Group, including allowing adults enrolled in TANF to be enrolled as a demonstration population, without a MAGI determination, extension of continuous eligibility for members of the Adult Group who turn 65 during their continuous eligibility period and temporary coverage for members of the Adult Group who are determined eligible to receive coverage through the Marketplace.

On November 30, 2016, CMS approved an extension of the demonstration, but in response to comments by the state, that extension was rescinded and superseded by a modified approval effective December 7, 2016. Under the most recent extension, the Partnership Plan is renamed New York Medicaid Redesign Team (MRT) and will be referred as MRT throughout the STCs. The extension included time limited authorization to extend the DSRIP program first authorized in 2014, through March 31, 2020. The extension also included a new time-limited DSHP authority to the extent that the state increases its Medicaid expenditures through its DSRIP program and achieves metrics that will result in anticipated cost savings that offset the DSHP expenditures. DSHP funding will be phased down over the demonstration period. The DSRIP and DSHP authorities are intended to be a one-time investment in system transformation that can be sustained through ongoing payment mechanisms and/or state and local initiatives.

The Behavioral Health Self-Direction Pilot was included as part of the renewal. This pilot makes self-direction services available to HARP and HIV SNP enrollees receiving BH HCBS. The program is authorized to be in effect from January 1, 2017 through March 31, 2021.

On April 19, 2019, CMS approved an amendment to allow a waiver of comparability which permits managed care enrollees to only be assessed a drug copay. The state will not assess the non-drug benefit cost sharing described in the Medicaid state plan.

On August 2, 2019, CMS approved an amendment containing the following changes:

Allow children with HCBS under the State´s 1915(c) Children´s Waiver and children placed in foster care through a Voluntary Foster Care Agency (VFCA) to enroll in Mainstream Managed Care or an HIV SNP.
Continues Medicaid eligibility for Fo1 Non- 1915 children who would have been eligible under the Children´s Waiver had case management not been moved under the State Plan as a Health Home service or who were in a non-SSI category and receive HCBS or HH comprehensive case management.
Include Children´s Waiver HCBS and State Plan behavioral health services in the Medicaid managed care benefit package.
Include children receiving HCBS under the Children´s waiver in the Self Direction Pilot for Individual Directed Goods and Services.

On December 19, 2019, CMS approved an amendment with the following changes for Partially Capitated MLTC plans:

Implement a lock-in policy for partially capitated MLTC plans, pursuant to which enrollees of partially capitated MLTC plans are able to transfer to another partially capitated plan without cause during the first 90 days of a 12-month period and with good cause during the remainder of the period. A member of a partially capitated MLTC plan may transfer to another type of MLTC plan at any time.
Limit the nursing home benefit in the partially capitated MLTC plan to three months for those enrollees who have been designated as Long-Term Nursing Home Stays (LTNHS) in a skilled nursing or residential health care facility, at which time the individual will be involuntarily disenrolled from the partially capitated MLTC plan and payment for nursing home services will be covered by Medicaid fee for service for individuals that qualify for institutional Medicaid coverage. Consistent with this partially capitated MLTC benefit change, individuals age 21 years of age or older who are dually eligible for Medicare and Medicaid and LTNHS in a nursing home will be excluded from enrollment in a partially capitated MLTC plan.

|top of section| |top of page|

III. GENERAL PROGRAM REQUIREMENTS

Compliance with Federal Non-Discrimination Statutes. The state must comply with all applicable federal statutes relating to non-discrimination. These include, but are not limited to, the Americans with Disabilities Act of 1990, title VI of the Civil Rights Act of 1964, section 504 of the Rehabilitation Act of 1973, the Age Discrimination Act of 1975, and Section 1557 of the Affordable Care Act (Section 1557). Such compliance includes providing reasonable modifications to individuals with disabilities under the ADA, Section 504, and Section 1557 with eligibility and documentation requirements, understanding program rules and notices, and meeting other program requirements necessary to obtain and maintain benefits.
Compliance with Medicaid Law, Regulation and Policy. All requirements of the Medicaid program expressed in law, regulation and policy statement, not expressly waived or identified as not applicable in the waiver and expenditure authority documents (of which these terms and conditions are part), must apply to the demonstration.
Changes in Medicaid Law, Regulation and Policy. The state must, within the timeframes specified in law, regulation or policy statement, come into compliance with any changes in federal law, regulation or policy affecting the Medicaid program that occur during this demonstration approval period, unless the provision being changed is expressly waived or identified as not applicable.
Impact on Demonstration of Changes in Federal Law, Regulation and Policy.
1. To the extent that a change in federal law, regulation or policy requires either a reduction or an increase in federal financial participation (FFP) for expenditures made under this demonstration, the state must adopt, subject to CMS approval, a modified budget neutrality agreement for the demonstration as necessary to comply with such change. The modified agreement will be effective upon the implementation of the change. The trend rates for the budget neutrality agreement are not subject to change under this subparagraph. Further, the state may seek an amendment to the demonstration (as per STC 7 of this section) as a result of the change in FFP.
2. If mandated changes in the federal law require state legislation, the changes must take effect on the day such state legislation becomes effective, or on the last day such legislation was required to be in effect under the law.
State Plan Amendments. The state will not be required to submit title XIX or XXI state plan amendments for changes affecting any populations made eligible solely through the demonstration. If a population eligible through the Medicaid state plan is affected by a change to the demonstration, a conforming amendment to the state plan may be required, except as otherwise noted in these STCs. The state is required to submit new or revised title XIX state plan amendments for state plan services received by demonstration participants except for services provided through waiver or expenditure authority. In all such instances, the provisions of the Medicaid state plan, governs.
Changes Subject to the Amendment Process. Changes related to program design, eligibility, enrollment, expansion of program benefits, sources of non-federal share of funding and budget neutrality must be submitted to CMS as amendments to the demonstration. All amendments require are subject to approval at the discretion of the Secretary in accordance with section 1115 of the Social Security Act (the Act). The state must not implement changes to these elements without prior approval by CMS either through an approved amendment to the Medicaid state plan or an amendment to the demonstration. Amendments to the demonstration are not retroactive, and FFP will not be available for changes to the demonstration that have not been approved through the amendment process outlined in STC 7 of this section except as provided in STC 3.
Amendment Process. Requests to amend the demonstration must be submitted to CMS for approval no later than 120 days prior to the planned date of implementation of the change and may not be implemented until approved. CMS reserves the right to deny or delay approval of a demonstration amendment based on non-compliance with these STCs, including but not limited to failure by the state to submit required elements of a viable amendment request as found in this STC, and failure by the state to submit reports required in the approved STCs and other deliverables in a timely fashion according to the deadlines specified herein. Amendment requests must include, but are not limited to, the following:
1. An explanation of the public process used by the state, consistent with the requirements of STC 16 of this section, to reach a decision regarding the requested amendment;
2. A data analysis which identifies the specific "with waiver" impact of the proposed amendment on the current budget neutrality agreement. Such analysis shall include current total computable "with waiver" and "without waiver" status on both a summary and detailed level through the current approval period using the most recent actual expenditures, as well as summary and detailed projections of the change in the "with waiver" expenditure total as a result of the proposed amendment, which isolates (by Eligibility Group) the impact of the amendment;
3. A detailed description of the amendment, including impact on beneficiaries, with sufficient supporting documentation;
4. If applicable, a description of how the evaluation design will be modified to incorporate the amendment provisions.
5. A draft evaluation design submitted to CMS no later than 120 days after the approval of an amendment.
6. An updated Comprehensive Quality Strategy (CQS) submitted to CMS for approval within 90 days of approval of an amendment.
Extension of the Demonstration.
1. States that intend to request an extension of the demonstration must submit an application to CMS from the Governor or Chief Executive Officer of the state in accordance with the requirements of 42 Code of Federal Regulations (CFR) 431.412(c). States that do not intend to request an extension of the demonstration beyond the period authorized in these STCs, must submit a phase-out plan consistent with the requirements of STC 9.
2. Compliance with Transparency Requirements of 42 CFR §431.412. As part of the demonstration extension requests, the state must provide documentation of compliance with the transparency requirements of 42 CFR §431.412 and the public notice and tribal consultation requirements outlined in STC 16 of this section regarding Public Notice, Tribal Consultation and Consultation with Interested Parties. The financial data described in 42 CFR §431.412(c)(2)(v) must include five years of recent historical expenditure and enrollment data for the Medicaid and demonstration populations that are to be included in the demonstration extension, and a proposed budget neutrality test for the extension period based on recent data.
Demonstration Phase-Out. The state may suspend or terminate this demonstration in whole, or in part, consistent with the following requirements:
1. Notification of Suspension or Termination: The state must promptly notify CMS in writing of the reason(s) for the suspension or termination, together with the effective date and phase-out plan. The state must submit its notification letter and a draft phase-out plan to CMS no less than six months before the effective date of the demonstration´s suspension or termination. Prior to submitting the draft transition and phase-out plan to CMS, the state must publish on its website the draft transition and phase-out plan for 30-day public comment period. In addition, the state must conduct tribal consultation in accordance with its approved tribal consultation state plan amendment and in accordance with STC 16, if applicable. Once the 30-day public comment period has ended, the state must provide a summary of each public comment received, the state´s response to the comment, and the way the state incorporated the received comment into a revised transition and phase-out plan.
2. Transition and Phase-Out Plan Requirements: The state must include, at a minimum, in its transition and phase out plan its process by which it will notify affected beneficiaries, the content of said notices (including information on the beneficiary´s appeal rights), the process by which the state will conduct administrative reviews of Medicaid eligibility prior to the termination of the demonstration for the affected beneficiaries, ensure ongoing coverage for those beneficiaries whether currently enrolled or determined to be eligible individuals, including community resources that are available.
3. Transition and Phase-Out Plan Approval: The state must obtain CMS approval of the transition and phase-out plan prior to the implementation of transition and phase- out activities. Implementation of transition and phase-out activities must be no sooner than 14 days after CMS approval of the transition and phase-out plan.
4. Transition and Phase-Out Procedures: The state must comply with all notice requirements found in 42 CFR, part E, including CFR §431.206, §431. 210, §431.211, and §431.213. In addition, the state must ensure all appeal and hearing rights afforded to demonstration participants as outlined in 42 CFR § 431.220 and §431.221. If a demonstration participant requests a hearing before the date of action, the state must maintain benefits as required in 42 CFR § 431.230. In addition, the state must conduct administrative renewals for all affected beneficiaries in order to determine whether they qualify for Medicaid eligibility under a different eligibility category as discussed in the October 1, 2011 State Health Official Letter #10-008 and as required under 42 C.F.R. 435.916(f)(1). For individuals determined ineligible for Medicaid, the state must determine potential eligibility for other insurance affordability programs and comply with the procedures set forth in 42 CFR 435.1200(e).
5. Exemption from Public Notice Procedures, 42 CFR Section 431.416(g). CMS may expedite the federal and state public notice requirements under circumstances described in 42 CFR 431.416(g).
6. Enrollment Limitation during Demonstration Phase-Out. If the state elects to suspend, terminate, or not extend this demonstration, during the last six months of the demonstration, enrollment of new individuals into the demonstration must be suspended. The limitation of enrollment into the demonstration does not impact the state´s obligation to determine Medicaid eligibility in accordance with the approved Medicaid´s state plan.
7. Federal Financial Participation (FFP): If the project is terminated or any relevant waivers suspended by the state, FFP shall be limited to normal closeout costs associated with terminating the demonstration including services and administrative costs of disenrolling participants.
Expiring Demonstration Authority. For any waiver or expenditure authority that expires prior to the demonstration´s expiration date, the state must submit a demonstration authority expiration plan to CMS no later than six months prior to the applicable demonstration authority´s expiration date, consistent with the following requirements:
1. Expiration Requirements. The state must include, at a minimum, in its demonstration authority expiration plan the process by which it will notify affected beneficiaries, the content of said notices (including information on the beneficiary´s appeal rights), the process by which the state will conduct administrative reviews of Medicaid eligibility prior to the termination of the demonstration authority for the affected beneficiaries, and ensure ongoing coverage for eligible beneficiaries, as well as any community outreach activities.
2. Expiration Procedures. The state must comply with all applicable notice requirements found in 42 CFR, part 431 subpart E, including sections 431.206, 431.210, 431.211, and 431.213. In addition, the state must assure all applicable appeal and hearing rights are afforded to demonstration beneficiaries as outlined in 42 CFR, part 431 subpart E, including sections 431.220 and 431.221. If a demonstration beneficiary requests a hearing before the date of action, the state must maintain benefits as required in 42 CFR 431.230. In addition, the state must conduct administrative renewals for all affected beneficiaries in order to determine if they qualify for Medicaid eligibility under a different eligibility category prior to termination as discussed in October 1, 2010, State Health Official Letter #10-008 and as required under 42 C.F.R. 435.916(f)(1). For individuals determined ineligible for Medicaid, the state must determine potential eligibility for other insurance affordability programs and comply with the procedures set forth in 42 CFR 435.1200(e).
3. Federal Public Notice. CMS will conduct a 30-day federal public comment period consistent with the process outlined in 42 CFR 431.416 in order to solicit public input on the state´s demonstration authority expiration plan. CMS will consider comments received during the 30-day period during its review and approval of the state´s demonstration authority expiration plan. The state must obtain CMS approval of the demonstration authority expiration plan prior to the implementation of the expiration activities. Implementation of expiration activities must be no sooner than fourteen (14) days after CMS approval of the demonstration authority expiration plan.
4. Federal Financial Participation (FFP). FFP will be limited to normal closeout costs associated with the expiration of the demonstration authority including services, continued benefits as a result of beneficiaries´ appeals, and administrative costs of disenrolling beneficiaries.
Medicaid Authorities Transition. During the demonstration period, the state must evaluate which portions of the demonstration could be transitioned to 1915(c) and 1915(i) authorities. This analysis will be conducted as follows:
1. At the time of any proposed amendment to this demonstration as described in STC #7 - the state will provide a "1915(c)/(i) Authorities" analysis, consistent with this STC´s purpose, and include as a section in the state´s amendment application; and
2. September 2019 through September 2020 - CMS and the state will conduct joint transition planning activities in order to identify which portions can be transferred out of this demonstration.
  1. In lieu of the "joint transition planning activities" outlined above in this STC, the state may also seek CMS concurrence for an attestation that its previous analysis was inclusive of all potential 1915(c) and 1915(i) authorities under this demonstration.
    1. If the state seeks concurrence for an attestation, it must be submitted to CMS in the form of a memorandum by September 30, 2019 and CMS will work towards approval, or request additional information (RAI), within 90 days of state submission.
3. September 2020 through March 2021 - If the state does not seek and receive CMS concurrence as described in (b)(i) above, it must begin developing for submission 1915(c) and 1915(i) authorities for the portions to be transitioned out of this demonstration
CMS Right to Terminate or Suspend. CMS may suspend or terminate the demonstration, subject to adequate public notice, (in whole or in part) at any time before the date of expiration, whenever it determines following a hearing that the state has materially failed to comply with the terms of the project. CMS will promptly notify the state in writing of the determination and the reasons for the suspension or termination, together with the effective date.
Finding of Non-Compliance. The state does not relinquish its rights to challenge CMS findings that the state materially failed to comply.
Withdrawal of Waiver or Expenditure Authority. CMS reserves the right to withdraw waivers and/or expenditure authorities at any time it determines that continuing the waivers or expenditure authorities would no longer be in the public interest or promote the objectives of title XIX or title XXI. CMS will promptly notify the state in writing of the determination and the reasons for the withdrawal, together with the effective date and afford the state an opportunity to request a hearing to challenge CMS´ determination prior to the effective date. If a waiver or expenditure authority is withdrawn, FFP is limited to normal closeout costs associated with terminating the waiver of expenditure authority, including services and administrative costs of disenrolling participants.
Adequacy of Infrastructure. The state must ensure the availability of adequate resources for implementation and monitoring of the demonstration, including education, outreach and enrollment; maintaining eligibility systems; compliance with cost sharing requirements; monitoring and oversight of managed care plans providing long-term services and supports and HCBS, including quality and enrollment processes; and reporting on financial and other demonstration components.
Public Notice, Tribal Consultation and Consultation with Interested Parties. The state must comply with the state notice procedures as required in 42 CFR 431.408 prior to submitting an application to extend the demonstration. For applications to amend the demonstration, the state must comply with the state notice procedures set forth in 59 Fed. Reg. 49249 (September 27, 1994) prior to submitting such request.

The state must also comply with tribal and Indian Health Program/Urban Indian Health Organization consultation requirements at section 1902(a) (73) of the Act, 42 CFR 431.408(b), State Medicaid Director Letter #01-024, or as contained in the state´s approved Medicaid State Plan, when any program changes to the demonstration, either through amendment as set out in STC 7 or extension, are proposed by the state.

The state must also comply with the Public Notice Procedures set forth in 42 CFR 447.205 for changes in statewide methods and standards for setting payment rates.
Federal Financial Participation (FFP). No federal matching for expenditures, both administrative and service, for this demonstration will take effect until the effective date identified in the demonstration approval letter, or if later, as expressly stated within these STCs.
Transformed Medicaid Statistical Information Systems (T-MSIS) Requirements. The state shall comply with all data reporting requirements under Section 1903(r) of the Act, including but not limited to Transformed Medicaid Statistical Information Systems Requirements. More information on T-MSIS is available in the August 23, 2013 State Medicaid Director Letter.

Should the MMIS fail to maintain and produce all federally required program management data and information, including the required T-MSIS, eligibility, provider, and managed care encounter data, in accordance with requirements in the State Medicaid Manual Part II, FFP may be suspended or disallowed as provided for in federal regulations at 42 CFR §433 Subpart C, and 45 CFR Part 95.
Protection against Duplication. The state must have processes in place to ensure that there is no duplication of federal funding for any aspect of the demonstration.

|top of section| |top of page|

IV. POPULATIONS AFFECTED BY AND ELIGIBILITY UNDER THE DEMONSTRATION

Eligible under the Medicaid State Plan (State Plan Eligibles). Mandatory and optional Medicaid state plan populations derive their eligibility through the Medicaid state plan and are subject to all applicable Medicaid laws and regulations in accordance with the Medicaid state plan, except as expressly waived and as further described in these STCs. Should the state amend the state plan to make any changes to eligibility for Medicaid mandatory populations, upon submission of the state plan amendment, the state must notify CMS in writing of the pending state plan amendment. The Eligibility Groups (EG) listed in the Reporting and the Budget Neutrality sections of the STCs will be updated upon approval of changes to State plan eligibility and will be considered a technical change to the STCs.
Individuals Not Otherwise Eligible under the Medicaid State Plan. Beneficiary eligibility groups who are made eligible for the demonstration by virtue of the expenditure authorities expressly granted in this demonstration are subject to Medicaid laws or regulations, except for those identified as non-applicable in the expenditure authorities for this document. Eligibility criteria are described elsewhere in this section. Individuals made eligible under this demonstration by virtue of the expenditure authorities expressly granted include:
1. individuals in the HCBS Expansion program;
2. individuals moved from Institutional Settings to Community Settings and receiving MLTC but who would have excess income or resources under the state plan;
3. adults who are receiving TANF benefits and have not been determined eligible using MAGI-based methods; and
4. Individuals previously eligible in the new adult group who are no longer eligible in that group but are still within a 12-month continuous eligibility period.
5. children under age 21 who are medically needy (both SSI-related and non-SSI related) and have parental income and resources (if applicable) waived and otherwise meet eligibility criteria for 1915(c) waiver #.4125 as Fo1 Demonstration children.
6. so are no longer eligible under the Children´s waiver. Note: Unlike the Fo1 Children´s (Demonstration Population 12) expenditures authorized under section 1115(a)(2) in these STCs, additional Family of One Children (SSI-related) that receive their HCBS under the state´s Children´s 1915(c) Waiver do not require this demonstration´s expenditure authority.
Program Components. The Medicaid Redesign demonstration includes two distinct components-Mainstream Medicaid Managed Care (MMMC) and Managed Long Term Care (MLTC) -each of which affects different populations, some of which are eligible under the state plan and some of which are eligible only as an expansion population under the demonstration. In addition, subsets of MMMC and MLTC are eligible for additional benefits. Table 1 summarizes the Medicaid state plan populations that are affected by the demonstration. In addition, the following expansion populations must participate in MLTC: Demonstration Population 9 (HCBS Expansion) and Demonstration Population 10 (Institution to Community). More detailed descriptions follow.

Table 1: State Plan Populations Affected by the Demonstration

State Plan Mandatory and Optional Groups	MMMC: Medicaid-eligible; not otherwise excluded from MMMC enrollment (includes HARP and SNP for eligible individuals)	MLTC: Need more than 120 days of community-based long-term care services
Pregnant Women
Pregnant women (42 CFR §435.116) Income up to 218% of FPL Pregnant minors under age 21 (42 CFR §435.222) No income test	Demonstration Population 2 [TANF Adult]	Without Medicare: Demonstration Population 5 [Non-Duals 18-64] With Medicare: Demonstration Population 7 [MLTC Adult Age 18-64 Duals]
Children
Infants (218% FPL) and children under age 19 (149% FPL) (42 CFR §435.117 and §435.118)	Demonstration Population 1 [TANF Child]	N/A
Children age 19 and 20 (42 CFR §435.222) Income up to 133% of FPL if living alone and 150% if living with parents	Demonstration Population 1 TANF CHILD	Without Medicare: Demonstration Population 5 [Non-Duals 18-64] With Medicare: Demonstration Population 7 [MLTC Adult Age 18-64 Duals]
Medically needy children age 19 and 20 (42 CFR §435.308) Income at or below the monthly income standard or with spenddown	N/A	Without Medicare: Demonstration Population 5 [Non-Duals 18-64] With Medicare: Demonstration Population 7 [MLTC Adult Age 18-64 Duals]
Adults
Adult group (42 CFR §435.119) Over age 18, under age 65, non-disabled, non- pregnant with income up to 133% of FPL, not eligible for Medicare Part A or B benefits, not eligible under the parents and other caretaker relative group, the foster care child group, or the former foster care child group.	Demonstration Population 11 [New Adult Group]	New Adult Group: Demonstration Population 11
Parents and Caretakers
Parents and other caretaker relatives (42 CFR §435.110 and §435.220) Income up to 133% of FPL Includes low-income adults enrolled in TANF who are exempt from receiving a MAGI determination in accordance with §1902(e)(14)(D)(i)(I) of the Act. Includes Transitional Medical Assistance under sections 1902(a)(52) and (e)(1); 1925; and 1931(c)(2) of the Social Security Act	Demonstration Population 2 [TANF Adult]	Without Medicare: Demonstration Population 5 [Non-Duals 18-64] With Medicare: Demonstration Population 7 [MLTC Adult Age 18-64 Duals]
Medically needy parents and other caretaker relatives (42 CFR §435.310) Income at or below the monthly income standard or with spenddown	N/A	Without Medicare, Demonstration population 5 [Non-Duals 18-64] With Medicare, Demonstration population 7 [MLTC Adult Age 18-64 Duals]
Disabled
Blind and disabled individuals age 64 and under receiving SSI (42 CFR §435.120)	Voluntarily enrolled or required to enroll in managed care in those counties participating in the MRT (formerly Partnership Plan) as of October 1, 2006, Demonstration Population 3 [SSI 0 through-64]	Without Medicare, Demonstration Population 5 [Non-Duals 18-64] With Medicare, Demonstration Population 7 [MLTC Adults 18 -64 Duals]
Medically needy adults/children aged 18 through 64 blind and disabled (42 CFR §435.322 and §324) Income at or below the monthly income standard, or with spend down to monthly income standard	N/A	Without Medicare, Demonstration Population 5 [Non-Duals 18-64] With Medicare, Demonstration Population 7 [MLTC Adults 18 -64 Duals]
Aged 18 through 64 Medicaid Buy In for Working People with Disabilities Income up to 250% of FPL	Demonstration Population 2 [TANF Adult]	Without Medicare, Demonstration Population 5 [Non-Duals 18-64] With Medicare, Demonstration Population 7 [MLTC Adults 18 -64 Duals]
Aged
Aged Individuals Age 65 and Over Receiving SSI (42 CFR §435.120) Optional Adults aged 65 or older (42 CFR §435.210)	Voluntarily enrolled or required to enroll in managed care in those counties participating in the MRT (formerly Partnership Plan) as of October 1, 2006, Demonstration Population 4, [SSI 65 and above]	Without Medicare, Demonstration Population 6 [Non-Duals 65+] With Medicare, Demonstration Population 8 [MLTC age 65+ Duals]
Medically needy age 65 and over (42 CFR §435.320) Income at or below the monthly income standard, or with spend down to monthly income standard	N/A	Without Medicare, Demonstration Population 6 [Non-Duals 65+] With Medicare, Demonstration Population 8 [MLTC age 65+ Duals]
Foster Care
Children with adoption assistance, foster care or guardianship under title IV-E (42 CFR §435.145)	Demonstration Population 1 [TANF Child]	N/A
Children in state foster care Children receiving non IV-E guardianship assistance (42 CFR §435.222)	Demonstration Population 1 [TANF Child]	N/A
Former foster care children up to age 26 (42 CFR §435.150)	Demonstration Population 1 [TANF Child]	N/A
Independent Foster Care Adolescents 18 through 20 (In foster care on the date of 18th birthday) (42 CFR §435.226)	Demonstration Population 1 [TANF Child]	Without Medicare, Demonstration Population 5 [Non-Duals 18-64] With Medicare, Demonstration Population 7 [MLTC Adults 18 -64 Duals]
Children who qualify under 1902 (a)(10)(A)(ii)(VIII) State adoption assistance.	Demonstration Population 1 [TANF Child]	N/A

Mainstream Medicaid Managed Care Program (MMMC). This component provides Medicaid state plan and demonstration benefits through a managed care delivery system comprised of MCOs and primary care case management (PCCM) arrangements to most recipients eligible under the state plan. (See Attachment A for a listing of MMMC benefits.) All state plan eligibility determination rules apply to these individuals.

Eligibility. Table 1 above lists the groups of individuals who receive Medicaid benefits through the mainstream Medicaid managed care component of the demonstration, as well as the relevant expenditure reporting category (demonstration population) for each. Individuals enrolled in Medicaid under the Adult Group (including individuals eligible through the continuous eligibility expenditure authority) who turn 65 years of age, upon receipt of Medicare, will be seamlessly disenrolled from MMMC and continue receiving Medicaid on a fee for service basis until a redetermination of eligibility can be made based on another category of assistance. MMIS will indicate that Medicaid is the secondary payer of any claims for these individuals, after Medicare or applicable third-party coverage.
Exclusions and Exemptions from MMMC. Notwithstanding the eligibility criteria in STC 3 of this section, certain individuals cannot receive benefits through the MMMC program (i.e., excluded), while others may opt out from receiving benefits through the MMMC program (i.e., exempted). Excluded individuals are outside the demonstration and are not included in Demonstration Populations. Exempt individuals are included in the demonstration and in Demonstration Populations regardless of whether they enroll in managed care. Tables 2 and 3 list those individuals either excluded or exempted from MMMC.

Table 2: Individuals Excluded from MMMC (including HARP and HIV SNP)

Individuals who become eligible for Medicaid only after spending down a portion of their income

Residents of state psychiatric facilities and residents of Residential Treatment Facilities for Children and Youth

Individuals under age 21 who are permanent residents of Residential Health Care Facilities or temporary residents of Residential Health Care Facilities at time of enrollment

Medicaid eligible infants living with incarcerated mothers

Youth in OCFS facilities and in the care and custody of the Office of Family & Children Services

Individuals with access to comprehensive private health insurance

Certified blind or disabled children living or expected to live separate and apart from their parents for 30 days or more

Individuals expected to be Medicaid eligible for less than 6 months (except for pregnant women)

Individuals receiving hospice services (at time of enrollment)

Individuals with a "county of fiscal responsibility" code of 97, except for individuals in the New York Office of Mental Health family care program who other than their residence in district 97 would be eligible to enroll in MMMC

Individuals with a "county of fiscal responsibility" code of 98 including Individuals in an Office for People with Developmental Disabilities/OPWDD facility or treatment center

Individuals who are under 65 years of age (screened and require treatment) in the Centers for Disease Control and Prevention breast, cervical, colorectal or prostate cancer, and who are not otherwise covered under creditable health coverage (Individuals with a "county of responsibility" code of 99)

Individuals who are eligible for Emergency Medicaid

Aliessa Court Ordered Individuals*

Medicare recipients

Residents of Assisted Living Programs

* Aliessa Aliens are NOT excluded from Managed Care but are excluded from FFP.

Table 3: Individuals who may be exempted from MMMC (including HARP and HIV SNP)

Individuals with chronic medical conditions who have been under active treatment for at least 6 months with a sub- specialist who is not a network provider for any Medicaid MCO in the service area or whose request has been approved by the New York State Department of Health Medical Director because of unusually severe chronic care needs. Exemption is limited to six months

Child and Youth residents of Residential Rehabilitation Services for Youth (RRSY). Note: as the RRSY services are phased into managed care through contract amendments, the children in RRSYs will mandatorily phase into the demonstration.

Individuals designated as participating in OPWDD-sponsored programs

Native Americans

Individuals in the following Section 1915(c) waiver programs: Traumatic Brain Injury (TBI) and Nursing Home Transition & Diversion (NHTD)

Individuals in the Office for People with Developmental Disabilities Home and Community Based Services (OPWDD HCBS) Section 1915 (c) waiver program

Managed Long Term Care (MLTC). This component provides a limited set of Medicaid state plan benefits including long-term services and supports through a managed care delivery system to individuals eligible through the state plan who require more than 120 days of community-based long-term care services as indicated on the uniform assessment tool. See Attachment B for a listing of MLTC services. Services not provided through the MLTC program are provided on a fee-for-service basis. The state has authority to expand mandatory enrollment into MLTC to all individuals identified in under the MLTC column in Table 1 (except those otherwise excluded or exempted as outlined in 3(a)(ii) of this section).

Eligibility for MLTC. Table 1 above lists the groups of individuals who may be enrolled in the Managed Long Term Care component of the demonstration as well as the relevant expenditure reporting category (demonstration population) for each. To be eligible, all individuals in this program must need more than 120 days of community-based long-term care services and for MAP and PACE also have a nursing home level of care.
Exclusions and Exemptions from MLTC. Notwithstanding the eligibility criteria in STC 3 of this section, certain individuals cannot receive benefits through the MLTC program (i.e., excluded) while others may request an exemption from receiving benefits through the MLTC program (i.e. exempted). Excluded individuals are outside the demonstration, and are not included in Demonstration Populations. Exempt individuals are included in the demonstration and in Demonstration Populations regardless of whether they enroll in managed care. Tables 4 and 5 list those individuals either excluded or exempted from MLTC.
Non-duplication of Payment. MLTC Programs will not duplicate services included in an enrollee´s Individualized Education Program under the Individuals with Disabilities Education Act, or services provided under the Rehabilitation Act of 1973.

Table 4: Individuals excluded from MLTC

Residents of psychiatric facilities (stays exceeding 30 days)

Residents of skilled nursing or residential health care facilities who have been LTNHS in such facility are excluded from enrollment in a partially capitated MLTC plan.

Residents of skilled nursing or residential health care facilities who are enrolled in a partially capitated MLTC plan are ineligible to continue their MLTC plan enrollment if they are LTNHS for more than three months.

Individuals expected to be Medicaid eligible for less than six months

Individuals eligible for Medicaid benefits only with respect to tuberculosis-related services

Individuals with a "county of fiscal responsibility" code 99 in MMIS (Individuals eligible only for breast and cervical cancer services)

Individuals receiving hospice services (at time of enrollment)

Individuals with a "county of fiscal responsibility" code of 97 (Individuals residing in a state Office of Mental Health facility)

Individuals with a "county of fiscal responsibility" code of 98 including Individuals in an OPWDD facility or treatment center

Individuals who are under 65 years of age (screened and require treatment) in the Centers for Disease Control and Prevention breast, cervical, colorectal and/or prostate early detection program and need treatment for breast, cervical, colorectal or prostate cancer and who are not otherwise covered under creditable health coverage

Residents of intermediate care facilities for individuals with intellectual disabilities (ICF/IID)

Individuals who could otherwise reside in an ICF/IID, but choose not to

Residents of alcohol/substance abuse long-term residential treatment programs

Individuals eligible for Emergency Medicaid

Individuals in the Office for People with Developmental Disabilities Home and Community Based Services (OPWDD HCBS) section 1915(c) waiver program

Individuals in the following section 1915(c) waiver programs: Traumatic Brain Injury (TBI), Nursing Home Transition & Diversion (NHTD) (see Attachment G)

Residents of Assisted Living Programs

Individuals in receipt of Limited Licensed Home Care Services

Individuals in the Foster Family Care Demonstration

Aliessa Court Ordered Individuals*

* Aliessa Aliens are NOT excluded from Managed Care but are excluded from FFP.

Table 5: Individuals who may be exempted from MLTC

Individuals aged 18 through 20 who are nursing home certifiable and require more than 120 days of community-based long-term care services

Native Americans

Individuals who are eligible for the Medicaid buy in for the working disabled and are nursing home certifiable

Home and Community Based Services Expansion Program (HCBS Expansion). This component provides home and community-based services similar to those provided under the state´s section 1915(c) HCBS waivers (Nursing Home Transition and Diversion Program/NHTD, and Traumatic Brain Injury Program/TBI) to certain medically needy individuals. These services enable these individuals to live at home with appropriate supports rather than in a nursing facility. See Attachment C for HCBS Expansion services. All HCBS Expansion individuals will be transitioned as appropriate to MLTC.
1. Eligibility for the HCBS Expansion. This group, identified as Demonstration Population 9/HCBS Expansion, includes married medically needy individuals¹:
  1. who meet a nursing home level of care;
  2. whose spouse lives in the community; and
  3. who would be income-eligible for Medicaid services in the community but for the application of the spousal impoverishment eligibility and post-eligibility rules of section 1924 of the Act.
Health and Recovery Plans (HARP): This component provides integrated Medicaid covered services and services specifically to address the needs of individuals with a serious mental illness (SMI) and substance use disorder (SUD) conditions under the demonstration. Members enrolled in the Health and Recovery Plans described below may elect to remain enrolled in mainstream MCOs. Within the HARPs, a benefit package of behavioral health (BH) home and community-based services (HCBS) is provided, in addition to the existing MMMC benefit package (excluding long-term nursing facility services). See Attachment D for a listing of BH HCBS.
1. Eligibility for HARP. Eligible individuals include Medicaid adult beneficiaries age 21 or over eligible for Medicaid furnished in MMMC under the demonstration with a specified SMI and/or serious SUD diagnosis and who meet categorical criteria or risk factors specified by New York´s Office of Mental Health (OMH) or New York´s Office of Alcoholism and Substance Abuse Services (OASAS) identified by a:
  1. review of behavioral health service utilization, or
  2. receipt of a qualifying score on a State-approved assessment tool.

Population-Specific Program Requirements
1. MMMC Enrollment of Individuals Living with HIV. The state is authorized to require individuals living with HIV to receive benefits through MMMC. Individuals living with HIV will have 30 days in which to select a health plan. If no selection is made, the individual will be auto assigned to an MCO. Individuals living with HIV who are enrolled in an MCO (voluntarily or by default) may request transfer to an HIV Special Needs Plan (SNP) at any time if one or more HIV SNPs are in operation in the individual´s district. Further, transfers between HIV SNPs will be permitted at any time. Individuals in HIV SNPs will be eligible for BH HCBS if meeting the targeting, risk and functional needs requirements for BH HCBS. HIV SNPs will meet all requirements of MMMC plans providing LTSS as well as HARP plans relating to delivery of BHHCBS.
2. Restricted Recipient Programs. The state may require individuals participating in a restricted recipient program administered under 42 CFR §431.54(e) to enroll in MMMC or MLTC. Furthermore, MCOs may establish and administer restricted recipient programs, through which they identify individuals that have utilized Medicaid services at a frequency or amount that is not medically necessary, as determined in accordance with utilization guidelines established by the state, and restrict them for a reasonable period of time to obtain Medicaid services from designated providers only. The state must adhere to the following terms and conditions in this regard.
  1. Restricted recipient programs operated by MCOs must adhere to the requirements in 42 CFR §431.54(e) (1) through (3), including the right to a hearing conducted by the state.
  2. The state must require MCOs to report to the state whenever they want to place a new person in a restricted recipient program. The state must maintain summary statistics on the numbers of individuals placed in restricted recipient programs, and the reasons for those placements, and must provide the information to CMS upon request.
3. Individuals Moved from Institutional Settings to Community Settings for Long Term Services and Supports. Individuals discharged from a nursing facility who enroll into or remain enrolled in the MLTC program in order to receive community-based long-term services and supports or who move from an adult home as defined in subdivision 25 of section 2 of the social services law, to the community and, if applicable, enroll into the MLTC program, are eligible based on a special income standard. The special income standard is also available to MLTC members who were enrolled in the program as a result of the mandatory Nursing Facility transition, and subsequently able to be discharged to the community from the nursing facility, with the services of MLTC program in place. For married individuals who meet the criteria to be considered an "institutionalized spouse" spousal impoverishment rules shall apply. Eligibility is not based on the special income standard for individuals subject to spousal impoverishment rules. The special income standard will be determined by utilizing the average Housing and Urban Development (HUD) Fair Market Rent (FMR) dollar amounts for each of the seven regions in the state, and subtracting from that average, 30 percent of the Medicaid income level (as calculated for a household of one) that is considered available for housing. The seven regions of the state include: Central, Northeastern, Western, Northern Metropolitan, New York City, Long Island and Rochester.
  
  The state shall work with Nursing Home Administrators, nursing home discharge planning staff, family members and the MLTC health plans to identify individuals who may qualify for the housing disregard as they are able to be discharged from a nursing facility back into the community and remain enrolled in or newly enrolled into the MLTC program.
  
  Enrollees receiving community-based long-term services and supports must be provided with nursing facility coverage through managed care, if nursing facility care is needed for 120 days or less and there is an expectation that the enrollee will return to community-based settings. During the short-term nursing facility stay, the state must retain the enrollees´ community maintenance needs allowance. In addition, the state will ensure that the MLTC Managed Care Organizations (MCOs) work with individuals, their families, nursing home administrators, and discharge planners to help plan for the individual´s move back into the community, as well as to help plan for the individual´s medical care once he/she has successfully moved into his/her home. For dually eligible enrollees, the MCO is responsible for implementing and monitoring the plan of care between Medicare and Medicaid. The MCO must assure the services are available to the enrollee.
4. Continuous Eligibility Period
  1. Duration. The state is authorized to provide a 12-month continuous eligibility period to the groups of individuals specified in Table 1, regardless of the delivery system through which they receive Medicaid benefits. Each newly eligible individual´s 12-month period shall begin at the initial determination of eligibility; for those individuals who are re-determined eligible consistent with Medicaid state plan rules, the 12-month period begins at that point. At each annual eligibility redetermination thereafter, if an individual is re-determined eligible under the Medicaid state plan the individual is guaranteed a subsequent 12-month continuous eligibility period. 12-month continuous eligibility is also authorized for the new Adult Group under section 1902(a)(10)(A)(i)(VIII) of the Act.
  2. Exceptions. Notwithstanding subparagraph (a), if any other following circumstances occur during an individual´s 12-month continuous eligibility period, the individual´s Medicaid eligibility shall be terminated, suspended or re-determined:
    1. The individual cannot be located
    2. The individual is no longer a New York State resident
    3. The individual requests termination of eligibility
    4. The individual dies
    5. The individual fails to provide, or cooperate in obtaining a Social Security Number, if otherwise required
    6. The individual provided an incorrect or fraudulent Social Security Number
    7. The individual was determined eligible for Medicaid in error
    8. The individual is receiving treatment in a setting where Medicaid eligibility is not available (e.g. institution for mental disease)
    9. The individual is receiving care, services or other supplies under a section 1915 waiver
    10. The individual was previously otherwise qualified for emergency medical assistance benefits only, based on immigration status, but is no longer qualified because the emergency has been resolved
    11. The individual fails to provide the documentation of citizenship or immigration status required under federal law
    12. The individual is incarcerated
    13. The individual turns 65 years of age and is no longer eligible for the Adult Group (beginning January 1, 2016)
    14. The individual policy holder fails to provide documentation of third-party health insurance

Table 6: Groups Eligible for a 12 Month Continuous Eligibility Period
State Plan Mandatory and Optional Groups	Statutory or Regulatory Reference
Individuals determined eligible as pregnant women	42 CFR §435.116
Individuals determined eligible as the Adult Group	42 CFR §435.119
Individuals determined eligible as parents or other caretaker relatives	42 CFR §435.110
Low income families, except for children	§1931 of the SSA

|top of section| |top of page|

V. DEMONSTRATION BENEFITS AND ENROLLMENT

Alternative Benefit Plan. The Affordable Care Act Adult Group will receive benefits provided through the state´s approved Alternative Benefit Plan (ABP) SPA.
Demonstration Benefits. The following benefits are provided through the indicated delivery system to individuals eligible for the Medicaid managed care components of the demonstration:
1. Mainstream Medicaid Managed Care (MMMC). State plan and demonstration benefits are delivered through MCOs with the exception of certain services carved out of the MMMC contract and delivered directly by the state on a fee-for-service basis. All MMMC benefits (regardless of delivery method), as well as the co- payments charged to MMMC recipients, are listed in Attachment A. In addition to state plan benefits, there are three demonstration services provided only to all enrollees in MMMC under the demonstration…
  1. Cost Sharing for MMMC. MMMC beneficiaries including HARPs and HIV- SNPs, who are not otherwise exempt from cost sharing consistent with §447.56(a)(1), will be charged drug copays that are approved in the Medicaid state plan. MMMC beneficiaries will not be subject to any non-drug copays that are described in the Medicaid state plan.
  2. Children´s HCBS. MMMC plans will provide HCBS for children not otherwise excluded or exempted from MMMC under the concurrent authority of the 1915(c) Children´s waiver and this 1115 demonstration. Independent assessments and person-centered services planning for HCBS under the Children´s waiver will be conducted by a State Plan Health Home provider or the State´s Independent Entity as described and included in the approved Children´s waiver. All HCBS benefits are listed in the approved Children´s waiver or the approved State Plan for CFCO. All reimbursement for Children´s Waiver HCBS will be non-risk for the first 24 months subject to the non-risk UPL at 42 CFR 447.362. The MCO must pay the FFS fee schedule for non-risk services as long as the HCBS are non-risk (i.e., 24 months). There are no co-payments for Children´s waiver services.
2. Managed Long-Term Care. State plan benefits are delivered through MCOs or, in certain districts, prepaid inpatient health plans, with the exception of certain services carved out of the MLTC contract and delivered directly by the state on a fee-for- service basis. All MLTC benefits are listed in Attachment B.
  1. For those individuals receiving a nursing home benefit in the partially capitated MLTC plan they will be limited to three months for those enrollees who have been designated as Long-Term Nursing Home Stays (LTNHS) in a skilled nursing or residential health care facility as of the effective date of this amendment. After three months the individual will be involuntarily disenrolled from the partially capitated MLTC plan and payment for nursing home services will be covered by Medicaid fee for service for individuals who qualify for institutional Medicaid coverage.
  2. Should an individual prefer discharge-and an assessment of the individual´s medical needs indicates they may be safely discharged to the community- they may remain enrolled in their MLTC plan, while residing in the nursing home on a temporary basis for more than three months, until their discharge plans are resolved and the individual is transitioned out of the nursing home.
3. Health and Recovery Plans (HARP). State plan and demonstration benefits that are identical to MMMC with an additional component that provides BH HCBS for SMI and SUD needs will be provided by the HARPs. Long term care services (in excess of 120 days) or permanent placement in a Nursing facility, however, are not provided by HARPs. There are no co-payments for HARP services. All BH HCBS benefits are listed in Attachment D. BH HCBS for HARP enrollees meeting targeting, risk, and need-based functional criteria are only provided under the demonstration. The state must update the Medicaid state plan for rehabilitation and other mental health and substance use disorder services as identified through a companion letter to TN 10-38 as well as substance use disorder demonstration services not described in the current state plan. HIV SNPs also provide BH HCBS to enrollees meeting targeting, risk, and needs-based criteria. All reimbursement for BH HCBS in HARPs and HIV SNPs will be non-risk.
  1. HARPs Services Tiers. HARPs enrollees receive BH HCBS services under the following tier structure in accordance with their person-centered plan of care. HARP enrollees are permitted to appeal any service denial decisions.
    1. Tier 1 BH HCBS services include:
      1. Peer supports
      2. Employment supports
      3. Education supports
    2. Tier 2 includes all Tier 1 BH HCBS services plus additional services as specified in Attachment D to individuals whose medical need surpasses the need for Tier 1 services.
    3. Crisis respite services under the HARPs are available to all HARPs enrollees, regardless of the tier under which they receive services. This includes:
      1. Intensive crisis respite
      2. Short term crisis respite, in a dedicated facility
  2. HARPs Services Utilization Thresholds. The following thresholds will limit coverage of HARPs-specific services for individual HARPs enrollees. These limits will not affect state plan or other demonstration benefits. The state will track and report overall utilization, including any utilization threshold exceeded for clinical reasons, to ensure cost containment as well as compile sufficient fee for service data to submit HARPs capitation rates to CMS for approval.
    1. Tier 1 -- Threshold of $8,000 per person, per 12-month period. Up to $10,000 in services are permitted. For ROS, the thresholds will be adjusted to reflect the HCBS rate differentials.
    2. Tier 2 -- Threshold of $16,000 per person, per 12-month period. Up to $20,000 in services are permitted. For ROS, the thresholds will be adjusted to reflect the HCBS rate differentials.
    3. Crisis Respite - Threshold of 7 days per service, up to 21 days per 12-month period.
  3. Self-Direction Pilot. The Self-direction Demonstration will be available to HARP and HIV/SNP enrollees eligible for receiving BH HCBS services or children meeting the target criteria of the Children´s waiver and receiving HCBS under the Children´s waiver through MMMC. The program will be in effect from January 1, 2017 through March 31, 2021. It will include 8 pilot sites phased in over the demonstration.
    1. Voluntary Enrollment and Disenrollment from Self-Direction Pilot. Participation in the Self-Direction pilot is voluntary, and participants may opt out at any time.
    2. Enrollee Notification. The state must notify eligible enrollees about the option to self-direct services. The state must develop a waiting list for enrollees who wish to participate in the pilot should the demand exceed capacity.
    3. Choice of Providers. Self-direction pilot participants will have a choice of support broker within the service center. Each participant should have the choice of provider and location for self-directed services, except as noted in iv(e) below.
    4. Services Eligible for Self-Direction: This pilot includes all behavioral health HCBS services offered by HARPs and HIV SNPs and Individual Directed Goods and Services (IDGS) detailed in Attachment N. Children meeting targeting criteria of the 1915(c) Children´s waiver and receiving HCBS through MMMC are eligible to self-direct up to $2,000 in IDGS only using a Fiscal Management Services provider within the service center. Each participant will have the choice of provider and location for IDGS. Children´s IDGS should be used as the funding source of last resort - only for those costs that cannot be covered by any other source and that are vital to the implementation of the POC. Individual Directed Goods and Services are services, equipment, or supplies not otherwise provided through this waiver, the 1915(c) Children´s Waiver, or through the Medicaid State Plan that address an identified need in the service plan. The item or service must be identified in the service plan and either:
      1. decrease the need for other Medicaid services;
      2. promote inclusion in the community; or
      3. increase the participant´s safety in the home environment.
      4. To be an eligible service:
        
        the participant must lack funds to purchase the item or service; and
        
        the service is not available through another source.
    5. Services Ineligible for Self-Direction: Individual goods and services that are not eligible are listed below.
      1. Experimental or prohibited treatments
      2. Purchases for or from third parties who are family members, friends, or significant others aside from family or social functions that promote social inclusion and are incorporated in the service plan
      3. Room and Board in a residential facility, including assisted living facilities
      4. Tobacco products, alcohol products, firearms, contraband or illegal items
      5. Pornographic materials, prostitution services, escort services
      6. Payment of court-ordered costs, attorney fees, fines, restitution, or similar debts
      7. Credit card payments of any kind, or similar debts
      8. Items purchased for the purpose of resale
      9. >Gift cards or prepaid debit cards
      10. Services or goods that are recreational in nature
      11. Goods and services not in the service plan or related to a recovery goal, or that is solely for recreation that a household does not include a person with a disability would be expected to pay for as a household expenses (e.g. subscription to a cable television service)
    6. Evaluation. The state shall follow the evaluation requirements specified in Section XI below.
    7. Reporting. Information from the pilot must be incorporated into the quarterly and annual reports detailed in section X of the STCs.
    8. Protocols. Payment and operational protocols must be submitted by New York to CMS within 120 days of award.
Home and Community Settings Qualities. Enrollees receiving Medicaid HCBS and LTSS services furnished through the 1115 demonstration, including individuals who receive services under the demonstration´s HCBS Expansion program, MMMC and HARP, including HIV SNP, must receive services in residential and non-residential settings located in the community, which meet CMS standards for HCBS settings as articulated in current 1915(c) policy, including regulations at 42 CFR §441.301. The Statewide Transition Plan must include HARPs BH HCBS settings and meet CMS approval for required settings to be funded beyond November 30, 2015. A full list of home and community-based qualities are provided in Attachment C.
Individuals Provided with LTSS under the Demonstration. The state is authorized to require certain individuals using long-term services and supports to enroll in either Mainstream Medicaid Managed Care, or Managed Long-Term Care as identified in Section I. Once these individuals are enrolled in managed care, the state is required to provide the following protections for the population.²
1. Person Centered Service Planning. All individuals utilizing long-term services and supports will have a person-centered individual service plan maintained at the MCO. Person-centered planning includes consideration of the current and unique psycho- social and medical needs and history of the enrollee, as well as the person´s functional level, and support systems. The person-centered plan is developed by the enrollee with the assistance of the MCO and individuals the enrollee chooses to include.
  
  When a service provider is an approved State Plan Health Home³ provider and also a HCBS provider, this entity may conduct person-centered service planning, care coordination, and provision of HCBS provision as long as firewalls are constructed between the service planning, care coordination, and service provision. A home and community-based service provider who is not also an approved State Plan Health Home provider may not conduct person-centered service planning with individuals who they also provide HCBS, unless that service provider is the only qualified and willing entity available to conduct the service planning. If a service provider is the only willing and qualified entity to conduct service planning, the state must require such provider to establish firewalls between the service provision and planning functions. The person-centered plan is developed in accordance with 42 CFR §441.301(c)(4)(F)(1) through (8).
2. Health home program will have administrative safeguards in place when providing person-centered planning and care coordination and services that have transitioned from 1915(c) waivers to eligible health home individuals. In addition, the state agrees to meet all health home requirements including reporting annually on quality and utilization measures.
3. Verification of MLTC Plan Enrollment. The state shall implement a process for MLTC plans, network and non-network providers for the state to confirm enrollment of enrollees who do not have an enrollee identification card or seek services from a provider before developing a person-centered service plan.
4. Health and Welfare of Enrollees. The state shall ensure a system is in place to identify, address, and seek to prevent instances of abuse, neglect, and exploitation of its enrollees on a continuous basis. This should include provisions such as critical incident monitoring and reporting to the state, investigations of any incident including, but not limited to, wrongful death, restraints, or medication errors that resulted in an injury. In each quarterly report, the state will provide information regarding any such incidents by plan. The state will also ensure that children and adults receiving MLTC or LTSS are afforded linkages to child and/or adult protective services through all service entities, including the MCOs.
5. Maintaining Accurate Beneficiary Address. New York will complete return mail tracking for enrollment notification mailings. The state will use information gained from returned mail to make additional outreach attempt through other methods (phone, email, analysis of prior claims, etc.).
6. Network of Qualified Providers. The provider credentialing criteria described at 42 CFR §438.214 must apply to all providers participating in the state´s Medicaid managed care and managed long-term care programs. To the extent possible, the MCO shall incorporate criminal background checks, reviewing abuse registries as well as any other mechanism the state includes within the MCO contract.
7. MMMC or MLTC Enrollment and Transition of Care Period. For initial transitions into MLTC or MMMC from fee-for-service, each enrollee receiving community-based LTSS must continue to receive services under the enrollee´s pre- existing service plan for at least 90 days after enrollment or until a care assessment has been completed. Any reduction, suspension, denial or termination of previously authorized services shall trigger the required notice under 42 CFR § 438.404 and applicable appeal rights.
Option for Consumer Directed Personal Assistance Program (CDPAP). Enrollees shall have the option to elect self-direction of Personal Assistance under the MMMC program. The state shall ensure through its contracts with the MCOs that enrollees are afforded the option to select self-direction and enrollees are informed of CDPAP as a voluntary option. Individuals who select self-direction must have the opportunity to have choice and control over how services are provided and who provides the service, except as noted in STC 2(E) of this section.
1. Information and Assistance in Support of Participant Direction. The state/MCO shall have a support system that provides participants with information, training, counseling, and assistance, as needed or desired by each participant, to assist the participant to effectively direct and manage their self-directed services. Participants shall be informed about self-directed care, including feasible alternatives, before electing the self-direction option.
2. Participant Direction by Representative. The participant who self-directs the personal care service may appoint a volunteer designated representative to assist with or perform employer responsibilities to the extent approved by the participant. Services may be directed by a legal representative of the participant. Consumer- directed services may be directed by a non-legal representative freely chosen by the participant. A person who serves as a representative of a participant for the purpose of directing services cannot serve as a provider of personal attendant services for that participant.
3. Participant Employer Authority. The participant (or the participant´s representative) must have decision making authority over workers who provide personal care services.
  1. Participant. The participant (or the participant´s representative) provides training, supervision and oversight to the worker who provides services. A Fiscal/Employer Agent that follows IRS and local tax code laws functions as the participant´s agent in performing payroll and other employer responsibilities that are required by federal and state law.
  2. Decision-Making Authorities. The participants exercise the following decision-making authorities: recruit staff, hire staff, verify staff´s ability to perform identified tasks, schedule staff, evaluate staff performance, verify time worked by staff and approve time sheets, and discharge staff.
4. Disenrollment from Self-Direction. A participant may voluntarily disenroll from the self-directed option at any time and return to a traditional service delivery system through the MMMC, or MLTC program. To the extent possible, the member shall provide his/her intent to withdraw from participant direction. A participant may also be involuntarily disenrolled from the self-directed option if continued participation in the consumer-directed services option would not permit the participant´s health, safety, or welfare needs to be met, or the participant demonstrates the inability to self- direct by consistently demonstrating a lack of ability to carry out the tasks needed to self-direct services, or if there is fraudulent use of funds such as substantial evidence that a participant has falsified documents related to participant-directed services. If a participant is terminated voluntarily or involuntarily from the self-directed service delivery option, the MCO must transition the participant to the traditional agency direction option and must have safeguards in place to ensure continuity of services.
5. Payment for Services Will be Made Following the Service Being Rendered and only upon receipt of an acceptable receipt, invoice or signed and approved timesheet, as applicable.
6. Appeals. The following actions shall be considered adverse action under both 42 CFR §431 subpart E and 42 CFR §438 subpart F:
  1. a reduction, suspension or termination of authorized CDPAP services;
  2. A denial of a request to change Consumer Directed Personal Assistance Program services.
Adding Services to the MMMC, and/or MLTC Plan Benefit Package. At any point in time the state intends to add to either the MMMC, or MLTC plan benefit package currently authorized state plan or demonstration services that have been provided on a fee-for-service basis, the state must provide CMCS, Disabled and Elderly Health Programs Group, Division of Managed Care Plans (DMCP) the following information, with at least 30 days´ notice prior to the inclusion of the benefit, in writing:
1. A description of the benefit being added to the benefit package;
2. A detailed description of the state´s oversight of the MCO´s readiness to administer the benefit including:
  1. readiness and implementation of activities, including onsite reviews, phone meetings and desk audits that review policies and procedures for new services;
  2. data sharing to allow plans to create services plans as appropriate;
  3. process to communicate the change to enrollees;
  4. MCO network development to include providers of that service; and
  5. any other activity performed by the state to ensure plan readiness.
3. Information concerning the changes being made to the MMMC and/or MLTC contract provisions and capitation payment rates in accordance with STC 2 in Section VI.
  
  DMCP reserves the right to delay implementation of the benefit transition until such time as appropriate documentation is provided showing evidence of MCO readiness. In addition, new services that are not currently authorized under the state plan or demonstration may be added only through approved amendments to the state plan or demonstration.
  
  DMCP will notify the state of concerns within 10 days of receiving the state´s written notice of the change. If no comments are received, the state may proceed with the scheduled benefit transition.
Adding Populations to MMMC and/or MLTC Enrollment. Any time the state is ready to expand mandatory MMMC and /or MLTC plan enrollment into a new Medicaid population, the state must submit an 1115 amendment in accordance with STC 7 in Section III. The amendment request must include the following:
1. a description of the population and the list of the counties that will have populations moving to mandatory enrollment;
2. a list of MCO with an approved state certificate of authority to operate in those counties demonstrating that enrollees will be afforded choice of plan that will be providing services;
3. confirmation that the MCO have met the network requirements in STC 10 in Section VI for each MCO; and
4. an analysis of why the most appropriate authority to implement mandatory MMMC and/or MLTC for the new population, i.e. what the state is demonstrating by implementing the change to the demonstration.
Assurances During LTSS Expansion for MMMC, HIV SNP, and HARP Enrollees. To provide and demonstrate seamless transitions for enrollees, the state must (where applicable):
1. Send sample notification letters. Existing Medicaid providers must receive sample beneficiary notification letters via widely distributed methods (mail, email, provider website, etc.) so that providers are informed of the information received by enrollees regarding their managed care transition.
2. Provide continued comprehensive outreach, including educational tours for enrollees and providers. The educational tour should educate enrollees and providers regarding plan enrollment options, rights and responsibilities and other important program elements. The state must provide webinars, meeting plans, and send notices through outreach and other social media (e.g. state´s website). The enrollment broker, choice counseling entities, ombudsman and any group providing enrollment support must participate.
3. Operate a call center independent of the MLTC, and MMMC, HIV SNP, and HARP plans. This entity must be able to help enrollees in making independent decisions about plan choice and be able to document complaints about the plans. During the first 60 days of implementation the state must review all call center response statistics to ensure all contracted plans are meeting requirements in their contracts. After the first 60 days, if all entities are consistently meeting contractual requirements the state can decrease the frequency of the review of call center statistics, but no more than 120 days should elapse between reviews.
4. Review the outcomes of the auto-assignment algorithm to ensure that MLTC and MMMC plans with more limited networks do not receive the same or larger number of enrollees as plans with larger networks.
5. Require MCO to maintain the current worker/recipient relationship for no less than 90 days.
Assessment of LTSS needs for MLTC, and MMMC and Behavioral Health Assessments for HARPs and HIV SNPs. LTSS needs assessments must be conflict free plans will not complete any LTSS needs assessments for individuals requesting such services prior to enrollment in a plan. Non-dually eligible individuals requesting LTSS will be assessed for criteria necessitating enrollment in MLTC or an alternate waiver program. An independent LTSS assessment or behavioral health assessment system must be in place in any geographic location where MLTC or, LTSS in MMMC will be mandated or where HARP enrollment is an option. LTSS assessments for skilled nursing facility services in MMMC, and behavioral health assessments of enrollees aged 21 and over for HARPs and HIV SNPs will be conflict free prior to implementation and geographic phase in.
Post Assessment Education. New Medicaid applicants must be provided the results of their assessment and educated on the steps in the Medicaid eligibility determination, including denial and fair hearing procedures. Individuals who are currently Medicaid eligible must be provided information regarding choice of plan.
Operation of the HCBS Expansion Program. The individuals eligible for this component of the demonstration will receive the same HCBS as those individuals determined eligible for and enrolled in the state´s Nursing Home Transition and Diversion Program (NHTDP) and Traumatic Brain Injury Program (TBIP) authorized under section 1915(c) of the Act. The specific benefits provided to participants in this program are listed in Attachment C. The state will operate the HCBS Expansion program in a manner consistent with approved NHTDP and TBIP 1915(c) waiver programs and must comply with all administrative, operational, quality improvement and reporting requirements contained therein. The state shall provide enrollment and financial information about the individuals enrolled in the HCBS Expansion program.
Facilitated Enrollment. Facilitated enrollers, which may include MCOs, health care providers, community-based organizations, and other entities under state contract, will engage in those activities described in 42 CFR § 435.904(d)(2), as permitted by 42 CFR §435.904(e)(3)(ii), within the following parameters:
1. Facilitated enrollers will provide program information to applicants and interested individuals as described in 42 CFR §435.905(a).
2. Facilitated enrollers must afford any interested individual the opportunity to apply for Medicaid without delay as required by 42 CFR §435.906.
3. If an interested individual applies for Medicaid by completing the information required under 42 CFR §435.907(a) and (b) and 42 CFR §435.910(a) and signing a Medicaid application, that application must be transmitted to New York State Department of Health for determination of eligibility.
4. The protocols for facilitated enrollment practices between the state and the facilitated enrollers must:
  1. ensure that choice counseling activities are closely monitored to minimize adverse risk selection; and
  2. Specify that determinations of Medicaid eligibility are made solely by the Medicaid agency or its designee.
Passive Enrollment. For any component that requires passive enrollment of potential enrollees, individuals must have the ability to "opt out" Where the notice is sent 30 days in advance of the passive enrollment. The individual may opt out at any time after receipt of the notice and within the first 90 days following the passive enrollment. The individual may also change after the 12-month lock-in period at any time. Enrollees who enrolled through the health exchange or the local social services district in an MMMC plan whose MCO also operates a HARP line of business will be passively enrolled with the ability to opt-out within the first 90 days following passive enrollment and return to their original MMMC plan. Following the 90 day opt out period, HARP enrollees may not change plans again until the remainder of the twelve-month lock-in period has lapsed. HARP eligible enrollees in an MMMC plan who´s MCO does not operate a HARP line of business will be allowed to voluntarily enroll in a HARP. The enrollee must be given the choice of HARPs available for enrollment and the current plan must assist the enrollee in transferring to the HARP. The state will notify CMS and the public at least 60 days before exercising the option to modify needs-based eligibility criteria. When a HARP enrollee leaves the HARP and transfers into another plan, care must be coordinated for physical and behavioral health during the transition to best meet the needs of the enrollee. The current and new plans must work together when an enrollee transfers to another plan.
HCBS Electronic Visit Verification System. The state will demonstrate compliance with the Electronic Visit Verification System (EVV) requirements for personal care services (PCS) by January 1, 2020 and home health services by January 1, 2023 in accordance with section 12006 of the 21st Century CURES Act.
HCBS Quality Systems and Strategy. The state is expected to implement systems that measure and improve its performance to meet the waiver assurances set forth in 42 CFR and 441.302. The Quality Review provides a comprehensive assessment of the state´s capacity to ensure adequate program oversight, detect and remediate compliance issues and evaluate the effectiveness of implemented quality improvement activities.
For 1915(c)-Approvable HCBS, for services that could have been authorized to individuals served under a 1915(c) waiver, the state must have an approved Quality Improvement Strategy and is required to develop and measure performance indicators for the following waiver assurances:
1. Administrative Authority: A performance measure should be developed and tracked any authority that the State Medicaid Agency (SMA) delegates to another agency, unless already captured in another performance measure.
2. Level of Care: Performance measures are required for the following two sub-assurances: applicants with reasonable likelihood of needing services receive a level of care determination and the processes for determining level of care are followed as documented. While a performance measure for annual levels of care is not required to be reported, the state is expected to be sure that annual levels of care are determined.
3. Qualified Providers: The state must have performance measures that track that providers meet licensure/certification standards, that non-certified providers are monitored to assure adherence to waiver requirements, and that the state verifies that training is given to providers in accordance with the waiver.
4. Service Plan: The state must demonstrate it has designed and implemented an effective system for reviewing the adequacy of service plans for HCBS participants. Performance measures are required for choice of waiver services and providers, service plans address all assessed needs and personal goals, and services are delivered in accordance with the service plan including the type, scope, amount, duration, and frequency specified in the service plan.
5. Health and Welfare: The state must demonstrate it has designed and implemented an effective system for assuring HCBS participants health and welfare. The state must have performance measures that track that on an ongoing basis it identifies, addresses and seeks to prevent instances of abuse, neglect, exploitation and unexplained death; that an incident management system is in place that effectively resolves incidents and prevents further singular incidents to the extent possible; that state policies and procedures for the use or prohibition of restrictive interventions are followed; and, that the state establishes overall health care standards and monitors those standards based on the responsibility of the service provider as stated in the approved waiver.
6. Financial Accountability: The state must demonstrate that it has designed and implemented an adequate system for insuring financial accountability of the HCBS program. The state must have performance measures that track that it provides evidence that claims are coded and paid for in accordance for services rendered, and that it provides evidence that rates remain consistent with the approved rate methodology throughout the five-year waiver cycle.
The state will submit a report to CMS which includes evidence on the status of the HCBS quality assurances and measures that adheres to the requirements outlined in the March 12, 2014, CMS Informational Bulletin, Modifications to Quality Measures and Reporting in §1915(c) Home and Community-Based Waivers. (1915(c) and 1915(i) HCBS). NOTE: This information could be captured in the 1115 Summary report detailed in section VIII.
The state must report annually the deficiencies found during the monitoring and evaluation of the HCBS waiver assurances, an explanation of how these deficiencies have been or are being corrected, as well as the steps that have been taken to ensure that these deficiencies do not reoccur. The state must also report on the number of substantiated instances of abuse, neglect, exploitation and/or death, the actions taken regarding the incidents and how they were resolved. Submission is due no later than 6 months following the end of the demonstration year. NOTE: This information could be included in the annual reports submitted for 1115 waivers detailed in section VIII.
For 1915(i)-Approvable HCBS, for services that could have been authorized to individuals served under a 1915(i) waiver, the state must have an approved Quality Improvement Strategy and is required to develop performance measures to address the following requirements:
1. Service plans that:
  1. address assessed needs of 1915(i) participants;
  2. are updated annually; and
  3. document choice of services and providers.
2. Eligibility Requirements: The state will ensure that:
  1. an evaluation for 1915(i) State plan HCBS eligibility is provided to all applicants for whom there is reasonable indication that 1915(i) services may be needed in the future;
  2. the processes and instruments described in the approved program for determining 1915(i) eligibility are applied appropriately; and
  3. the 1915(i) benefit eligibility of enrolled individuals is reevaluated at least annually (end of demonstration year) or if more frequent, as specified in the approved program.
3. Providers meet required qualifications.
4. Settings meet the home and community-based setting requirements as specified in the benefit and in accordance with 42 CFR 441.710(a)(1) and (2).
5. The SMA retains authority and responsibility for program operations and oversight.
6. The SMA maintains financial accountability through payment of claims for services that are authorized and furnished to 1915(i) participants by qualified providers.
7. The state identifies, addresses, and seeks to prevent incidents of abuse, neglect, and exploitation.
8. The state must also describe the process for systems improvement as a result of aggregated discovery and remediation activities.
Person-centered planning. The state assures there is a person-centered service plan for each individual determined to be eligible for HCBS. The person-centered service plan is developed using a person-centered service planning process in accordance with 42 CFR 441.301(c)(1) (1915(c)) or 42 CFR 441.725(c) (1915(i)), and the written person-centered service plan meets federal requirements at 42 CFR 441.301(c)(2) (1915(c)) or 42 CFR 441.725(b) (1915(i)). The person-centered service plan is reviewed and revised upon reassessment of functional need as required by 42 CFR 441.365(e), at least every 12-months, when the individual´s circumstances or needs change significantly, or at the request of the individual.
Conflict of Interest: The state agrees that the entity that authorizes the services is external to the agency or agencies that provide the HCB services. The state also agrees that appropriate separation of assessment, treatment planning and service provision functions are incorporated into the state´s conflict of interest policies except for as stated in section V, STC 4(a).
Each beneficiary eligible for long-term services and supports will have informed choice on their option to self-direct LTSS, have a designated representative direct LTSS on their behalf, or select traditional agency-based service delivery. Both level of care and person- centered service planning personnel will receive training on these options. (MLTSS with self-direction)
The state, either directly or through its MCO contracts must ensure that participants´ engagement and community participation is supported to the fullest extent desired by each participant. (MLTSS)
The state will assure compliance with the characteristics of HCBS settings as described in 1915(c) and 1915(i) regulations in accordance with implementation/effective dates as published in the Federal Register.
Beneficiaries may change managed care plans if their residential or employment support provider is no longer available through their current plan. (MLTSS)

|top of section| |top of page|

VI. DELIVERY SYSTEMS

Contracts. Procurement and the subsequent final contracts developed to implement selective contracting by the state with any provider group shall be subject to CMS approval prior to implementation. Payments under contracts with public agencies, that are not competitively bid in a process involving multiple bidders, shall not exceed the documented costs incurred in furnishing covered services to eligible individuals (or a reasonable estimate with an adjustment factor no greater than the annual change in the consumer price index that shall be rebased based on actual documented costs no less than every two years).
Managed Care Contracts. No FFP is available for activities covered under contracts and/or modifications to existing contracts that are subject to 42 CFR §438 requirements prior to CMS approval of model contract language. The state shall submit any supporting documentation deemed necessary by CMS. The state must provide CMS with a minimum of 45 days to review and approve changes. CMS reserves the right, as a corrective action, to withhold FFP (either partial or full) for the demonstration, until the contract compliance requirement is met.
Managed Care Data Requirements. All managed care organizations shall maintain an information system that collects, analyzes, integrates and reports data as set forth at 42 CFR §438.242. This system shall include encounter data that can be reported in a standardized format. Encounter data requirements shall include the following:
1. Encounter Data (Health Plan Responsibilities). The health plan must collect, maintain, validate and submit data for services furnished to enrollees as stipulated by the state in its contracts with the health plans.
2. Encounter Data (State Responsibilities). The state shall, in addition, develop mechanisms for the collection, reporting, and analysis of these, as well as a process to validate that each plan´s encounter data are timely, complete and accurate. The state will take appropriate actions to identify and correct deficiencies identified in the collection of encounter data. The state shall have contractual provisions in place to impose financial penalties if accurate data are not submitted in a timely fashion. Additionally, the state shall contract with its EQRO to validate encounter data through medical record review.
3. Encounter Data Validation Study for New Capitated Managed Care Plans. If the state contracts with new managed care organizations, the state shall conduct a validation study 18 months after the effective date of the contract to determine completeness and accuracy of encounter data. The initial study shall include validation through a sample of medical records of demonstration enrollees.
4. Submission of Encounter Data to CMS. The state shall submit encounter data to the Medicaid Statistical Information System (MSIS) and when required T-MSIS (Transformed MSIS) as is consistent with federal law and per STC 17 in Section III. The state must assure that encounter data maintained at managed care organizations can be linked with eligibility files maintained at the state.
Interpretation Services and Culturally Competent Care. The MCOs and other entities acting on behalf of the state Medicaid agency, including, but not limited to enrollment brokers, must have interpretation services and provide care that is consistent with the individual´s culture. MCOs must conduct analyses to determine any gaps in access to these services and will expand its workforce accordingly. The MCOs may also require the use of remote video and voice technology when necessary.
Marketing Oversight.
1. The state shall require each MCO to meet 42 CFR §438.104 and state marketing guidelines regarding prohibition of cold calls, use of government logos, and other standards.
2. All materials used to market the MCO shall receive prior approval from the state.
3. The state shall require through its contracts that each MCO provide all individuals who were not referred to the plan by the enrollment broker with information (in a format determined by the state) describing managed long-term care, a list of available plans and contact information to reach the enrollment broker for questions or other assistance.
Managed Care Benefit Packages. Individuals enrolled in managed care plans under the demonstration must receive from the managed care program the benefits as identified in Attachments A through D, respectively. As noted in plan readiness and contract requirements, the state must require that, for enrollees in receipt of LTSS, each MCO/PIHP coordinate, as appropriate, needs state plan services that are excluded from the managed care delivery system but available through a fee-for-service delivery system, and must also assure coordination with services not included in the established benefit package. Plans will be at risk for any Medicaid covered service that is currently delivered. BH HCBS in HARPs and HIV SNPs will be non-risk for the initial years in accordance with STC 2 of section V. If the MCO network is unable to provide necessary medical services covered under the contract to a particular enrollee, the MCO will be required to cover these services out of network for the enrollee. The costs of room and board may not be covered and cannot be included when determining the MCO payment rates.
Managed Care Rates Transition for HARPs. While working towards a managed care capitated rate for HARPs, the state may not proceed with implementation in a region until it has approved HCBS fee for service rates for such region. The state must submit HARP capitation rates to CMS for approval no later than December 31, 2017. Should the state not have the ability to submit proposed rates, it must request a temporary extension to continue using the most recently approved rates.
Managed Care Rate Transition for Nursing Facilities (NF). As of February 1, 2015, plans are required to pay contracted nursing homes either the existing FFS rate or a negotiated rate which allows the nursing home and the plan to engage in other financing arrangements. MLTC and MMMC plans will be reimbursed with an actuarial sound rate in compliance with 42 CFR §438.6. MLTC will develop a blended rate structure to promote community integration of institutional/Home and Community Based Services (HCBS). MMMC will develop a separate rate cell for the nursing home population and will include an HCBS "rate cohort" in its non-nursing home rate cells. The state shall submit an actuarial certification to CMS for approval of the April 1, 2015 rates that contains the following modifications:
1. MLTC transition rates must be phased out
2. Documentation must be submitted identifying the unique and cumulative impact of the various capitation rate withholds
3. Documentation must be submitted assessing gaps in rate setting for MLTC plans that necessitate funds to mitigate risks
Behavioral Health Services Furnished by MMMC, HIV SNPs, and HARPs. To the extent that an MCO is not able to meet the requirements for the management of the expanded behavioral health services, the MCO must contract with a managed care behavioral health organization to manage those services for enrollees. If the MCO network is unable to provide necessary medical services covered under the contract to a particular enrollee, the MCO will be required to cover these services out of network for the enrollee. This includes up to at least 2023 during which time the MCO will reimburse OMH ambulatory licensed and OASAS certified providers the FFS fee schedule to ensure continuity of care. MCOs must pay at least the FFS fee schedule for 24 months for the following services: Other Licensed Practitioner (OLP), Crisis Intervention, Community Psychiatric Support and Treatment (CPST), Psychosocial Rehabilitation (PSR), and Family and Youth Peer Support. After 90 days, the MCO may apply utilization review criteria to individuals under the care of non-participating providers. Plans will be required to authorize services and reimburse providers whether the behavioral health provider is contracted with the health plans or is an out of network provider. New York will reimburse MCOs for VFCA per diem/Preventive Residential Treatment Services on a non-risk basis subject to the non-risk UPL at 42 CFR 447.362. The MCO must pay the FFS fee schedule as long as the Preventive Residential Treatment Services are non-risk.

For SUD services and the delivery system changes associated with the new demonstration services and resulting state plan amendments including changes under the CMS IAP and with CMS approval, the state may require the MCOs through their contracts to adopt system-wide changes and rates to ensure that the innovations are adopted in a consistent manner statewide.
Independent Consumer Support Program. To support the beneficiary´s experience receiving and applying to receive long-term services and supports in a managed care environment, the state shall create and maintain a permanent independent consumer support program to assist beneficiaries in understanding the coverage model and in the resolution of problems regarding services, coverage, access and rights.
1. Organizational Structure. The Independent Consumer Support Program shall operate independently from any MRT MCO. Additionally, to the extent possible, the program shall also operate independently of the state Medicaid agency.
2. Accessibility. The services of the Independent Consumer Support Program shall be available to all Medicaid beneficiaries enrolled in MRT who are in need of LTSS (institutional, residential and community-based) and must be accessible through multiple entryways (e.g., phone, internet, office) and also provide outreach in the same manner as appropriate.
3. Functions. The Independent Consumer Support Program shall assist beneficiaries to navigate and access covered LTSS, including the following activities:
  1. offer beneficiaries support in the pre-enrollment state, such as unbiased health plan choice counseling and general program-related information;
  2. serve as an access point for complaints and concerns about health plan enrollment, access to services and other related matters;
  3. help enrollees understand the fair hearing, grievance and appeal rights and processes within the health plan and at the state level, and assist them through the process if needed/requested; and
  4. conduct trainings with MRT MCO and providers on community-based resources and supports that can be linked with covered plan benefits.
4. Staffing. The Independent Consumer Support Program must employ individuals who are knowledgeable about the state´s Medicaid programs; beneficiary protections and rights under Medicaid managed care arrangements; and the health and service needs of persons with complex needs, including those with a chronic condition, disability, and cognitive or behavioral needs.
5. Data Collection and Reporting. The Independent Consumer Support Program shall track the volume and nature of beneficiary contacts and the resolution of such contacts on a schedule and manner determined by the state, but no less frequently than quarterly.
6. Geographic expansion of MLTC and LTSS in MMMC. In any geographic location where the state is mandating MLTC or LTSS in MMMC, the state must have the Independent Consumer Support Program in place at least 30 days prior to enrollment procedures for that geographic location.
Revision of the State Quality Strategy. The state must update its Comprehensive Quality Strategy (CQS) to reflect all managed care plans operating under MMMC including HIV SNP, MLTC and HARP programs proposed through this demonstration and submit to CMS for approval within 90 days of approval of the most recent amendment. The state must obtain the input of recipients and other stakeholders in the development of its revised CQS and make the CQS available for public comment. The state must revise the CQS whenever significant changes are made, including changes through this demonstration. Pursuant to STC 5 in Section VIII the state must also provide CMS with annual reporting on the implementation and effectiveness of the updated CQS, as it impacts the demonstration. The CQS must also address the following elements:
1. The state´s goals for improvement, identified through claims and encounter data, quality metrics and expenditure data. The goals should align with the three-part aim but should be more specific in identifying specific pathways for the state to achieve these goals.
2. The specific quality metrics for measuring improvement in the goals. The metrics should be aligned with the Medicaid and CHIP adult and child core measures and should also align with other existing Medicare and Medicaid federal measure sets where possible.
3. Metrics should be measured at the following levels of aggregation: the state Medicaid agency, each managed care entity, and each direct health services provider. The state will work with CMS to further define what types of metrics will be measured for direct service providers.
4. The specific methodology for determining benchmark and target performance on these metrics for each aggregated level identified above (state, plan and provider).
5. MLTSS essential elements as defined in the May 21, 2013 CMS Information Bulletin to its MMMC quality reporting system (QARR).
6. The specific methodology for determining ongoing compliance with HCBS settings qualities.
Required Components of the State Quality Strategy for LTSS. The state must have a quality strategy specifically tailored to managed long-term services and supports. The quality strategy must address the following elements regarding the population utilizing long-term services and supports:
1. level of care assessments;
2. services planning;
3. health and welfare of enrollees;
4. MLTSS essential elements as defined in the May 21, 2013 CMS Information Bulletin to its MMMC quality reporting system (QARR); and
5. the specific methodology for determining ongoing compliance with HCBS settings qualities.
Required Monitoring Activities by the State and/or EQRO. The state´s EQR process for the MMMC and MLTC plans shall meet all the requirements of 42 CFR §438 Subpart E. In addition, the state, or its EQRO shall monitor and annually evaluate the MCO/PIHPs performance on specific new requirements under mandatory enrollment of individuals utilizing long-term services and supports. The state shall provide an update of the processes used to monitor the following activities as well as the outcomes of the monitoring activities within the annual report in STC 5 in Section VIII. The new requirements include, but are not limited to the following:
1. MLTC Plan Eligibility Assessments. To ensure that approved instruments are being used and applied appropriately and as necessary, and to ensure that individuals being served with LTSS meet the MLTC program eligibility requirements for plan enrollment. The state will also monitor assessments conducted by the plan where individuals are deemed ineligible for enrollment in an MLTC plan.
2. Service Plans. To ensure that MCOs are appropriately creating and implementing service plans based on the enrollee´s identified needs.
3. MCO credentialing and/or verification policies. To ensure that LTSS services are provided by qualified providers.
Access to Care, Network Adequacy and Coordination of Care Requirements for Long Term Services and Supports (LTSS). The state shall set specific access and coordination requirements for MCO. These standards should take into consideration individuals with special health care needs, out of network requirements if a provider is not available within the specific access standard, ensuring choice of provider with capacity to serve individuals, time/distance standards for providers who do not travel to the individual´s home, and physical accessibility of covered services. The MLTC or MMMC plan is not permitted to set these standards.
Demonstrating Network Adequacy. Annually, each MCO must provide adequate assurances that it has sufficient capacity to serve the expected enrollment in its service area and offers an adequate coverage of benefits as described for the anticipated number of enrollees in the service area.
1. The state must verify these assurances by reviewing demographic, utilization and enrollment data for enrollees in the demonstration as well as:
  1. the number and types of providers available to provide covered services to the demonstration population;
  2. the number of network providers accepting the new demonstration population; and
  3. the geographic location of providers and demonstration populations, as shown through GeoAccess, similar software or other appropriate methods.
2. The state must submit the documentation required in subparagraphs (i) - (iii) above to CMS with each annual report.
3. Enrollees and their representatives must be provided with reference documents to maintain information about available providers and services in their plans.
Advisory Committee as required in 42 CFR §438. The state must maintain for the duration of the demonstration a managed care advisory group comprised of individuals and interested parties appointed pursuant to state law by the Legislature and Governor. To the extent possible, the state will attempt to appoint individuals qualified to speak on behalf of seniors and persons with disabilities, including individuals with developmental disabilities, regarding the impact and effective implementation of the demonstration on individuals receiving LTSS.
Health Services to Native Americans Populations. The plan currently in place for patient management and coordination of services for Medicaid-eligible Native Americans developed in consultation with the Indian tribes and/or representatives from the Indian health programs located in participating counties shall continue in force for this extension period.

|top of section| |top of page|

VII. DELIVERY SYSTEM REFORM PROGRAM DESCRIPTION AND OBJECTIVES

Medicaid Redesign Team (MRT)

Background
- The purpose of this demonstration element is to describe a structure under which the federal government will provide up to $8 billion in new federal funds for all Medicaid Redesign Team (MRT) activities April 14, 2014 through March 31, 2020. The purpose of one component of MRT, the Delivery System Reform Incentive Payment (DSRIP) program, is to provide incentives for Medicaid providers to create and sustain an integrated, high performing health care delivery system that can effectively and efficiently meet the needs of Medicaid beneficiaries and low income uninsured individuals in their local communities by improving care, improving health and reducing costs. Up to $6.42 billion of the new MRT funding is available for DSRIP payments to providers. Up to $500 million in temporary, time limited, funding was available from an Interim Access Assurance Fund (IAAF) for payments to providers to protect against degradation of current access to key health care services in the near term, but this authority expired as of December 31, 2014. And, up to $1.08 billion in federal funding for other Medicaid Redesign purposes through State Plan Amendments is specifically designated for other Medicaid redesign initiatives or as described in the protocols. These initiatives must expand the capacity of the state´s provider network, facilitate delivery system reform, or enhance the ability of the state to monitor and oversee service delivery.
  
  The DSRIP program is focused on the following goals: (1) safety net system transformation at both the system and state level; (2) accountability for reducing avoidable hospital use and improvements in other health and public health measures at both the system and state level; and (3) efforts to ensure sustainability of delivery system transformation through leveraging managed care payment reform.
  1. Safety Net System Transformation. The DSRIP funds provider incentive payments to reward safety net providers when they undertake projects designed to transform the systems of care that support Medicaid beneficiaries and low income uninsured by addressing three key elements, which must be reflected in all DSRIP projects proposed by safety net providers participating in DSRIP (referred to as "Performing Provider Systems"). DSRIP projects will be designed to meet and be responsive to community needs while ensuring overall transformation objectives are met. As such, all projects must include the following elements, whose core components and associated outcome measures are further described in the DSRIP Strategies Menu and Metrics (Attachment J):
    1. Element 1: Appropriate Infrastructure. The DSRIP will further the evolution of infrastructure and care processes to meet the needs of their communities in a more appropriate, effective and responsive fashion to meet key functional goals. This will include changes in the workforce. Infrastructure evolution must support the broader goals of DSRIP, and key outcomes reflect the kinds of infrastructure to be supported under DSRIP. Appropriate infrastructure should ensure access to care, particularly to outpatient resources as well as effective care integration. In support of linking settings, the transforming infrastructure should place more emphasis on outpatient settings. Also, critical services such as care coordination may need to be expanded to meet the broad needs of the population served.
      
      Indicators related to this objective are included in the System Transformation Milestones (Domain 2) described in more detail in DSRIP Strategies Menu and Metrics (Attachment J). Because many of these indicators are difficult to benchmark, the state will be accountable for ensuring that these indicators are moving overall in the right directions across all systems as part of the statewide accountability described in STC 14(f) of this section.
    2. Element 2: Integration across settings. The DSRIP will further the transformation of patient care systems to create strong links between different settings in which care is provided, including inpatient and outpatient settings, institutional and community-based settings, and importantly behavioral and physical health providers. The goal will be to coordinate and provide care for patients across the spectrum of settings in order to promote health and better outcomes, particularly for populations at risk, while also managing total cost of care. The DSRIP will fund projects that include new and expanded care coordination programs, other evidence-based, data driven interventions and programs focused on key health and cost drivers and opportunities for providers to share information and learn from each other.
      
      Key outcomes to be measured are expected to reflect this ongoing transformation.
      
      Integration across settings will create alignments between providers. The DSRIP will include restructuring payments to better reward providers for improved outcomes and lower costs.
      
      Indicators related to this objective are included in the Clinical Improvement Milestones (Domain 3) described in more detail in DSRIP Strategies Menu and Metrics (Attachment J). Each system will be accountable for these indicators, and in addition, because the state should also work to support this goal, the state will also be accountable for statewide performance on these outcomes as described in STC 14(g) of this section.
    3. Element 3: Assuming responsibility for a defined population. The DSRIP projects will be designed in ways that promote integrated systems assuming responsibility for the overall health needs of a population of Medicaid beneficiaries and low-income uninsured people, not simply responding to the patients that arrive at the doors of a hospital. The state will approve a defined population for each DSRIP project based on geographic and member service loyalty factors, as described in DSRIP Program Funding and Mechanics Protocol (Attachment I). Safety net providers may propose to develop integrated systems that target the individuals served by a set of aligned community-based providers, or more ambitious systems to tackle accountability for an entire geographic population. Patient and beneficiary engagement through tools including community needs assessment and responsiveness to public health needs will be an important element of all DSRIP projects.
      
      Each indicator used to determine DSRIP awards should reflect a population, rather than the patients enrolled in a particular intervention. In addition, DSRIP Performing Provider Systems will be required to report on progress on priorities related to the Prevention Agenda as included in the Population-wide Strategy Implementation Milestones (Domain 4) described in more detail in DSRIP Strategies Menu and Metrics (Attachment J).
    4. Element 4: Procedures to reduce avoidable hospital use: guidepost for statewide reform. New York has identified a statewide goal of reducing avoidable hospital use and improving outcomes in other key health and public health measures. Effectively reducing avoidable hospital use requires alignment of outpatient and inpatient settings, requires systems that can take responsibility for a population, and requires investments in key infrastructure--and so this is a guidepost that can ensure that these transformations are aligned with our shared goals of better health, and better care at lower cost.
      
      Consistent with the fact that this is an integral guidepost to system transformation, key improvement outcomes for avoidable hospital use and improvements in other health and public health measures will be included for each project, and the state will be held accountable for these measures as part of the statewide accountability described in STC 14 (f) of this section.
    5. Element 5: State managed care contracting reforms to establish and promote DSRIP objectives. The state must also ensure that its managed care payment systems recognize, encourage and reward positive system transformation. To fully accomplish DSRIP goals and ensure sustainability of the initiatives supported by this demonstration, as a condition of receiving DSRIP project funding, the state shall develop and execute payment arrangements and accountability mechanisms with its managed care contractors. These payment and accountability changes, described further in STC 37 of this section, must be reflected in the state´s approved state plan and managed care contracts, and are funded through the approved state plan (without separate DSRIP funding). These changes are a condition for overall DSRIP project funding to be released.
      
      This goal will also be monitored as part of the statewide accountability test described in STC 14(f) of this section and will be tracked not at a DSRIP project level, but at the state level. The state must ensure state payments to managed care plans reflect and promote the establishment and continuation of integrated service delivery systems and procedures to reduce avoidable hospital use and ensure improvements in other health and public health measures.
  2. State and Provider Accountability. Overall DSRIP project funding is available up to the amounts specified in the special terms and conditions. Such funding is subject to the Performing Provider System meeting ongoing milestones established pursuant to this demonstration, and the state meeting overall state milestones as described in the STCs and DSRIP Program Funding and Mechanics Protocol (Attachment I). In addition, statewide achievement of performance goals and targets must be achieved and maintained for full access to the funding level as specified in the STCs. Specific reductions from statewide funds are taken from the state starting in Year 3 accordance with STC 14(h) of this section if these targets are not achieved.
    
    Individual projects are awarded based on the merit of the proposal itself, its support of the overall DSRIP goals, and the projected breadth and depth of the impact on Medicaid beneficiaries. Public transparency, a process that allows for community input, and independent expert evaluation are critical to the approval and funding levels for each project.
    
    It should be noted that federal funding for DSRIP activities is limited in any phase of the demonstration period to the amounts set forth in this demonstration authority, subject to all of the reductions based on milestones, even if the state expenditures exceed the amount for which federal funding is available.
Interim Access Assurance Fund (IAAF)
- Temporary, time limited, funding is available from an IAAF to protect against degradation of current access to key health care services in the near term, and the expenditure authority expired on December 31, 2014. The IAAF is available to provide supplemental payments that exceed upper payment limits, DSH limitations, or state plan payments, to ensure that current trusted and viable Medicaid safety net providers, according to criteria established by the state consistent with these STCs, can fully participate in the DSRIP, transformation without unproductive disruption. The IAAF is authorized as a separate funding structure from the DSRIP program to support the ultimate achievement of DSRIP goals. To the extent available funds are not expended in this time-limited IAAF, they are available for the DSRIP program itself. In addition, a separate fund is authorized to make DSRIP project design grants to providers. The IAAF and the design grant funds are both part of the overall DSRIP total funding.
  1. Interim Access Assurance Fund. To protect against degradation of current access to key health care services, limit unproductive disruption, and avoid gaps in the health delivery system, New York is authorized to make payments for the financial support of selected Medicaid providers.
    1. Limit on FFP. New York may expend up to $500 million in FFP for Interim Access Assurance payments for the period from the date of approval of the IAAF expenditure authority until December 31, 2014. Contingent upon renewal of the demonstration, the authority could be extended until March 31, 2015. To the extent available funds are not expended in this time-limited IAAF, they are available for the DSRIP program itself.
    2. Funding. The non-federal share of IAAF payments may be funded by state general revenue funds and transfers from units of local government that are compliant with section 1903(w) of the Act. Any IAAF payments must remain with the provider receiving the payment to be used for health care related purposes and may not be transferred back to any unit of government, directly or indirectly, or redirected for other purposes. The IAAF payments received by providers cannot be used for the non-federal share of any expenditures claimed under a federally supported grant.
  2. Interim Access Assurance Fund Requirements.
    1. The state will make all decisions regarding the distribution of IAAF payments to ensure that sufficient numbers and types of providers are available to Medicaid beneficiaries in the geographic area to provide access to care for Medicaid and uninsured individuals while the state embarks on its transformation path. The IAAF payments shall be limited to providers that serve significant numbers of Medicaid individuals, and that the state determines have financial hardship in the form of financial losses or low margins. In determining the qualifications of a safety net provider for this program and the level of funding to be made available, the state will take into consideration both whether the funding is necessary (based on current financial and other information on community need and services) to provide access to Medicaid and uninsured individuals. The state will also seek to ensure that IAAF payments supplement but do not replace other funding sources.
    2. Before issuing any payments to providers, the state must post on its Website a list of qualifications that providers must meet to receive payments under this section, provide an opportunity for public comment for at least 14 days, and consider such comments. On the day the proposed qualifications list is posted, the state must provide to CMS the URL where the list can be found. The state must take the public comments into account when qualifying providers and distributing funds from this account.
    3. Following the end of the public comment period in (ii), the state will initiate an open application period of at least 14 days duration for providers to submit applications.
    4. If a provider otherwise meeting the qualifications of this section is also receiving funds through the state´s vital access program, or any other supplemental payment program for which the federal government provides matching funds, or Medicaid disproportionate share hospital payments, the state must assure CMS of non-duplication. As part of the reporting requirements described in (iii) below, the state assures that the payment information for the IAAF will be maintained, as the reporting information is subject to CMS audit. A provider may receive both funding through this special fund and a planning grant as part of the DSRIP program.
  3. Reporting.
    1. Within 10 days of initiating payments under this section to a provider, the state must submit a report to CMS that states the total amount of the payment or payments, the amount of FFP that the state will claim, the source of the non-federal share of the payments, and documentation of the needs and purposes of the funds to assure CMS of non-duplication. The state should document all other Medicaid payments (e.g. base, supplemental, VAP, DSH) the provider receives to demonstrate that existing payments are not sufficient to meet financial needs of the providers.
    2. In each quarterly progress report, the state will include a summary of all payments under this section made during the preceding quarter, including all information required in (A), and attach copies all reports submitted under (A) for payments made during the quarter.
    3. When reporting payments under this section on the CMS-64, the state must include in Form CMS-64 Narrative a table that lists all payments by date, provider, and amount (broken down by source), and a reference to the quarterly progress report(s) where the payments and all of their required supporting documentation is presented.
  4. IAAF payments. The IAAF payments are not direct reimbursement for expenditures or payments for services. Payments from the IAAF are not considered patient care revenue and shall not be offset against disproportionate share hospital expenditures or other Medicaid expenditures that are related to the cost of patient care (including stepped down costs of administration of such care) as defined under these STCs, and/or under the state plan.

Delivery System Reform Incentive Payment (DSRIP) Fund. The terms and conditions in Section c apply to the State´s exercise of Expenditure Authority 9: Expenditures Related to the Delivery System Reform Incentive Payment (DSRIP) Fund. These requirements are further elaborated by Attachment I, "NY DSRIP Program Funding and Mechanics Protocol," Attachment J "NY DSRIP Strategies Menu and Metrics," and Attachment K "DSRIP Operational Protocol." For purposes of this section, the DSRIP program will have its own DSRIP demonstration years (DDY) and any reference to DDY is in reference to the DSRIP portion of the Partnership Plan demonstration and not to the DYs for the entire MRT demonstration. DSRIP funding for demonstration year DDY 1 through DDY 5 is contingent on renewal of the demonstration no later than December 31, 2014 and the revision of Attachments I, J and K based on the pre-implementation activities described in this section.

As described further below, DSRIP funding is available to Performing Provider Systems that consist of safety net providers whose project plans are approved and funded through the process described in these STCs and who meet particular milestones described in their approved DSRIP project plans. DSRIP project plans are based on the evidenced-based projects specified in the DSRIP Strategies Menu and Metrics (Attachment J) and are further developed by Performing Provider Systems to be directly responsive to the needs and characteristics of the low-income communities that they serve and to achieve the transformation objectives furthered by this demonstration.

Table 7 shows the definitions of DDY and correspondence with demonstration DYs.

Table 7: DSRIP Demonstration Years
DDY	Time Period	Demonstration DY
0	04/14/2014-03/31/2015	DY 16
1	04/01/2015-03/31/2016	DY 17
2	04/01/2016-03/31/2017	DY 18
3	04/01/2017-03/31/2018	DY 19
4	04/01/2018-03/31/2019	DY 20
5	04/01/2019-03/31/2020	DY 21

Health Homes. This component is to support health homes with building the infrastructure necessary to properly scale up the state´s capability to better assist patients with multiple chronic illness, serious mental health and/or HIV, as described in the State Plan Amendment # 14-0016 approved on March 10, 2015 for the amount specified in Table 8 of this section.
Behavioral Health (BH) Home and Community Based Services (HCBS). This component is to fund the BH HCBS available to eligible HARP and HIV SNP enrollees (listed in Attachment D). These services are designed to assist high needs individuals with serious mental illness and substance use disorders in remaining in home and community-based settings and achieving recovery-oriented outcomes.
MLTC Strategy. The MLTC Workforce strategy includes initiatives to retrain and recruit professionals in the long-term care sector. The state may not claim for MLTC Strategies until CMS approves revisions to Attachment I.

Safety Net Definition. The definition of safety net provider for hospitals will be based on the environment in which the Performing Provider System operates. Below is the safety net definition:
1. A hospital must meet one of the following criteria to participate in a Performing Provider System:
  1. Be either a public hospital, Critical Access Hospital or Sole Community Hospital
  2. Pass the two tests described below.
    1. At least 35 percent of all patient volume in their outpatient lines of business must be associated with Medicaid, uninsured and Dual Eligible individuals
    2. At least 30 percent of inpatient treatment must be associated with Medicaid, uninsured and Dual Eligible individuals
  3. Serve at least 30 percent of all Medicaid, uninsured and Dual Eligible members in the proposed county or multi-county community. The state will use Medicaid claims and encounter data as well as other sources to verify this claim. The state reserves the right to increase this percentage on a case by case basis so as to ensure that the needs of each community´s Medicaid members are met.
2. Non-hospital-based providers, not participating as part of a state-designated health home, must have at least 35 percent of all patient volume in their primary lines of business and must be associated with Medicaid, uninsured and Dual Eligible individuals.
3. Vital Access Provider Exception: The state will consider exceptions to the safety net definition on a case-by-case basis if it is deemed in the best interest of Medicaid members. Any exceptions that are considered must be approved by CMS and must be posted for public comment 30 days prior to application approval. Three allowed reasons for granting an exception are:
  1. A community will not be served without granting the exception because no other eligible provider is willing or capable of serving the community.
  2. Any hospital is uniquely qualified to serve based on services provided, financial viability, relationships within the community, and/or clear track record of success in reducing avoidable hospital use.
  3. Any state-designated health home or group of health homes.
4. Non-qualifying providers can participate in Performing Providers Systems. However, non-qualifying providers are eligible to receive DSRIP payments totaling no more than 5 percent of a project´s total valuation. CMS can approve payments above this amount if it is deemed in the best interest of Medicaid members attributed to the Performing Provider System.
Performing Provider Systems. The safety net providers that are funded to participate in a DSRIP project are called "Performing Provider Systems." Performing Provider Systems that complete project milestones and measures as specified in Attachment J, "DSRIP Strategies Menu and Metrics", are the only entities that are eligible to receive DSRIP incentive payments.
Two DSRIP Pools. Performing Provider Systems will be able to apply for funding from one of two DSRIP pools: Public Hospital Transformation Fund and Safety Net Performance Provider System Transformation Fund.
1. The Public Hospital Transformation Fund will be open to applicants led by a major public hospital system. The public hospital systems allowed to participate in this pool include:
  1. Health and Hospitals Corporation of New York City
  2. State University of New York Medical Centers
  3. Nassau University Medical Center
  4. Westchester County Medical Center
  5. Erie County Medical Center
2. The Safety Net Performance Provider System Transformation Fund would be available to all other DSRIP eligible providers.
3. Allocation of funds between the two pools will be determined after applications have been submitted, based on the valuation of applications submitted to each pool. The valuation framework is described in STC 8 of this section and will be further specified in the Program Funding and Mechanics Protocol.
4. There is also a Performance Pool within the two DSRIP pools, as described in the Program Funding and Mechanics Protocol (Attachment I).
Coalitions and Attributed Population. Major public general hospitals and other safety net providers are strongly required to form coalitions that apply collectively as a single Performing Provider System. Coalitions will be evaluated on performance on DSRIP milestones collectively as a single Performing Provider System. Coalitions are subject to the following conditions in addition to the requirements specified in the Program Funding and Mechanics Protocol:
1. Coalitions must designate a lead coalition provider who will be held responsible under the DSRIP for ensuring that the coalition meets all requirements of Performing Provider Systems, including reporting to the state and CMS.
2. Coalitions must establish a clear business relationship between the component providers, including a joint budget and funding distribution plan that specifies in advance the methodology for distributing funding to participating providers. The funding distribution plan must comply with all applicable laws and regulations, including, but not limited to, the following federal fraud and abuse authorities: the anti-kickback statute (sections 1128B(b)(1) and (2) of the Act); the physician self- referral prohibition (section 1903(s) of the Act); the gainsharing civil monetary penalty (CMP) provisions (sections 1128A(b)(1) and (2) of the Act); and the beneficiary inducement CMP (section 1128A(a)(5) of the Act). CMS approval of a DSRIP plan does not alter the responsibility of Performing Provider Systems to comply with all federal fraud and abuse requirements of the Medicaid program.
3. Each Performing Providers System must, in the aggregate, identify a proposed population for DSRIP. The proposed population will be aligned with the population attribution methodology specified in the Program Funding and Mechanics Protocol. The attribution methodology will assure non-duplication of members between DSRIP Performing Providers Systems.
4. Each coalition must have a data agreement in place to share and manage data on system- wide performance.
Objectives. Performing Provider Systems will design and implement projects that aim to achieve each of the following objectives or sub-parts of objectives, which are elaborated further in the DSRIP Strategies Menu and Metrics (Attachment J). To put in the context of the overall three objectives below, each Performing Provider System is responsible for project activity that addresses the first two objectives, for a defined population as specified in the third objective.
1. The creation of appropriate infrastructure and care processes based on community need, in order to promote efficiency of operations and support prevention and early intervention.
2. The integration of settings through the cooperation of inpatient and outpatient, institutional and community-based providers, in coordinating and providing care for patients across the spectrum of settings in order to promote health and better outcomes, particularly for populations at risk, while managing total cost of care.
3. Population health management as described in the attribution section of the Program Funding and Mechanics Protocol.
Project Milestones. Progress towards achieving the goals specified above will be assessed by specific milestones for each project, which are measured by particular metrics that are further defined in the DSRIP Strategies Menu and Metrics (Attachment J). These milestones are organized into the following domains:
1. Project progress milestones (Domain 1). Investments in technology, tools, and human resources that will strengthen the ability of the Performing Provider Systems to serve target populations and pursue DSRIP project goals. Performance in this domain is measured by a common set of project progress milestones, which will include milestones related to the monitoring of project spending and post-DSRIP sustainability. This includes at least semi-annual reports on project progress specific to the Performing Provider System´s DSRIP project and its Medicaid and uninsured patient population.
2. System transformation milestones (Domain 2). As described further in the Project Menu, this includes outcomes that reflect the four subparts of the goal on system transformation, including measures of inpatient/ outpatient balance, increased primary care/community-based services utilization, and rates of global capitation, partial capitation and bundled payment of providers by Medicaid managed care plans, and measures for patient engagement.
3. Clinical improvement milestones (Domain 3): As described further in the Project Menu, this domain includes metrics that reflect improved quality of care for Medicaid beneficiaries; including the goal of reducing avoidable hospital use and improvements in other health and public health measures. Payment for performance on these outcome milestones will be based on an objective demonstration of improvement over a baseline, using a valid, standardized method. Systems that are already high performers on these metrics, with the exception of avoidable hospitalization metrics, before initiation of projects must either explore alternative projects or align with lower performing providers such that the system as a whole has adequate room for improvement (as defined in DSRIP Program Funding and Mechanics Protocol (Attachment I).
4. Population-wide Strategy Implementation Milestones (Domain 4). DSRIP Performing Provider Systems will be responsible for reporting on progress on strategies they have chosen related to the Prevention Agenda as identified in DSRIP Strategies Menu and Metrics (Attachment J) for relevant populations as identified in DSRIP Program Funding and Mechanics Protocol (Attachment I) and as approved in their project plan.
DSRIP Project Plan. Performing Provider Systems must develop a DSRIP project plan that is based on one or more of the projects specified in the DSRIP Strategies Menu and Metrics (Attachment J) and complies with all requirements specified in the DSRIP Program Funding and Mechanics Protocol. Performing Provider Systems should develop DSRIP project plans, while leveraging community needs, including allowing community engagement during planning, to sufficiently address the delivery system transformation achievement that is expected from their projects. DSRIP project plans will be provided in a structured format developed by the state and approved by CMS and must be tracked by the state over the duration and close out of the program. DSRIP project plans must be approved by the state and may be subject to additional review by CMS, DSRIP project plans must include the following elements:
1. Rationale for Project Selection.
  1. Each DSRIP project plan must identify the target populations, program(s), and specific milestones for the proposed project, which must be chosen from the options described in the approved DSRIP Strategies Menu and Metrics.
  2. Goals of the project plan should be aligned with each of the objectives as described in STC 6 of this section.
  3. Milestones should be organized as described above in STC 7 of this section reflecting the three overall goals and subparts for each goal as necessary.
  4. The project plan must describe the need being addressed and the starting point (including baseline data consistent with the agreement between CMS and the state) of the Performing Provider System related to the project. The starting point of the project plan must be after April 1, 2015.
  5. Based on the starting point the Performing Provider System must describe its 5-year expected outcome for each of the domains described in STC 7 of this section. Supporting evidence for the potential for the interventions to achieve these changes should be provided in support of this 5-year projection for achievement in the goals of this DSRIP.
  6. The DSRIP Project Plan shall include a description of the processes used by the Performing Provider System to engage and reach out to stakeholders, including a plan for ongoing engagement with the public, based on the process described in the Operational Protocol (Attachment K).
  7. Performing Provider Systems must demonstrate how the project will transform the delivery system for the target population and do so in a manner that is aligned with the central goals of DSRIP, and in a manner that will be sustainable after DDY5. The projects must implement new, or significantly enhance existing health care initiatives; to this end, providers must identify the CMS and HHS funded delivery system reform initiatives in which they currently participate or in which they have participated in the previous five years, and explain how their proposed DSRIP activities are not duplicative of activities that are already or have recently been funded.
  8. The plan must include an approach to rapid cycle evaluation that informs the system in a timely fashion of its progress, how that information will be consumed by the system to drive transformation and who will be accountable for results, including the organizational structure and process to oversee and manage this process. The plan must also indicate how it will tie into the state´s requirement to report to CMS on a rapid cycle basis.
  9. The plan must contain a comprehensive workforce strategy. This strategy will identify all workforce implications - including employment levels, wages and benefits, and distribution of skills - and present a plan for how workers will be trained and deployed to meet patient needs in the new delivery system. Applicants will need to include workers and their representatives in the planning and implementation of their workforce strategy.
2. Description of Project Activities.
  1. Each plan must feature strategies from all domains described in STC 7 of this section and the DSRIP Strategies Menu and Metrics.
  2. For each domain of a project, there must be at least one associated outcome metric that must be reported in all years, years 1 through 5. The initially submitted DSRIP project plan must include baseline data on all measures, should demonstrate the ability to provide valid data and provide benchmarks for each measure. Baseline measurements should be based on the most recently available baseline data, as agreed to by CMS and the state.
3. Justification of Project Funding.
  1. The DSRIP project plan shall include a joint budget and funding distribution plan as provided for in DSRIP Program Funding and Mechanics Protocol (Attachment I) and a description of the Performing Provider System or provider coalition´s overall approach to valuing the project. Project valuations will be subject to a standardized analysis by the state as described below and further specified in the Program Funding and Mechanics Protocol.
  2. DSRIP project plans shall include any information necessary to describe and detail mechanisms for the state to properly receive intergovernmental transfer payments (as applicable and further described in the program funding and mechanics protocol).
Project Valuation. DSRIP payments are earned for meeting the performance milestones (as specified in each approved DSRIP project plan). The value of funding for each milestone and for DSRIP projects overall should be proportionate to its potential benefit to the health and health care of Medicaid beneficiaries and low-income uninsured individuals, and the potential costs of the project to the provider, as further explained in the Program Funding and Mechanics Protocol (Attachment I).
1. Maximum project valuation. As described further in the Program Funding and Mechanics Protocol, a maximum valuation for each project on the project menu shall be calculated based on the following valuation components as specified in the Program Funding and Mechanics Protocol (Attachment I).
  1. Index score of transformation potential. The state will use a standardized index to score each project on the project menu, based on its anticipated delivery system transformation. This index will include factors of anticipated transformation, such as potential for achieving the goals of DSRIP outlined in STC 6 of this section, expected cost savings, potential to reduce preventable events, capacity of the project to directly affect Medicaid and uninsured beneficiaries and robustness of evidence base. The index scoring process is described in the DSRIP Program and Funding and Mechanics Protocol and will be available for public comment in accordance with STC 10 of this section.
  2. Valuation benchmark. The project index score will be multiplied by a valuation benchmark in combination with the components below for all DSRIP projects in order to determine the maximum valuation for the project, as specified in the Program Funding and Mechanics Protocol (Attachment I). The valuation benchmark should be externally justified based on evidence for the value and scope of similar system transformations and delivery system reforms and may not be based on the total statewide limit on DSRIP funding described in STC 15 of this section. By no later than 15 days after the public comment period for initial DSRIP applications, the state will establish a state- wide valuation benchmark based on its assessment of the cost of similar delivery reforms. This valuation benchmark will be expressed in a per- member per month (PMPM) format and may not exceed $15 PMPM. Project valuation will be calculated by multiplying this valuation benchmark against the DSRIP Project Application Score and number of DSRIP months below.
  3. DSRIP Project Plan Application Score. Based on the Performing Provider System´s application, each project plan will receive a score based on the fidelity to the project description and likelihood of achieving improvement by using that project.
    1. Number of Attributed Beneficiaries. Number of beneficiaries attributed to each performing provider´s project plan
    2. Number of DSRIP Months. Number of DSRIP months that will be paid for under the DSRIP project plan.
2. Progress milestones and outcome milestones. A DSRIP project´s total valuation will be distributed across the milestones described in the DSRIP project plan, according to the specifications described in the Program Funding and Mechanics Protocol (Attachment I). An increasing proportion of DSRIP funding will be allocated to performance on outcome milestones each year, as described in DSRIP Program Funding and Mechanics Protocol (Attachment I).
3. Performance based payments. Performing Provider Systems may not receive payment for metrics achieved prior to the baseline period set by CMS and the State in accordance with these STCs and the funding and mechanics protocol and achievement of all milestones is subject to audit by CMS, the state, and the state´s independent assessor described in STC 10 of this section. The state shall also monitor and report proper execution of project valuations and funds distribution as part of the implementation monitoring reporting required under STC 12 of this section. In addition to meeting performance milestones, the state and performing providers must comply with the financial and reporting requirements for DSRIP payments specified in STC 13 of this section and any additional requirements specified in the Program Funding and Mechanics Protocol (Attachment I).
Pre-implementation activities. In order to authorize DSRIP funding for DDY 1 to 5, the state must meet the following implementation milestones according to the timeline outlined in these STCs and must successfully renew the demonstration according to the process outlined in STC 8 in Section III. Failure to complete these requirements will result in a state penalty, as described in paragraph 14 (h) below.
1. Project Design Grants. During DSRIP Year 0, the state may provide allotted amounts to providers for DSRIP Design Grants from a designated Design Grant Fund. These grants will enable providers to develop specific and comprehensive DSRIP Project Plans. New York may expend up to $100 million in FFP for the grant payments from the Design Grant Fund. Unspent funds will be carried over to DSRIP. DSRIP Project Design Grant payments count against the total amounts allowed for DSRIP under the demonstration.
  1. Submitting a proposal for a DSRIP Project Design Grant. Providers and coalitions must submit a DSRIP design proposal as an application for a design. The state will review proposals and award design grants at any time during the pre-implementation activities.
  2. Use of Design Grant Funds. The providers and coalitions that receive DSRIP project design grants must use their grant funds to prepare a DSRIP project plan to prepare the provider´s application for a DSRIP award. Providers and coalitions that receive DSRIP project design grants must submit a DSRIP application.
2. Public comment period. The state must engage the public and all affected stakeholders (including community stakeholders, Medicaid beneficiaries, physician groups, hospitals, and health plans) by publishing the development of the DSRIP Program Funding and Mechanics Protocol and DSRIP Strategies Menu and Metrics (Attachments I and J), including all relevant background material, and providing a public comment period that will be no less than 30 days that includes submission of comments through electronic means as well as public meetings across the State.
3. Allowable changes to DSRIP protocols. The state must post the public comments received and any technical modifications the state makes to the DSRIP Program Funding and Mechanics Protocol and DSRIP Strategies Menu and Metrics (Attachments I and J). Only changes to the protocol and menu that are related to the public comments will be allowed and incorporated into final protocols for DDY 1 to DDY 5. The state will submit the final protocols and menu and CMS will review and take action on the changes (i.e. approve, deny or request further information or modification) no later than 90 days after the state´s submission.
4. Baseline data on DSRIP measures. The state must use existing data accumulated prior to implementation to identify performance goals for performing providers. The state must identify high performance levels for all anticipated measures in order to ensure that providers select projects that can have the most meaningful impact on the Medicaid population, and may not select projects for which they are already high performers, with the exception of projects specifically focused on avoidable hospitalization.
5. Procurement of entities to assist in the administration and evaluation of DSRIP. The state will identify independent entities with expertise in delivery system improvement, including an independent assessor, an independent evaluator and monitoring any other administrative costs. The independent entities will work in cooperation with one another to do the following:
  1. Independent Assessor: Conduct a transparent review of all proposed DSRIP project plans and make project approval recommendations to the state.
  2. Independent Evaluator: Assist with the continuous quality improvement activities.
  3. Administrative Costs: Monitor administrative costs the state incurs associated with the management of DSRIP reports and other data.
    1. The state must describe the functions of each independent entity and their relationship with the state as part of its Operational Protocol (Attachment K)
    2. The state may elect to require IGTs to be used to fund the non-federal share of the administrative activities, as permitted under the state plan.
    3. Spending on the independent entities and other administrative cost associated within the DSRIP fund is classified as a state administrative activity of operating the state plan as affected by this demonstration. The state must ensure that all administrative costs for the independent entities are proper and efficient for the administration of the DSRIP Fund.
6. Submit evaluation plan. The state must submit an evaluation plan for DSRIP consistent with the requirements of STC 19 of this section no later than 120 days after award of the DSRIP program. Further, the state must identify an independent evaluator. The evaluation plan, including the budget and adequacy of approach to meet the scale and rigor of the requirements of STC 21 of this section, is subject to CMS approval. The state must submit a revised evaluation plan by May 31, 2016.
7. Update comprehensive quality strategy. The state must update its comprehensive quality strategy, defined in Section VI, to ensure the investment in DSRIP programs will complement and be supported by the state´s managed care quality activities and other quality improvements in the state, including the state´s Medicaid Redesign Team and Health Homes initiatives.
8. DSRIP Operational Protocol. The state shall submit for CMS approval a draft operational protocol for approving, overseeing, and evaluating DSRIP project grants no later than 90 days after the award of the Demonstration. The protocol is subject to CMS approval. The State shall provide the final protocol within 30 days of receipt of CMS comments. If CMS finds that the final protocol adequately accommodates its comments, then CMS will approve the final protocol within 30 days. This protocol will become an appendix to Attachment K of these STCs.
  1. The Operational Protocol, including required baseline and ongoing data reporting, independent assessor protocols, performing provider requirements, and monitoring/evaluation criteria shall align with the CMS approved evaluation design and the monitoring requirements in STC 34 of this section.
  2. The state shall make the necessary arrangements to assure that the data needed from the Performing Provider Systems, and data needed from other sources, are available as required by the CMS approved monitoring protocol.
  3. The Operational Protocol and reports shall be posted on the state Medicaid website within 30 days of CMS approval.
9. CMS Oversight of Pre-implementation Activities. CMS reserves the right to provide oversight over the state´s pre-implementation activities in order to document late submissions and missed deliverables without notice of a delay from the state. Notice of delay from of any deliverable must be received by CMS no less than 10 days before the due date of the deliverable. As part of CMS´ review of the state´s deliverables, CMS will assess completeness based on listed deliverable requirements in the STCs.
10. Updated DSRIP Program Funding and Mechanics Protocol (Attachment I) and DSRIP Strategies Menu and Metrics Protocol (Attachment J). Attachments I and J will be updated by December 31, 2016.
DSRIP proposal and project plan review. In accordance with the schedule outlined in these STCs and the process described further in the Program Funding and Mechanics Protocol (Attachment I), the state and the assigned independent assessor must review and approve DSRIP project plans in order to authorize DSRIP funding for DDY 1 and DDY 2 and must conduct ongoing reviews of DSRIP project plans as part of a mid-point assessment in order to authorize DSRIP funding for DDY 3, DDY4 and DDY5. The state is responsible for conducting these reviews for compliance with approved protocols. CMS reserves the right to review projects in which the state did not accept the finding of the independent assessor or other outlier projects, as specified in the Program Funding and Mechanics Protocol (Attachment I).
1. Review tool. The state will develop a standardized review tool that the independent assessor will use to review DSRIP project plans and ensure compliance with these STCs and associated protocols. The review tool will be available for public comment for a 30-day period according to the timeframe specified in the Program Funding and Mechanics Protocol (Attachment I). The review tool will define the relevant factors, assign weights to each factor, and include a scoring for each factor. Each factor will address the anticipated impact of the project on the Medicaid and uninsured populations consistent with the overall purpose of the DSRIP program.
2. Role of the Independent assessor. An independent assessor will review project proposals using the state´s review tool and consider anticipated project performance. The independent assessor shall make recommendations to the state regarding approvals, denials or recommended changes to project plans to make them approvable. This entity (or another entity identified by the state) will also assist with the mid-point assessment and any other ongoing reviews of DSRIP project plan.
3. Public comment. Project proposals will be public documents and subject to public comment. The public will have no less than 30 days from the date of project posting to submit comments for specific project proposals, according to the process described in the Operational Protocol (Attachment K). After the comment period for the projects closes, a method for which the public can continue to comment must remain available, to obtain feedback on the ongoing implementation of the projects. The state must periodically compile comments received over the life of the demonstration and ensure that responses to comments are provided and released for public view.
4. Mid-point assessment. During DDY 2, the state´s independent assessor shall assess project performance to determine whether DSRIP project plans merit continued funding and provide recommendations to the state. If the state decides to discontinue specific projects, the project funds may be made available for expanding successful project plans in DDY 3, DDY 4 and DDY 5, as described in the Program Funding and Mechanics Protocol (Attachment I).
Monitoring. With the assistance of the independent assessor, the state will be actively involved in ongoing monitoring of DSRIP projects, including but not limited to the following activities.
1. Review of milestone achievement. At least two times per year, Performing Provider Systems seeking payment under the DSRIP program shall submit reports to the state demonstrating progress on each of their projects as measured by project-specific milestones and metrics achieved during the reporting period. The reports shall be submitted using the standardized reporting form approved by the state and CMS. Based on the reports, the Independent Assessor will calculate the incentive payments for the progress achieved according to the approved DSRIP project plan. The Independent Assessor´s determination shall be considered final. The Performing Provider System shall have available for review by New York or CMS, upon request, all supporting data and back-up documentation. These reports will serve as the basis for authorizing incentive payments to Performing Provider Systems for achievement of DSRIP milestones.
2. Quarterly DSRIP Operational Protocol Report. The state shall provide quarterly updates to CMS and the public on the operation of the DSRIP program. The reports shall provide sufficient information for CMS to understand implementation progress of the demonstration and whether there has been progress toward the goals of the demonstration. The reports will document key operational and other challenges, to what they attribute the challenges and how the challenges are being addressed, as well as key achievements and to what conditions and efforts they attribute the successes.
3. Learning collaboratives. With funding available through this demonstration, the state will support regular learning collaboratives regionally and at the state level, which will be a required activity for all Performing Provider Systems, and may be organized either geographically, by the goals of the DSRIP, or by the specific DSRIP projects as described in the DSRIP Strategies Menu and Metrics (Attachment J). Learning collaboratives are forums for Performing Provider Systems to share best practices and get assistance with implementing their DSRIP projects. Learning collaboratives should primarily be focused on learning (through exchange of ideas at the front lines) rather than teaching (i.e. large conferences), but the state should organize at least one face-to-face statewide collaborative meeting a year. Learning collaboratives should be supported by a web site to help providers share ideas and simple data over time (which should not need to be developed from scratch). In addition, the collaboratives should be supported by individuals (regional "innovator agents") with training in quality improvement who can travel from site to site in the network to rapidly answer practical questions about implementation and harvest good ideas and practices that they systematically spread to others.
4. Rapid cycle evaluation. In addition to the comprehensive evaluation of DSRIP described in STC 22 of this section, the state will be responsible for compiling data on DSRIP performance after each milestone reporting period and summarizing DSRIP performance to-date for CMS in its quarterly reports. Summaries of DSRIP performance must also be made available to the public on the state´s website along with a mechanism for the public to provide comments.
5. Additional progress milestones for at risk projects. Based on the information contained in the Performing Provider System´s semiannual report or other monitoring and evaluation information collected, the state or CMS may identify particular projects as being "at risk" of not successfully completing its DSRIP project in a manner that will result in meaningful delivery system transformation. The state or CMS may require these projects to meet additional progress milestones in order to receive DSRIP funding in a subsequent semi-annual reporting period. Projects that remain "at risk" are likely to be discontinued at the midpoint assessment, described in STC 11 (d) of this section.
6. Annual discussion and site visits. In addition to regular monitoring calls, the State shall on an annual basis present to and participate in a discussion with CMS on implementation progress of the demonstration including progress toward the goals, and key challenges, achievements and lessons learned. The state and the independent assessor will conduct annual site visits of a subset of Performing Provider Systems to ensure continued compliance with DSRIP requirements. At its discretion, CMS may also conduct annual site visits to select Performing Provider Systems.
7. Application, review, oversight, and monitoring database. The state will ensure that there is a well maintained and structured database, containing as data elements all parts and aspects of Performing Provider Systems´ DSRIP project plans including the elements discussed in paragraph 8; independent assessor, state, and CMS review comments and scores; project planning, process, improvement, outcome, and population health milestones, with indicators of their required timing, incentive payment valuation, and whether or not they were achieved; and any other data elements required for the oversight of DSRIP. Along with the database, the state will develop software applications that will support:
  1. electronic submission of project plans by Performing Provider Systems;
  2. public comment on project plans;
  3. review of project plans by the independent assessor, state, and other independent participants in project plan review and scoring;
  4. electronic submission by Performing Provider Systems of their performance data;
  5. generation of reports, containing (at a minimum) the elements in STC 34 of this section, that can be submitted to CMS to document and support amounts claimed for DSRIP payments on the CMS-64;
  6. summaries of DSRIP project plans submissions, scoring, approval/denial, milestone achievement, and payments that can be accessed by the public;
  7. database queries, and export all or a portion of the data to Excel, SAS, or other software platforms; and
  8. On-line access rights for CMS.
Financial requirements applying to DSRIP payments generally.
1. The non-Federal share of Fund payments to providers may be funded by state general revenue funds, and transfers from units of local government consistent with federal law. However, Federal Participation received from Designated State Health Programs (DSHP), IAAF, Planning Grants, Administration, Health Homes, and DSRIP awards shall not be used as the non-federal share in claiming Federal Participation.
  
  Any DSRIP payment must remain with the provider specified in the DSRIP project plan and may not be transferred back to any unit of government, including public hospitals, either directly or indirectly. In the case of coalitions that are performing DSRIP projects collectively, the DSRIP funding will flow to the participating providers and/or the coalition coordinating entity according to the methodology specified in the DSRIP project plan but may not be transferred between coalition providers.
2. The state must inform CMS of the funding of all DSRIP payments to providers through a quarterly payment report to be submitted to CMS within 60 days after the end of each quarter, as required under STC 34 of this section. This report must identify the funding sources associated with each type of payment received by each provider. In addition, this report must identify and fully disclose all the underlying primary and secondary funding sources of the non-Federal share (including health care related taxes, intergovernmental transfers, general revenue appropriations, and any other mechanism) for each type of payment received by each provider.
3. The state will ensure that any lack of adequate funds from local sources will not result in lowering the amount, duration, scope or quality of Medicaid services available under the state plan or this demonstration. The preceding sentence is not intended to preclude the state from modifying the Medicaid benefit through the state plan amendment process.
4. The state may not claim FFP for DSRIP Payments until both the state and CMS, have concluded that the Performing Provider Systems have met the performance indicated for each payment. Performing Providers Systems´ reports must contain sufficient data and documentation to allow the state and CMS to determine if the Performing Provider Systems´ have fully met the specified metric and Performing Provider Systems must have available for review by the state or CMS, upon request, all supporting data and back-up documentation. FFP will be available only for payments related to activities listed in an approved DSRIP project plan.
5. Each quarter the State makes DSRIP Payments or IAAF payments and claims FFP, appropriate supporting documentation will be made available for CMS to determine the appropriate amount of the payments. Supporting documentation may include, but is not limited to, summary electronic records containing all relevant data fields such as Payee, Program Name, Program ID, Amount, Payment Date, Liability Date, Warrant/Check Number, and Fund Source. Documentation regarding the Funds revenue source for payments will also identify all other funds transferred to such fund making the payment. This documentation should be used to support claims made for FFP for DSRIP Payments that are made on the CMS-64.9 Waiver forms.
6. DSRIP Payments are not direct reimbursement for expenditures or payments for services. Payments from the DSRIP Fund are intended to support and reward Performing Provider Systems for improvements in their delivery systems that support the simultaneous pursuit of improving the experience of care, improving the health of populations, and reducing per capita costs of health care. Payments from the DSRIP Fund are not considered patient care revenue and shall not be offset against disproportionate share hospital expenditures or other Medicaid expenditures that are related to the cost of patient care (including stepped down costs of administration of such care) as defined under these Special Terms and Conditions, and/or under the State Plan.
7. DSRIP payments will be applied to the quarter in which the award was earned.

Limits on Federal Financial Participation.

Use of FFP. The state will receive up to a total of $8 billion FFP to support MRT activities: $6.92 billion for DSRIP, $500 million of which will be for the IAAF (which expired December 31, 2014), and the remaining amount to be authorized for other activities in accordance with a CMS approved protocol. This includes support to health homes authorized under SPA #14-0016 approved March 10, 2015 that establishes a rate add on to existing payment rates for health home services to distribute payments up to the amounts approved in the Table 8.
MRT Cap. The State can claim FFP for MRT expenditures in each DSRIP Year up to the limits shown in the table below. Each DSRIP Project Plan must specify the DSRIP Year to which each milestone pertains; all incentive payments associated with meeting the milestone must count against the annual limit for the DSRIP Year identified. The state or its contractor shall monitor and report proper execution of project valuations and funds distribution as part of the implementation monitoring and reporting required under STC 34 of this section.
One-year DSRIP funding carry-over. If a Performing Provider System does not fully achieve a metric in Domains 2, 3 or 4 that was specified in its approved DSRIP project plan for completion in a particular DSRIP year, the Performing Provider System must report on the missed metrics in the given DSRIP year. Performing Provider Systems that do not meet annual milestones for a given metric will not be eligible to receive incentive payments for the missed metrics in that given DSRIP year. Any funding that would have been allocated to the Performing Provider System during that DSRIP year will be placed in the performance pool fund to be redistributed to Performing Provider Systems that have exceeded their set performance benchmarks for that DSRIP year. When a Performing Provider System does not meet its DSRIP year performance metrics, the missed metrics milestone will be recalibrated based on the procedures in DSRIP Program Funding and Mechanics Protocol (Attachment I) for the next DSRIP year and the Performing Provider System will be eligible to receive payments from the DSRIP payment pool for that next year if it reaches the recalibrated milestone in that next DSRIP year.

Fund Allocations According to MRT Demonstration Year. Table 8 below shows funding limits for the life of the demonstration.

Changes to the amounts specified in Table 8 requires an amendment, following processes outlined in STC 7 of section III.
Sources of non-federal funding must be compliant with section 1903(w) of the Act and applicable regulations. To the extent that federal funds from any federal programs are received for the DSHP listed in Attachment I, they shall not be used as a source of non- federal share.

Table 8: Federal Financial Participation (in Millions)

	Year-0	Year-1	Year-2	Year-3	Year-4	Year-5	Total
Sources of Funding
Public Hospital IGT Transfers (Supports DSRIP IGT Funding for Public Performing Provider Transformation Fund, Safety Net Performance Provider System Transformation Fund, DSRIP, State Plan and Managed Care Services)	505.1	657.2	960.8	1533.7	1418	925.2	6000
State Appropriated Funds	134.3	429.8	614.5	497.4	249.3	74.6	2000
Total Sources of Funding	639.4	1087	1575.3	2031.1	1667.4	999.8	8000

Uses of Funding
DSRIP Expenditures	602.3	1049.1	1249.3	1698.3	1410.5	908.9	6918.5
Interim Access Assurance Fund (IAAF)	500	0	0	0	0	0	500
Planning Payments	69.9	0	0	0	0	0	69.9
Performance Payments	0	981.8	1144.3	1668.4	1379.5	874.7	6048.6
Administration	32.4	67.4	105	29.9	31	34.2	300

Health Home	37.1	37.9	20.3	0	0	0	95.3

MC Programming	0	0	305.7	332.8	256.8	90.9	986.2
Health Workforce MLTC Strategy	0	0	91.2	80.2	52.5	47.4	271.2
Home and Community Based Services/1915i Services	0	0	214.5	252.6	204.3	43.6	715
Total Uses of Funding	639.4	1087	1575.3	2031.1	1667.4	999.8	8000

Notwithstanding the limits in STC 1(a) and 15(a) in this section, to the extent that the state elects to limit supplemental payments to an institutional provider class otherwise authorized under its state plan in any state fiscal year during which the DSRIP demonstration is in effect, an amount equal to the federal share of the amount not paid to such providers, up to $600 million may be added to the overall MRT and DSRIP limits on federal funding. This election will be available only to the extent that the state does not increase the authorized levels of such supplemental payments, or initiate new supplemental payments, during the authorized demonstration period. The state must develop and use a tracking spreadsheet (following a format approved by CMS) to ensure that the amounts of the DSRIP increase do not exceed the amount of authorized but unpaid supplemental payments.
Statewide accountability. Beginning in DSRIP Year 3, the limits on DSHP funding and on total DSRIP payments described in paragraph (a) above may be reduced based on statewide performance, according to the process described in the Program Funding and Mechanics Protocol.
Statewide performance Statewide performance will be assessed on a pass or fail basis, for a set of 4 milestones.
1. Statewide performance on universal set of delivery system improvement metrics (as defined in Attachment J). Metrics for delivery system reform will be determined at a statewide level. Each metric will be calculated to reflect the performance of the entire state. Each of these statewide metrics will be assigned a direction for improving and worsening. This milestone will be considered passed in any given year if more metrics in these domains are improving on a statewide level than are worsening, as compared to the prior year as well as compared to initial baseline performance.
2. A composite measure of success of projects statewide on project-specific and population wide quality metrics. This test is intended to reflect the success of every project in achieving the goals that have been assigned to each project, including pay for reporting for certain outcome measures as specified in DSRIP Strategies Menu and Metrics (Attachment J). As described in DSRIP Program Funding and Mechanics Protocol (Attachment I), each metric that determines project level incentive payments for each project will be determined at the project level to be meeting the improvement standards. This statewide milestone will be considered passed in any given year if the number of metrics for each project that trigger an award as the improvement standards in DSRIP Program Funding and Mechanics Protocol (Attachment I) are greater than the number of metrics for each project that fail to trigger an award as per the improvement standard in DSRIP Program Funding and Mechanics Protocol (Attachment I).
3. Growth in statewide total Medicaid spending, including MRT spending, that is at or below the target trend rate (Measure applies in DDY4 and DDY5). The per member per month (PMPM) amounts will be adjusted to exclude growth in federal funding associated with the Affordable Care Act. The state will not be penalized if it uses these higher FMAP rates generated by the Affordable Care Act to reinvest in its Medicaid program.
  
  Growth in statewide total inpatient and emergency room spending that is at or below the target trend rate (Measure applies in DDY 3, DDY 4 and DDY 5).
  
  Both of the above measures will be measured on a PMPM basis by comparing data from the most recent state fiscal year with data from the state fiscal year that immediately precedes it, with applicable spending including both federal and non-federal shares combined. Such data must be verified and available for comparison within 90 days after the end of the state fiscal year, or the state will fail on this target. Adjustment outside of this time period will be made only to reflect mathematical calculation errors. Per member per month spending in each measure is determined by dividing statewide total spending by the number of person-months of Medicaid eligibility in the state for the state fiscal year. The most recent state fiscal year is the last state fiscal year ending prior to the start of the DSRIP year? For total Medicaid spending, the target trend rate is the ten-year average rate for the long-term medical component of the Consumer Price Index (as used to determine the state´s Medicaid Global Spending Cap for that year), for DDYs 4 and 5 only. For inpatient and emergency room spending the target trend rate is the ten-year average rate for the long-term medical component of the Consumer Price Index (as used to determine the state´s Medicaid Global Spending Cap for that year) minus 1 percentage points for DDY 3 and 2 percentage points for DDYs 4 and 5.
4. Implementation of the managed care plan, including targets agreed upon by CMS and the state after receipt of the managed care contracting plan in STC 38 of this section related to reimbursement of plans and providers consistent with DSRIP objectives and measures. These targets will include one associated with the degree to which plans move away from traditional fee for service payments to payment approaches rewarding value.

The state must achieve all four milestones to avoid DSRIP reductions. If the state fails on any of the 4 targets, the amount of the reduction is as described in table 9.

The state must pass 50 percent of the inpatient/emergency room spending reduction goals to avoid DSHP penalties. This will be the sole test for any DSHP penalty. The amount of the potential reduction is set as outlined in Table 9.

Table 9: DSHP and DSRIP Penalties
	DY-3	DY-4	DY-5
DSHP Penalty	$24.87 (5%)	$24.93 (10%)	$14.93 (20%)
DSRIP Penalty	$76.68 (5%)	$141.80 (10%)	$185.04 (20%)

If DSRIP and DSHP penalties are applied, the state reduces funds in an equal distribution of projects and will not affect the high performance fund.

Designated State Health Programs (DSHPs). To the extent that the state increases its Medicaid expenditures through its DSRIP program and achieves the metrics that are a condition for DSRIP payment, the state may claim federal matching funding for certain DSHP expenditures to support the initial investment costs of the DSRIP program. The expectation, which will be addressed in the demonstration evaluation, is that long-term savings achieved through the DSRIP investment will offset the amount of time limited federal DSHP funding. DSHP expenditures cannot exceed the amount spent on DSRIP and DSHP funding will also be subject to the annual and total DSHP spending limits described below. DSHP funding is at-risk at the statewide level based on the state´s ability to meet DSRIP spending reduction goals, including but not limited to meeting inpatient/emergency room spending reduction goals. DSHP funding will be phased down over the demonstration period. No payments will be available for expenditures that are claimed under Medicaid or are reimbursed by third parties. DSHP expenditures maybe claimed following procedures and subject to limits as described below.

Limit on FFP for DSHP. The amount of FFP that the state may receive for DSHP may not exceed the limit described below. If upon review, the amount of FFP received by the state is found to have exceeded the applicable limit, the excess must be returned to CMS as a negative adjustment to claimed expenditures on the CMS-64.

Table 10: Limit on FFP for DSHP ($ Millions)
Year-0	Year-1	Year-2	Year-3	Year-4	Year-5	Total
$134.30	$429.80	614.5	497.4	249.3	74.6	2000

The FFP limit for 2014 is the lowest of the following amounts:

$188 million
Combined non-federal share of IAAF Payments, DSRIP Project Design Grant payments and DSRIP administrative costs in 2014
Federal share of total matchable DSHP expenditures in 2014 as outlined below

DSHP List 1. The state may claim FFP in support of DSRIP for List 1 DSHP expenditures (excluding expenditures that are otherwise eligible for federal support or that are eligible for payment by third-party payers) made after March 31, 2014. The state may not claim FFP until after the date on which CMS has approved a DSHP Claiming Protocol for the specific DSHP.
1. Health Care Reform Act programs including:
  1. AIDS Drug Assistance. The ADAP (AIDS Drug Assistance Program) provides life-saving medications; ADAP Plus provides HIV primary care services; the Home Care Program provides care in the home; and the ADAP plus Insurance Continuation (APIC) program provides assistance in paying health insurance premiums and offers comprehensive coverage in a cost-effective manner. The program improves health outcomes for Medicaid beneficiaries with HIV/AIDS.
  2. Tobacco Use Prevention and Control. The New York State Department of Health Tobacco Control Program (NY TCP) goal is to establish a tobacco-free for all New Yorkers and works towards that goal by implementing a policy-driven, population and evidence-based approach designed to prevent youth from smoking and motivate adult smokers to quit. The programs long-term impact is to reduce tobacco- related illness in Medicaid beneficiaries.
  3. Health Workforce Retraining. This program trains health care workers for positions and occupations with shortages of health care workers and provides employment for health care workers who need new jobs and/or new skills because of changes in the health care delivery system. The program supports a major component of the demonstration by improving provider networks available to Medicaid and low-income individuals.
2. State Office on Aging programs including:
  1. Community Services for the Elderly. This county-administered program provides a broad range of community-based supportive services to allow frail, low-income elderly (non-Medicaid eligible) to maintain their independence and remain in the community, thus avoiding the need for institutional care and ultimately avoid requiring Medicaid financing.
  2. Expanded In-Home Services to the Elderly. This county-administered program provides in-home services for the functionally impaired low-income elderly (non-Medicaid eligible) to allow them to remain in the community, thus avoiding the need for institutional care and the need to enroll in Medicaid.
3. Office of Children and Family Services, Committees on Special Education direct care programs. Committees on Special Education (CSE) are the primary placing system for providing special education services for children with educational disabilities. Placements are made by CSEs into day and residential schools. These services can improve health outcomes in vulnerable populations.
4. State Department of Health, Early Intervention Program Services (EPAC). Part C of the Individuals with Disabilities Education Act established the Early Intervention Program (EIP) to provide a comprehensive system of early intervention services for infants and toddlers with disabilities and their families, and ultimately improving health outcomes for these individuals.
DSHP List 2. The state may claim FFP in support of DSHP for List 2 DSHP expenditures (excluding expenditures that are otherwise eligible for federal support or that are eligible for payment by third-party payers) made after December 31, 2014. The state may not claim FFP until after December 31, 2015 for i through VI and viii below. The General Public Health Work is still under review. The state may claim for the General Public Health Work once CMS approves expenditures in this category.
1. Childhood Lead Poisoning Primary Prevention. In an effort to eliminate childhood lead poisoning in New York State, the Childhood Lead Poisoning Primary Prevention Program´s goal is to increase the availability and number of housing units that are free of lead-based paint hazards in targeted communities identified with high incidence of childhood lead poisoning. Approximately 3,000 children are diagnosed with lead poisoning each year in New York State that could result in long-term adverse health effects and substantial costs to the State and local governments. Reducing led poisoning improves health outcomes and associated treatment costs.
2. Healthy Neighborhoods Program. The New York State Healthy Neighborhoods Program (HNP) seeks to reduce the burden of housing related illnesses and injury. The program targets housing in high-risk areas that are identified using house, health and socioeconomic indicators from census and surveillance data. The HNP uses a combination of neighborhood canvassing and referrals to reach residents in these high-risk areas. During a visit, the home is assessed for environmental health and safety issues. For problems or potential hazards identified during the visit, an outreach worker provides education, referrals and products to help residents correct or reduce housing hazards. Twenty-five per cent of homes receive a revisit to provide additional service and assess outcomes. The program can improve health outcomes and reduce costs for associated ED visits and other healthcare costs.
3. Cancer Services Programs. The NYSDOH Cancer Services Program (CSP) oversees the delivery of comprehensive breast, cervical and colorectal cancer screening and diagnostic services to eligible uninsured and underinsured individuals in New York State through contracts with community-based organizations known as CSP partnerships. Contractors develop relationships with regional providers (e.g., hospitals, clinics, health care providers) and community-based organizations to conduct outreach to priority populations, provide screening, diagnostic and case management services, public education, data management and quality assurance, as well as other activities outlined later in this document. The program can improve health outcomes and reduce costs through early intervention.
4. Obesity and Diabetes Programs. The Obesity and Diabetes Prevention Programs are designed to raise public and professional awareness of the twin epidemics of obesity and diabetes, reduce the prevalence of these diseases and their risk factors, and develop and implement programs to prevent or reduce these diseases and their complications. Reducing obesity and diabetes can reduce Medicaid costs and improve health outcomes.
5. TB Treatment, Detection and Prevention. The Public Health Campaign funds support Tuberculosis (TB) contracts with twelve local health departments (including the New York City Department of Health) for maintenance of local public health infrastructure that provides direct patient care. These health departments are located in the city/counties with the highest TB morbidity in the State. This direct care can improve health outcomes and reduce costs by preventing costlier TB cases.
6. TB Directly Observed Therapy. National Tuberculosis (TB) treatment guidelines strongly recommend using a patient-centered case management approach including directly observed therapy (DOT) when treating persons with active TB disease. DOT is especially critical for patients with drug- resistant TB, HIV-infected patients, and those on intermittent treatment regimens (i.e., 2 or 3 times weekly). DOT decreases the chances of treatment failure and relapse and is highly effective at preventing the spread of TB and the development of multi-drug resistant (MDR) strains of TB. Providing these services improves health outcomes and reduces costs.
7. General Public Health Work. This program is under review by CMS and is not yet an allowed DSHP.
8. Newborn Screening Programs. The Newborn Screening Program performs more than 11 million screens annually for more than 40 congenital disorders and exposure to the human immunodeficiency virus (HIV). The tests are conducted on the approximately quarter of a million babies born each year in New York State. The program improves outcomes through early interventions and saves long-term costs through early intervention.
DSHP List 3. The state may claim FFP in support of DSRIP for List 3 DSHP expenditures not used for DD Transformation and exclude expenditures that are otherwise eligible for federal support or that are eligible for payment by third-party payers. The state may not claim FFP until after the date on which CMS has approved a DSHP Claiming Protocol for the specific DSHP.
1. Office of Mental Health. Funds are used for a range of services and in a range of settings to provide treatment designed to reduce symptoms, improve functioning and ensure ongoing support for individuals experiencing serious and persistent mental illness and ensure that their basic needs are met. This program has a focus on improving an individual´s quality of life in the community and reducing the need for inpatient care through the provision of community-based treatment services. Providing these services can improve outcomes and reduce costs for individuals with mental illness. Specific components are noted below.
  1. Licensed Outpatient Programs
  2. Care Management
  3. Emergency Programs
  4. Rehabilitation Services
  5. Residential (Non-Treatment)
  6. Community Support Programs
2. Office for People with Developmental Disabilities. These programs provide a range of programs designed to identify people with developmental disabilities (including autism), improve functioning for those with developmental disabilities, and to provide support to families and caregivers. Services improve health outcomes and functioning for individuals with developmental disabilities, and in the long-term, lower Medicaid costs by improving the self- sufficiency of individuals with developmental disabilities. Specific components are noted below.
  1. Day Training
  2. Family Support Services
  3. Jervis Clinic (diagnostic center)
  4. Intermediate Care Facilities
  5. HCBS Residential
  6. Supported Work (SEMP)
  7. Day Habilitation
  8. Care Management
  9. Pre-vocational Services
  10. Waiver Respite (temporary relief to care-givers)
  11. Clinics - Article 16 (primarily long-term therapies)
3. Office of Alcoholism and Substance Abuse Services. These programs support prevention and provide funds support safety net services for individuals who would otherwise be at risk of hospitalization or more costly Medicaid services requiring increased Federal Financial Participation.
  1. Outpatient and Opioid Treatment Programs.
  2. Prevention and Program Support Services
DSHP Claiming Protocol. The state will develop a CMS-approved DSHP claiming protocol with which the state will be required to comply in order to draw down DSHP funds for DSRIP. State expenditures for the DSHP listed above must be documented in accordance with the protocols. The state is not eligible to receive FFP until an applicable protocol is approved by CMS. Once approved by CMS, the protocol becomes Attachment L of these STCs, and thereafter may be changed or updated with CMS approval. Changes and updates are to be applied prospectively. For each DSHP, the protocol must contain the following information:
1. The sources of non-federal share revenue, full expenditures and rates.
2. Program performance measures, baseline performance measure values, and improvement goals. (CMS may, at its option, approve the DSHP Claiming Protocol for a DSHP without this feature.)
3. Procedures to ensure that FFP is not provided for any of the following types of expenditures:
  1. Grant funding to test new models of care
  2. Construction costs (bricks and mortar)
  3. Room and board expenditures
  4. Animal shelters and vaccines
  5. School based programs for children
  6. Unspecified projects
  7. Debt relief and restructuring
  8. Costs to close facilities
  9. HIT/HIE expenditures
  10. Services provided to undocumented individuals
  11. Sheltered workshops
  12. Research expenditures
  13. Rent and utility subsidies
  14. Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave
  15. Revolving capital fund
  16. Expenditures made to meet a maintenance of effort requirement for any federal grant program
  17. Administrative costs
  18. Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans)
  19. Cost of services for which payment was made by Medicare or Medicare Advantage
  20. Funds from other federal grants
  21. Needle-exchange programs
4. Procedures to ensure that FFP is not claimed for expenditures that are claimed for any other federal funding purpose, including as part of a state maintenance of effort requirement under other grant programs.
DSHP Claiming Process.
1. Documentation of each designated state health program´s expenditures, as specified in the DSHP Protocol, must be clearly outlined in the state´s supporting work papers and be made available to CMS.
2. In order to assure CMS that Medicaid funds are used for allowable expenditures, the state will be required to document through an Accounting and Voucher system its request for DSHP payments. The vouchers will be detailed in the services being requested for payment by the state and will be attached to DSHP support.
3. Federal funds must be claimed within two years following the calendar quarter in which the state disburses expenditures for the DSHP.
4. Federal funds are not available expenditures disbursed before April 1, 2014, or for services rendered prior to April 1, 2014.
5. Federal funds are not available for expenditures disbursed after March 31, 2020, or for services rendered after March 31, 2020.
6. Sources of non-federal funding must be compliant with section 1903(w) of the Act and applicable regulations. To the extent that federal funds from any federal programs are received for the DSHP listed above, they shall not be used as a source of non- federal share.
7. The administrative costs associated with the DSHP listed above, and any others subsequently added by amendment to the demonstration, shall not be included in any way as demonstration and/or other Medicaid expenditures.
8. Any changes to the DSHP listed above shall be considered an amendment to the demonstration and processed in accordance with STC 7 in Section III.
Reporting DSHP Expenditure. The state will report all DSHP expenditures listed above on the forms CMS-64.9 Waiver and/or 64.9P Waiver as well as on the appropriate forms CMS-64.9I and CMS-64PI under the waiver name:
1. "DSHP for DSRIP" (if in support of DSRIP)
2. "DSHP for IAAF" (if in support of Interim Access Assurance Fund payments)
To address New York´s unique restrictions on Medicaid spending, the state may claim FFP for DSHP expenditures incurred in the current demonstration year or a prior demonstration year, provided it is within the two year limit of when the state paid the claim and within the total DSHP cap for the demonstration year and for demonstration as a whole.

Budget Neutrality Review. In conjunction with any demonstration renewal beyond December 31, 2014, CMS reserves the right to modify the budget neutrality agreement consistent with budget neutrality policy.
Improved Management Controls. The state and CMS agree that, in conjunction with any demonstration renewal beyond December 31, 2014, the state will undertake additional activities and steps to strengthen internal controls, compliance with federal and state Medicaid requirements and financial reporting to ensure proper claiming of federal match for the Medicaid program, and to self-identify and initiate timely corrective action on problems and issues. To support the development of these additional special terms and conditions, the state reported on its assessment of current strengths and weaknesses of the state´s system of internal and financial management controls (taking into account any audit findings from federal or state oversight agencies including the HHS Office of Inspector General, the state Office of Inspector General, and CMS); the steps the state proposes to take to strengthen compliance, documentation and transparency; and the expected path for resolution of any outstanding deferrals or disallowances initiated by CMS as of the date of this amendment.
DSRIP Transparency. During the 30-day public comment period for the DSRIP Program Funding and Mechanics protocol (Attachment I), DSRIP Strategies Menu and Metrics (Attachment J), the state must have conducted at least two public hearings regarding the state´s DSRIP amendment approval. The state must utilize teleconferencing or web capabilities for at least one of the public hearings to ensure statewide accessibility. The two public hearings must be held on separate dates and in separate locations and must afford the public an opportunity to provide comments. Once the state develops its standardized review tool the independent assessor will use for the DSRIP project plans, the tool must also be posted for public comment for 30 days.
1. Administrative Record. CMS will maintain, and publish on its public Web site, an administrative record that may include, but is not limited to the following:
  1. the demonstration application from the state;
  2. written public comments sent to the CMS and any CMS responses;
  3. if an application is approved, the final special terms and conditions, waivers, expenditure authorities, and award letter sent to the state;
  4. if an application is denied, the disapproval letter sent to the state;
  5. the state acceptance letter, as applicable;
  6. specific requirements related to the approved and agreed upon terms and conditions, such as implementation reviews, evaluation design, quarterly progress reports, annual reports, and interim and/or final evaluation reports; and
  7. Notice of the demonstration´s suspension or termination, if applicable.
2. Other Documentation. CMS will provide sufficient documentation to address substantive issues relating to the approval documentation that should comprehensively set forth the basis, purpose, and conditions for the approved demonstration.
Submission of Draft Evaluation Design. The state shall submit a draft DSRIP evaluation design to CMS no later than 120 days after the award of the demonstration, including, but not limited to data that the state proposes to be used to evaluate DSRIP. The state must employ aggressive state-level standards that align with its managed care evaluation approach.
Submission of Final Evaluation Design. The state shall provide the Final Evaluation Design within 30 days of selecting the Independent Evaluator. If CMS finds that the Final Evaluation Design adequately accommodates its comments, then CMS will approve the Final Evaluation Design and the final evaluation plan will be included as Attachment M of these STCs.
Evaluation Requirements. The state must conform to all requirements noted in STC 2 of Section XI. The state shall engage the public in the development of its evaluation design. The demonstration evaluation will meet the prevailing standards of scientific and academic rigor, as appropriate and feasible for each aspect of the evaluation, including standards for the evaluation design, conduct, and interpretation and reporting of findings. The demonstration evaluation will use the best available data; use controls and adjustments for and reporting of the limitations of data and their effects on results; and discuss the generalizability of results.

The state shall acquire an independent entity to conduct the evaluation. The evaluation design shall discuss the strategy for each aspect of the evaluation, including standards for the evaluation design, conduct, and qualifications the entity must possess, how the state will assure no conflict of interest, and a budget for evaluation activities.

The evaluation design shall incorporate an interim and summative evaluation and will discuss the following requirements as they pertain to each:
1. the scientific rigor of the analysis;
2. a discussion of the goals, objectives and specific hypotheses that are to be tested;
3. specific performance and outcomes measures used to evaluate the demonstration´s impact;
4. how the analysis will support a determination of cost effectiveness;
5. data strategy including sources of data, sampling methodology, and how data will be obtained;
6. the unique contributions and interactions of other initiatives; and
7. how the evaluation and reporting will develop and be maintained.
Evaluation Design. The Evaluation Design shall include the following core components to be approved by CMS:
1. Research questions and hypotheses. This includes a statement of the specific research questions and testable hypotheses that address the goals of the demonstration, including:
  1. safety net system transformation at both the system and state level;
  2. accountability for reducing avoidable hospital use and improvements in other health and public health measures at both the system and state level; and
  3. efforts to ensure sustainability of transformation of/in the managed care environment at the state level.
    
    The research questions will be examined using appropriate comparison groups and studied in a time series.
2. Design. The design will include a description of the quantitative and qualitative study design (e.g., cohort, controlled before-and-after studies, interrupted time series, case-control, etc.), including a rationale for the design selected. The discussion will include a proposed baseline and approach to comparison. The discussion will also include an approach to benchmarking and should consider applicability of national and state standards. The application of sensitivity analyses as appropriate shall be considered.
3. Performance Measures: This includes identification, for each hypothesis, of quantitative and/or qualitative process and/or outcome measures that adequately assess the effectiveness of the Demonstration in terms of cost of services and total costs of care, change in delivery of care from inpatient to outpatient, quality improvement, and transformation of incentive arrangements under managed care.
  
  Nationally recognized measures should be used where appropriate. Measures will be clearly stated and described, with the numerator and dominator clearly defined. To the extent possible, the state will incorporate comparisons to national data and/or measure sets. A broad set of metrics will be selected. To the extent possible, metrics will be pulled from nationally recognized metrics such as from the National Quality Forum, Center for Medicare and Medicaid Innovation, meaningful use under HIT, and the Medicaid Core Adult sets, for which there is sufficient experience and baseline population data to make the metrics a meaningful evaluation of the New York Medicaid system.
4. Data Collection. This discussion shall include: a description of the data sources; the frequency and timing of data collection; and the method of data collection. The following shall be considered and included as appropriate:
  1. Medicaid encounter and claims data in Transformed Medicaid Statistical Information System (TMSIS);
  2. enrollment data;
  3. EHR data, where available;
  4. semiannual financial and other reporting data;
  5. managed care contracting data;
  6. consumer and provider surveys; and
  7. other data needed to support performance measurement.
5. Assurances Needed to Obtain Data. The design report will discuss the state´s arrangements to assure needed data to support the evaluation design are available.
6. Data Analysis. This includes a detailed discussion of the method of data evaluation, including appropriate statistical methods that will allow for the effects of the Demonstration to be isolated from other initiatives occurring in the state. The level of analysis may be at the beneficiary, provider, health plan and program level, as appropriate, and shall include population and intervention specific stratifications, for further depth and to glean potential non-equivalent effects on different sub-groups. Sensitivity analyses shall be used when appropriate. Qualitative analysis methods shall also be described, if applicable.
7. Timeline: This includes a timeline for evaluation related milestones, including those related to procurement of an outside contractor, if applicable, and deliverables.
8. Evaluator: This includes a discussion of the state´s process for obtaining an independent entity to conduct the evaluation, including a description of the qualifications that the selected entity must possess; how the state will assure no conflict of interest, and a budget for evaluation activities.
Interim Evaluation Report. The state is required to submit a draft Interim Evaluation Report 90 days following completion of DDY 4 of the demonstration. The Interim Evaluation Report shall include the same core components as identified in STC 24 of this section for the Summative Evaluation Report and should be in accordance with the CMS approved evaluation design. CMS will provide comments within 60 days of receipt of the draft Interim Evaluation Report. The state shall submit the final Interim Evaluation Report within 30 days after receipt of CMS´ comments.
Final Summative Evaluation Report. The Final Summative Evaluation Report will include analysis of data from DDY 5. The state is required to submit a preliminary summative report within 180 days of the expiration of the demonstration including documentation of outstanding assessments due to data lags to complete the summative evaluation. Within 360 days of the end for DDY 5, the state shall submit a draft of the final summative evaluation report to CMS. CMS will provide comments on the draft within 60 days of draft receipt. The state should respond to comments and submit the Final Summative Evaluation Report within 30 days. The Final Summative Evaluation Report shall include the following core components:
1. Executive Summary. This includes a concise summary of the goals of the Demonstration; the evaluation questions and hypotheses tested; key findings including whether the evaluators find the demonstration to be budget neutral and cost effective; and policy implications.
2. Demonstration Description. This includes a description of the Demonstration programmatic goals and strategies, particularly how they relate to budget neutrality and cost effectiveness.
3. Study Design. This includes a discussion of the evaluation design employed including research questions and hypotheses; type of study design; impacted populations and stakeholders; data sources; and data collection; analysis techniques, including controls or adjustments for differences in comparison groups, controls for other interventions in the state and any sensitivity analyses, and limitations of the study.
4. Discussion of Findings and Conclusions. This includes a summary of the key findings and outcomes, particularly a discussion of cost effectiveness, as well as implementation successes, challenges, and lessons learned.
5. Policy Implications. This includes an interpretation of the conclusions; the impact of the demonstration within the health delivery system in the state; the implications for state and federal health policy; and the potential for successful demonstration strategies to be replicated in other state Medicaid programs.
6. Interactions with Other State Initiatives. This includes a discussion of this demonstration within an overall Medicaid context and long-range planning; interrelations of the demonstration with other aspects of the state´s Medicaid program; and interactions with other Medicaid waivers and other federal awards affecting service delivery, health outcomes and the cost of care under Medicaid.
State Presentations for CMS. The state will present to and participate in a discussion with CMS on the final design plan at post approval. The state will present on its interim evaluation report that is described in STC 23 of this section. The state will present on its summative evaluation in conjunction with STC 24 of this section.
Public Access. The state shall post the final approved Evaluation Design, Interim Evaluation Report, and Summative Evaluation Report on the State Medicaid website within 30 days of approval by CMS.
CMS Notification. For a period of 24 months following CMS approval of the Summative Evaluation Report, CMS will be notified prior to the public release or presentation of these reports and related journal articles, by the state, contractor or any other third-party. Prior to release of these reports, articles and other documents, CMS will be provided a copy including press materials. CMS will be given 30 days to review and comment on journal articles before they are released. CMS may choose to decline some or all of these notifications and reviews.
Electronic Submission of Reports. The state shall submit all required plans and reports using the process stipulated by CMS, if applicable.
Cooperation with Federal Evaluators. Should CMS undertake an evaluation of the demonstration or any component of the demonstration, or an evaluation that is isolating the effects of DSRIP, the state and its evaluation contractor shall cooperate fully with CMS and its contractors. This includes, but is not limited to, submitting any required data to CMS or the contractor in a timely manner and at no cost to CMS or the contractor.
Cooperation with Federal Learning Collaboration Efforts. The state will cooperate with improvement and learning collaboration efforts by CMS.
Evaluation Budget. In addition to a detailed evaluation design, a proposed budget for the evaluation will be a requirement for applications submitted under the Request for Proposals (RFP) to procure the Independent Evaluator. It must include the total estimated cost, as well as a breakdown of estimated staff, administrative and other costs for all aspects of the evaluation such as any survey and measurement development, quantitative and qualitative data collection and cleaning analyses, and reports generation. A justification of the costs may be required by CMS if the estimates provided do not appear to sufficiently cover the costs of the design or if CMS finds that the design is not sufficiently developed.
DSRIP Implementation Monitoring. The state must ensure that they are operating its DSRIP program according to the requirements of the governing STCs. In order to demonstrate adequate implementation monitoring towards the completion of these requirements, the state will submit the following:
1. DSRIP monitoring activities, in STC 33of this section as a part of the operational protocol in STC 10 (h) of this section, indicating how the state will monitor compliance with demonstration requirements in the implementation of this demonstration, including monitoring and performance reporting templates. Monitoring and performance templates are subject to review and approval by CMS.
2. Data usage agreements demonstrating the availability of required data to support the monitoring of implementation.
3. Quarterly Report Framework indicating what metrics and data will be available to submit a quarterly report consistent with STC 34 of this section.
DSRIP Monitoring Activities. As part of the state´s Operational Protocol described in STC 10 (h) of this section and Attachment K, the state will submit its plans for how it will meet the DSRIP STCs through internal monitoring activities. The monitoring plans should provide, at a minimum, the following information:
1. The monitoring activities aligned with the DSRIP deliverables as well as the CMS evaluation design to ensure that entities participating in the DSRIP process are accountable for the necessary product and results for the demonstration.
2. The state shall make the necessary arrangements to assure that the data needed from the Performing Provider Systems, coalitions, administrative activities, independent assessor and independent evaluator that are involved in the process for DSRIP deliverables, measurement and reporting are available as required by the CMS approved monitoring protocol.
3. The state shall identify areas within the state´s internal DSRIP process where corrective action, or assessment of fiscal or non-fiscal penalties may be imposed for the entities described in STC 10 (e) of this section, should the state´s internal DSRIP process or any CMS monitored process not be administered in accordance with state or federal guidelines.
4. The monitoring protocol and reports shall be posted on the state Medicaid website within 30 days of submission to CMS.
DSRIP Quarterly Progress Reports. The state must submit progress reports in the format specified by CMS, no later than 60-days following the end of each quarter along with the Operational Protocol Report described above. The intent of these reports is to present the state´s analysis and the status of the various operational areas in reaching the three goals of the DSRIP activities. These quarterly reports use the quarterly report guideline outlined in Attachment L. The state may comment and submit a revised Attachment L no later than 30 days after approval of these STCs. CMS will approve necessary changes and update the attachment as necessary. Any subsequent changes to Attachment L must be submitted to CMS prior the end of the reporting period in which the change to the Quarterly Report would take place.

Quarterly reports must include, but are not limited to the following reporting elements:
1. summary of quarterly expenditures related to IAAF, DSRIP Project Design Grant, and the DSRIP Fund;
2. summary of all public engagement activities, including, but not limited to the activities required by CMS;
3. summary of activities associated with the IAAF, DSRIP Project Design Grant, and the DSRIP Fund. This shall include, but is not limited to, reporting requirements in STC 34of this section and Attachment K, the Operational Protocol:
  1. provide updates on state activities, such as changes to state policy and procedures, to support the administration of the IAAF, DSRIP Project Design Grant and the DSRIP Fund;
  2. provide updates on provider progress towards the pre-defined set of activities and associated milestones that collectively aim towards addressing the state´s goals;
  3. provide summary of state´s analysis of DSRIP Project Design;
  4. provide summary of state analysis of barriers and obstacles in meeting milestones;
  5. provide summary of activities that have been achieved through the DSRIP Fund; and
  6. provide summary of transformation and clinical improvement milestones and that have been achieved.
4. summary of activities and/or outcomes that the state and MCOs have taken in the development of and subsequent approval of the Managed Care DSRIP plan; and
5. evaluation activities and interim findings.
Annual Onsite with CMS. In addition to regular monitoring calls, the state shall on an annual basis present to and participate in a discussion with CMS on implementation progress of the demonstration including progress toward the goals, and key challenges, achievements and lessons learned.
Rapid Cycle Assessments. The state shall specify for CMS approval a set of performance and outcome metrics and network characteristics, including their specifications, reporting cycles, level of reporting (e.g., the state, health plan and provider level, and segmentation by population) to support rapid cycle assessment in trends under premium assistance and Medicaid fee-for-service, and for monitoring and evaluation of the demonstration.
Medicaid Managed Care DSRIP Contracting Plan. In recognition that the DSRIP investments represented in this waiver must be recognized and supported by the state´s managed care plans as a core component of long-term sustainability, and will over time improve the ability of plans to coordinate care and efficiently deliver high quality services to Medicaid beneficiaries through comprehensive payment reform, strengthened provider networks and care coordination, the state must take steps to plan for and reflect the impact of DSRIP in managed care contracts and rate-setting approaches. Prior to the state submitting contracts and rates for approval for the April 1, 2015 to March 31, 2016 contract cycle, the state must submit a roadmap for how they will amend contract terms and reflect new provider capacities and efficiencies in managed care rate-setting.

Recognizing the need to formulate this plan to align with the stages of DSRIP, this should be a multi-year plan. It will necessarily be flexible to properly reflect future DSRIP progress and accomplishments. This plan must be approved by CMS before the state may claim FFP for managed care contracts for the 2015-16 state fiscal year. The state shall update and submit the Managed Care DSRIP plan annually on the same cycle and with the same terms, until the end of this demonstration period and its next renewal period. Progress on the Managed Care DSRIP plan will also be included in the quarterly DSRIP report. The Managed Care DSRIP plan should address the following:
1. What approaches MCOs will use to reimburse providers to encourage practices consistent with DSRIP objectives and metrics, including how the state will plan and implement its stated goal of 90% of managed care payments to providers using value- based payment methodologies.
2. How and when plans´ currents contracts will be amended to include the collection and reporting of DSRIP objectives and measures.
3. How the DSRIP objectives and measures will impact the administrative load for MCOs, particularly insofar as plans are providing additional technical assistance and support to providers in support of DSRIP goals, or themselves carrying out programs or activities for workforce development or expansion of provider capacity. The state should also discuss how these efforts, to the extent carried out by plans, avoid duplication with DSRIP funding or other state funding; and how they differ from any services or administrative functions already accounted for in capitation rates.
4. How alternative payment systems deployed by MCOs will reward performance consistent with DSRIP objectives and measures.
5. How the state will assure that providers participating in and demonstrating successful performance through DSRIP will be included in provider networks.
6. How managed care rates will reflect changes in case mix, utilization, cost of care and enrollee health made possible by DSRIP, including how up to date data on these matters will be incorporated into capitation rate development.
7. How actuarially-sound rates will be developed, taking into account any specific expectations or tasks associated with DSRIP that the plans will undertake, and how the state will use benchmark measures (e.g., MLR) to ensure that payments are sound and appropriate. How plans will be measured based on utilization and quality in a manner consistent with DSRIP objectives and measures, including incorporating DSRIP objectives into their annual utilization and quality management plans submitted for state review and approval by January 31 of each calendar year.
8. How the state will use DSRIP measures and objectives in their contracting strategy approach for managed care plans, including reform.

|top of section| |top of page|

VIII. GENERAL REPORTING REQUIREMENTS

General Financial Requirements. The state must comply with all general financial requirements set forth in Section IX.
Reporting Requirements Related to Budget Neutrality. The state must comply with all reporting requirements for monitoring budget neutrality set forth in Section X.
Monthly Calls. CMS shall schedule monthly conference calls with the state. The purpose of these calls is to discuss any significant actual or anticipated developments affecting the demonstration. Areas to be addressed include, but are not limited to: MCO operations, including contract amendments and rate certifications; transition and implementation activities; health care delivery; enrollment of individuals using LTSS and non- LTSS users broken out by dually eligible and non-dually eligible populations; cost sharing; quality of care; access; benefits; audits; lawsuits; financial reporting and budget neutrality issues; MCO financial performance that is relevant to the demonstration; progress on evaluations; state legislative developments; services being added to the MMMC, HIV SNP, HARPs or MLTC benefit package pursuant to Section V; and any demonstration amendments or concept papers. CMS shall update the state on any amendments or concept papers under review, as well as federal policies and issues that may affect any aspect of the demonstration. The state and CMS shall jointly develop the agenda for the calls.
Quarterly Operational Reports. The state must submit progress reports in accordance with the guidelines in Attachment E taking into consideration the requirements in STC 7 of this section, no later than 60 days following the end of each quarter (December, March, and June of each demonstration year). The state may combine the quarterly report due for the quarter ending September with the annual report in STC 5 of this section. The intent of these reports is to present the state´s analysis and the status of the various operational areas. In addition to the guidelines for quarterly reporting in Attachment E, the state´s report shall also include the following:
1. Recipient choice of plans and capacity of plans participating in the following programs: MMMC, including HIV SNP and HARPs; MLTC, including Fully Integrated Duals Advantage (FIDA), and the number of enrollees who made an affirmative choice.
2. LTSS Assessment statistics in accordance with the requirements of STC 9 in Section V, including corrective actions against MCOs that do not meet the 30-day assessment requirement.
3. Total enrollment in each MCO by month. Data should reflect a rolling 12-month period.
4. Total enrollees who chose to opt out of HARP, the reason for opting out and the number who voluntarily enrolled or re-enrolled.
5. Progress toward compliance with T-MSIS requirements.
6. Status of managed care plan performance, initiatives and activities as measured by HEDIS, CAHPs and other quality metrics.
7. Implementation updates and metrics related to the Children´s Design Demonstration Program Component
8. Metrics to track the number of beneficiaries that are deemed LTNHS and, if applicable, where they are being discharged to.
Annual Report. The state must submit an annual report documenting accomplishments, project status, quantitative and case study findings, interim evaluation findings, utilization data, and policy and administrative difficulties in the operation of the demonstration. The state must submit this report no later than 90 days following the end of each demonstration year. Additionally, the annual report must include:
1. a summary of the elements included within each quarterly report;
2. an update on the progress related to the quality strategy as required STC 12 in Section VI, including:
  1. outcomes of care, quality of care, cost of care and access to care for demonstration populations; and
  2. the results of beneficiary satisfaction survey, grievances and appeals.
3. the status of the evaluation required in Section XII and information regarding progress in achieving demonstration evaluation criteria including the results/impact of any demonstration programmatic area defined by CMS that is unique to the demonstration design or evaluation hypotheses;
4. an aggregated enrollment report showing the total number of individuals enrolled in each plan;
5. a list of the benefits added to the managed care benefit package;
6. an updated transition plan which shows the intended transition and timeline for any new benefits and/or populations into the demonstration;
7. network adequacy reporting as required in STC 15 of Section VI;
8. state efforts related to the collection and verification of encounter data and utilization data, including the required transition to T-MSIS, encounter data validation activities and outcomes conducted by the EQRO;
9. any other topics of mutual interest between CMS and the state related to the demonstration; and
10. any other information the state believes pertinent to the demonstration, such as:
  1. any policy or administrative difficulties that may impact the demonstration;
  2. any state legislative developments that may impact the demonstration;
  3. the status of the health care delivery system under the demonstration with respect to issues and/or complaints identified by beneficiaries;
  4. the impact of the demonstration in providing insurance coverage to beneficiaries and uninsured population;
  5. the existence or results of any audits, investigations or lawsuits that impact the demonstration;
  6. the financial performance of the demonstration (budget neutrality);
  7. a summary of the annual post-award forum, including all public comments received regarding the process of the demonstration project.
Transition Plan. On or before July 1, 2012, and consistent with guidance provided by CMS, the state is required to prepare, and incrementally revise, a Transition Plan consistent with the provisions of the Affordable Care Act (ACA) for individuals enrolled in the demonstration, including how the state plans to coordinate the transition of these individuals to a coverage option available under the ACA without interruption in coverage to the maximum extent possible. The plan must include the required elements and milestones described in paragraphs (a)-(e) outlined below. In addition, the Plan will include a schedule of implementation activities that the state will use to operationalize the Transition Plan. For any elements and milestones that remain under development as of July 1, 2012, the state will include in the Transition Plan a description of the status and anticipated completion date.
1. Seamless Transitions. Consistent with the provisions of the ACA, the Transition Plan will include details on how the state plans to obtain and review any additional information needed from each individual to determine eligibility under all eligibility groups, and coordinate the transition of individuals enrolled in the demonstration (by FPL) (or newly applying for Medicaid) to a coverage option available under the ACA without interruption in coverage to the maximum extent possible. Specifically, the state must:
  1. determine eligibility under all January 1, 2014, eligibility groups for which the state is required or has opted to provide medical assistance, including the group described in §1902(a)(10)(A)(i)(VIII) for individuals under age 65 and regardless of disability status with income at or below 133 percent of the FPL;
  2. identify demonstration populations not eligible for coverage under the ACA and explain what coverage options and benefits these individuals will have effective January 1, 2014;
  3. implement a process for considering, reviewing and making preliminary determinations under all January 1, 2014 eligibility groups for new applicants for Medicaid eligibility;
  4. conduct an analysis that identifies populations in the demonstration that may not be eligible for or affected by the ACA and the authorities the state identifies that may be necessary to continue coverage for these individuals; and
  5. Develop a modified adjusted gross income (MAGI) calculation for program integrity.
2. Access to Care and Provider Payments.
  1. Provider Participation. The state must identify the criteria that will be used for reviewing provider participation in (e.g., demonstrated data collection and reporting capacity) and means of securing provider agreements for the transition.
  2. Adequate Provider Supply. The state must provide the process that will be used to assure adequate provider supply for the state plan and demonstration populations affected by the demonstration on December 31, 2013. The analysis should address delivery system infrastructure/capacity, provider capacity, utilization patterns and requirements (i.e., prior authorization), current levels of system integration, and other information necessary to determine the current state of the of service delivery. The report must separately address each of the following provider types:
    1. primary care providers,
    2. mental health services,
    3. substance use services, and
    4. dental.
  3. Provider Payments. The state will establish and implement the necessary processes for ensuring accurate encounter payments to providers entitled to the prospective payment services (PPS) rate (e.g., certain FQHCs and RHCs) or the all-inclusive rate (e.g., certain Indian Health providers).
3. System Development or Remediation. The Transition Plan for the demonstration is expected to expedite the state´s readiness for compliance with the requirements of the Affordable Care Act and other federal legislation. System milestones that must be tested for implementation on or before January 1, 2014 include replacing manual administrative controls with automotive processes to support a smooth interface among coverage and delivery system options that is seamless to beneficiaries.
4. Progress Updates. After submitting the initial Transition Plan for CMS approval, the state must include progress updates in each quarterly and annual report. The Transition Plan shall be revised as needed.
5. Implementation
  1. By October 1, 2013, the state must begin to implement a simplified, streamlined process for transitioning eligible enrollees in the demonstration to Medicaid, the Exchange or other coverage options in 2014. In transitioning these individuals from coverage under the waiver to coverage under the state plan, the state will not require these individuals to submit a new application.
  2. On or before December 31, 2013, the state must provide notice to the individual of the eligibility determination using a process that minimizes demands on the enrollees.
Reporting Requirements Related to Individuals using Long Term Services and Supports. In each quarterly report required by Section VIII the state shall report:
1. Any critical incidents reported within the quarter and the resulting investigations as appropriate.
2. The number and types of grievance and appeals for this population filed and/or resolved within the reporting quarter for this population.
3. The total number of assessments for enrollment performed by the plans, with the number of individuals who did not qualify to enroll in an MLTC plan.
4. The number of individuals referred to an MLTC plan that received an assessment within 30 days.
5. The number of people who were not referred by the enrollment broker and contacted the plan directly and were provided MLTC materials.
6. Rebalancing efforts performed by the MLTC and MMMC plans once the benefit is added. Rebalancing reporting should include but is not limited to the total number of individuals transitioning in and out of a nursing facility within the quarter.
7. The total number of complaints, grievances and appeals by type of issue with a listing of the top 5 reasons for the event.
Final Evaluation Report. The state shall submit a Final Evaluation Report pursuant to the requirements of section 1115 of the Act.

|top of section| |top of page|

IX. GENERAL FINANCIAL REQUIREMENTS

Quarterly Expenditure Reports. The state must provide quarterly expenditure reports using Form CMS-64 to separately report total expenditures for services provided under the Medicaid program, including those provided through the demonstration under section 1115 authority. This project is approved for expenditures applicable to services rendered during the demonstration period. CMS shall provide FFP for allowable demonstration expenditures only as long as they do not exceed the pre-defined limits on the costs incurred as specified in Section X. Effective September 30, 2017, the state will be assessed a $5,000,000 penalty if it falls behind more than 2 quarters in reporting EG expenditures.

Reporting Expenditures under the Demonstration. The following describes the reporting of expenditures under the demonstration:

In order to track expenditures under this demonstration, New York must report demonstration expenditures through the Medicaid and State Children´s Health Insurance Program Budget and Expenditure System, following routine CMS-64 reporting instructions outlined in Section 2500 of the State Medicaid Manual. All demonstration expenditures must be reported each quarter on separate Forms CMS-64.9 Waiver and/or 64.9P Waiver, identified by the demonstration project number assigned by CMS (including the project number extension, which indicates the DY in which services were rendered or for which capitation payments were made). In addition,
1. DSRIP expenditures must be reported for the DY corresponding to the DDY for under which the expenditures were made (e.g., expenditures for DDY 0 are reported for DY 16), and
2. expenditures for DSHP must be reported for the DY during which the state program expenditures were incurred.
The state shall have a Budget Neutrality Specifications Manual that outlines the Medicaid coverage expenditures extracted from New York´s Medicaid Management Information system and reported on the CMS-64 Waiver sheets for all Member Eligibility Groups identified in this section of these Special Terms and Conditions prior the effective date of this renewal.

DY reporting shall be consistent with the periods specified below:

Table 11: DY Reporting Periods
Demonstration Year	Time Period
1	10/1/1997-9/30/1998
2	10/1/1998-9/30/1999
3	10/1/1999-9/30/2000
4	10/1/2000-9/30/2001
5	10/1/2001-3/30/2003
6	04/1/2003-9/30/2004
7	10/1/2004-9/30/2005
8	10/1/2005-9/30/2006
9	10/1/2006-09/30/2007
10	10/1/2007-09/30/2008
11	10/1/2008-09/30/2009
12	10/1/2009-09/30/2010
13	10/1/2010-09/30/2011
14	10/1/2011-09/30/2012
15	10/1/2012-09/30/2013
16	10/1/2013-03/31/2015
17	04/01/2015-03/31/2016
18	04/01/2016-03/31/2017
19	04/01/2017-03/31/2018
20	04/01/2018-03/31/2019
21	04/01/2019-03/31/2020
22	04/01/2020-03/31/2021

Demonstration expenditures will be correctly reported on Forms CMS-64.9 Waiver. Quarterly cost settlements and pharmaceutical rebates relevant to the demonstration will be allocated to the demonstration populations specified in subparagraph (g) and offset against current quarter waiver expenditures. Demonstration expenditures net of these cost settlement offsets will be reported on Form CMS-64.9 Waiver. Amounts offset will be identifiable in the state´s supporting work papers and made available to CMS.
1. Allocation of cost settlements. The state will calculate the percentage of Medicaid expenditures for each demonstration eligibility group to expenditures for all Medicaid population groups from a DataMart file produced for the latest completed federal fiscal year. Quarterly recoveries will be allocated to the eligibility groups based on those percentages. These percentages will be updated annually to reflect the most recent completed federal fiscal year.
2. Allocation of pharmacy rebates. The state will calculate the percentage of pharmacy expenditures for each demonstration eligibility group to pharmacy expenditures for all population groups from a DataMart file produced for the latest completed federal fiscal year. Rebates will be allocated to the eligibility groups based on those percentages. These percentages will be updated annually to reflect the most recent completed federal fiscal year.
For the HCBS Expansion component of the demonstration, the state shall report only the home and community-based services expenditures for Demonstration Population 9 on line 19A on Forms CMS-64.9 Waiver and/or 64.9P.
Special Claiming Rules
1. To account for Continuous Eligibility, for individuals who are no longer eligible as new adults the state will claim 97.4 percent of New Adult Group expenditures at the enhanced federal matching rate and 2.6 percent at the regular matching rate for medical assistance.

For each DY, separate waiver Forms CMS-64.9 Waiver and/or 64.9P Waiver must be completed, using the waiver name noted in Table 12 and Table 13 below, to report expenditures for the following demonstration populations and services.

Table 12. Reporting for Demonstration Populations
Reporting Name	Demonstration Population #
TANF Child	Demonstration Population 1
TANF Adult	Demonstration Population 2
SSI 0 through-64	Demonstration Population 3
SSI 65 and above	Demonstration Population 4
Non-Duals 18-64	Demonstration Population 5
Non-Duals 65+	Demonstration Population 6
MLTC Adults 18-64 Duals	Demonstration Population 7
MLTC Age 65+ Duals	Demonstration Population 8
HCBS Expansion	Demonstration Population 9
Institution to Community	Demonstration Population 10
New Adult Group CMS 64.9 VIII Waiv-New Adult Group Newly Eligible CMS 64.9VIII Waiv-New Adult Group Not Newly Childless Adults CMS 64.9VIII Waiv—New Adult Group Not Newly Children 19-20 CMS 64.9VIII Waiv-New Adult Group Not Newly Parents & Caretakers	Demonstration Population 11
Family of One Non-1915 Children	Demonstration Population 12

Table 13: Demonstration Services
Reporting Name	Description
Demonstration Services 7 [DSHP-APTC]	Designated State Health Program for expenditures made for the period January 1, 2014 through December 31, 2015 for the state-funded Marketplace subsidy program who purchases health care coverage in the Marketplace.
Demonstration Services 8 [BH HCBS]	Expenditures made for BH HCBS services for individuals enrolled in HARPs and HIV SNPs. Note: Expenditures under this EG will be claimed in the manner necessary to ensure the correct claiming of FMAP for all populations. (e.g., BH HCBS services for the adult expansion groups will be claimed at the FMAP rate at STC 6 of section X)
Demonstration Services 9 [Demonstration Only Services in MMMC]	Expenditures made for provision of residential addiction services, crisis intervention and licensed behavioral health practitioner services to MMMC enrollees only and are not provided under the state plan.
Demonstration Services 10 [DSHP for DSRIP]	Expenditures available through designated state health programs as specified in STC 15 of section VII.
Demonstration Services 11 [DSRIP-Performance Payments]	Expenditures for payments to New York that provide project funding and incentive payments to Performing Provider Systems under DSRIP.
Demonstration Services 11.5 [DSRIP-Home and Community Based Services - 1915i]	Expenditures for Home and Community Based Services - 1915i as specified in Table 8 of section VII.
Demonstration Services 12 [DSRIP-Health Homes]	Expenditures for Health Homes, as specified in STC 1 (d) of section VII.
Demonstration Services 13 [DSRIP- Workforce MLTC]	Expenditures for Workforce MLTC as specified in STC 1 (f) of section VII.
Demonstration Services 14 [DSRIP-Planning Payments-PDG] Demonstration Services 15 IAAF	Expenditures for Planning Payments as specified in STC 9 of section VII. Expenditures to ensure safety net providers can fully participate in the DSRIP transformation as specified in STC 1.b. of section VII.

Expenditures Subject to the Budget Neutrality Agreement. For purposes of this section, the term "expenditures subject to the budget neutrality agreement" must include all Medicaid expenditures described in STC 2 (g) of this section (Tables 12 and 13). All expenditures that are subject to the budget neutrality agreement are considered demonstration expenditures and must be reported on Forms CMS-64.9 Waiver and/or 64.9P Waiver.
Mandated Increase in Physician Payment Rates in 2013 and 2014. Section 1202 of the Health Care and Education Reconciliation Act of 2010 (Pub. Law 110-152) requires state Medicaid programs to reimburse physicians for primary care services at rates that are no less than what Medicare pays, for services furnished in 2013 and 2014, with the Federal Government paying 100 percent of the increase. The entire amount of this increase will be excluded from the budget neutrality test for this demonstration.
Administrative Costs. Administrative costs will not be included in the budget neutrality limit, but the state must separately track and report additional administrative costs that are directly attributable to the demonstration. All administrative costs must be identified on the Forms CMS-64.10 Waiver and/or 64.10P Waiver.
Claiming Period. All claims for expenditures subject to the budget neutrality cap (including any cost settlements) must be made within 2 years after the calendar quarter in which the state made the expenditures. All claims for services during the demonstration period (including any cost settlements) must be made within 2 years after the conclusion or termination of the demonstration. During the latter 2-year period, the state must continue to identify separately net expenditures related to dates of service during the operation of the demonstration on the CMS-64 waiver forms in order to properly account for these expenditures in determining budget neutrality.
Reporting Member Months. The following describes the reporting of member months for demonstration populations:
1. For the purpose of calculating the budget neutrality expenditure cap and for other purposes, the state must provide to CMS, as part of the quarterly report required under STC 1 in Section IX, the actual number of eligible member months for the demonstration populations defined in STC 2 of this section, for months prior to or including the ending date indicated in STC 2 (c) of this section for each demonstration population. The state must submit a statement accompanying the quarterly report, which certifies the accuracy of this information.
  
  To permit full recognition of "in-process" eligibility, reported counts of member months may be subject to revisions after the end of each quarter. Member month counts may be revised retrospectively for up to 2 years as needed.
2. The term "eligible member months" refers to the number of months in which persons are eligible to receive services. For example, a person who is eligible for 3 months contributes 3 eligible member months to the total. Two individuals who are eligible for 2 months each contribute 2 eligible member months, for a total of 4 eligible member months.
3. If there are duplicate expenditures of member months between demonstration populations, the state will ensure that duplicate member months will be omitted from any official tallies under the demonstration.
Standard Medicaid Funding Process. The standard Medicaid funding process must be used during the demonstration. New York must estimate matchable demonstration expenditures (total computable and federal share) subject to the budget neutrality expenditure cap and separately report these expenditures by quarter for each federal fiscal year on the Form CMS-37 for both the Medical Assistance Payments and State and Local Administration Costs. CMS shall make federal funds available based upon the state´s estimate, as approved by CMS. Within 30 days after the end of each quarter, the state must submit the Form CMS- 64 quarterly Medicaid expenditure report, showing Medicaid expenditures made in the quarter just ended. CMS shall reconcile expenditures reported on the Form CMS-64 with federal funding previously made available to the state and include the reconciling adjustment in the finalization of the grant award to the state. As part of the state´s amendment to introduce eligibility flexibilities that seamlessly enroll adult TANF recipients into Medicaid, extend Adult Group coverage to individuals who turn 65 for a limited period and provide Medicaid during a temporary Marketplace coverage gap, the state will work with CMS to determine the best method to reconcile actual member months and actual expenditures for individuals in affected populations to ensure appropriate FMAP is claimed.
Extent of FFP for the Demonstration. Subject to CMS approval of the source(s) of the non-federal share of funding, CMS shall provide FFP at the applicable federal matching rates for the demonstration as a whole as outlined below, subject to the limits described in section XI:
1. Administrative costs, including those associated with the administration of the demonstration.
2. Net expenditures and prior period adjustments of the Medicaid program that are paid in accordance with the approved Medicaid state plan and waiver authorities.
3. Net expenditures and prior period adjustments made under approved expenditure authorities granted through section 1115(a) (2) of the Act, with dates of service during the operation of the demonstration.
Sources of Non-Federal Share. The state certifies that the non-federal share of funds for the demonstration is state/local monies. The state further certifies that such funds shall not be used to match for any other federal grant or contract, except as permitted by law. All sources of non- federal funding must be compliant with section 1903(w) of the Act and applicable regulations. In addition, all sources of the non-federal share of funding are subject to CMS approval.
1. CMS may review the sources of non-federal share of funding for the demonstration at any time. The state agrees that all funding sources deemed unacceptable by CMS shall be addressed within the time frames set by CMS.
2. Any amendments that impact the financial status of the program shall require the state to provide information to CMS regarding all sources of the non-federal share of funding.
State Certification of Funding Conditions. The state must certify that the following conditions for the non-federal share of demonstration expenditures are met:
1. Units of government, including governmentally operated health care providers, may certify that state or local tax dollars have been expended as the non-federal share of funds under the demonstration.
2. To the extent the state utilizes certified public expenditures (CPEs) as the funding mechanism for the title XIX (or under section 1115 authority) payments, CMS must approve a cost reimbursement methodology. This methodology must include a detailed explanation of the process by which the state would identify those costs eligible under title XIX (or under section 1115 authority) for purposes of certifying public expenditures.
3. To the extent the state utilizes CPEs as the funding mechanism to claim federal match for payments under the demonstration, governmental entities to which general revenue funds are appropriated must certify to the state the amount of such tax revenue (state or local) used to satisfy demonstration expenditures. The entities that incurred the cost must also provide cost documentation to support the state´s claim for federal match.
4. The state may use intergovernmental transfers to the extent that such funds are derived from state or local tax revenues and are transferred by units of government within the state. Any transfers from governmentally operated health care providers must be made in an amount not to exceed the non-federal share of title XIX payments.
5. Under all circumstances, health care providers must retain 100 percent of the claimed expenditure. Moreover, no pre-arranged agreements (contractual or otherwise) exist between health care providers and state and/or local government to return and/or redirect any portion of the Medicaid payments. This confirmation of Medicaid payment retention is made with the understanding that payments that are the normal operating expenses of conducting business, such as payments related to taxes (including health care provider- related taxes), fees, business relationships with governments that are unrelated to Medicaid and in which there is no connection to Medicaid payments, are not considered returning and/or redirecting a Medicaid payment.
Expenditure Reconciliation and Limitations. Since DY 13 (10/1/2010 through 9/30/2011), New York has not reported demonstration expenditures consistently to CMS through the CMS-64 reports, leading to significant discrepancy between the expenditures reported on budget neutrality monitoring spreadsheets and the CMS-64. The CMS-64 is supposed to provide the official record of amounts expended under Medicaid demonstrations.
1. The state must correct and complete reporting of expenditures subject to the budget neutrality limit for DY 14 through DY 17. By December 31, 2016, the state must submit to CMS a draft plan and timeline for remediation that will include the following elements:
  1. completion of the Budget Neutrality Specifications to support reporting of expenditures in compliance with the requirements in these STCs;
  2. a detailed methodology and approach for identifying demonstration relevant expenditures, including any past expenditures that may have been reported on CMS-64.9 Base or CMS-64.9P Base forms instead of CMS 64.9 Waiver and 64.9P Waiver forms; and
  3. submission of appropriate prior period adjustments to reassign reported expenditures from Base to Waiver (or vice versa) so all expenditures subject to budget neutrality during the DY 14 through 17 period are reported as Waiver expenditures.
2. Time Frame and Limitations. The State must complete the reconciliation process by September 30, 2017. Failure to complete the reconciliation process will result in forfeiture by the state of all budget neutrality savings from DY 14 through 17.
3. By September 30, 2017, the state must provide a final analysis of the FSHRP budget neutrality.
Monitoring the Demonstration. The state will provide CMS with information to effectively monitor the demonstration, upon request, in a reasonable time frame.

|top of section| |top of page|

X. MONITORING BUDGET NEUTRALITY

Limit on Title XIX Funding. The state shall be subject to a limit on the amount of federal title XIX funding that the state may receive on selected Medicaid expenditures during the period of approval of the demonstration. The limit is determined by using a per capita cost method, and budget neutrality expenditure caps are set on a yearly basis with a cumulative budget neutrality expenditure limit for the length of the entire demonstration. The data supplied by the state to CMS to set the annual limits is subject to review and audit, and, if found to be inaccurate, will result in a modified budget neutrality expenditure limit.
Risk. New York shall be at risk for the per capita cost (as determined by the method described below) for demonstration eligibles under this budget neutrality agreement, but not for the number of demonstration eligibles in each of the groups. By providing FFP for all demonstration eligibles, New York shall not be at risk for changing economic conditions that impact enrollment levels. However, by placing New York at risk for the per capita costs for demonstration eligibles under this agreement, CMS assures that federal demonstration expenditures do not exceed the level of expenditures that would have occurred had there been no demonstration.
Demonstration Populations Used to Calculate Budget Neutrality Expenditure Limit. The following demonstration populations are used to calculate the budget neutrality expenditure limit subject to the limitations outlined in STC 4 of this section and are incorporated into the following eligibility groups (EG):
1. Demonstration Population 1 [TANF Child]
2. Demonstration Population 2 [TANF Adult]
3. Demonstration Population 3 [SSI 0 through-64]
4. Demonstration Population 4 [SSI 0-64]
5. Demonstration Population 5 [Non-Duals 18-64]
6. Demonstration Population 6 [Non-Duals 65+]
7. Demonstration Population 7 [MLTC Adults 18-64 Duals]
8. Demonstration Population 8 [MLTC Age 65+ Duals]
9. Demonstration Population 11 [New Adults]
10. Demonstration Population 12 [Fo1 Children]

Budget Neutrality Expenditure Limit. The following describes the method for calculating the budget neutrality expenditure limit for the demonstration:

For each year of the budget neutrality agreement, an annual budget neutrality expenditure limit is calculated for each EG described in STC 3 of this section as follows:

An annual EG estimate must be calculated as a product of the number of eligible member months reported by the state for each EG, times the appropriate estimated per member per month (PMPM) costs from the table in subparagraph (iii) below. Should EGs 3 and 4 be incorporated into the budget neutrality expenditure limit, as outlined in this STC, the PMPM costs may be revised.
The PMPM costs in subparagraph (iii) below are net of any premiums paid by demonstration eligible.
The PMPM costs for the calculation of the annual budget neutrality expenditure limit for the eligibility groups subject to the budget neutrality agreement under this demonstration are specified in Table 14.
The annual budget neutrality expenditure limit for the demonstration as a whole is the sum of the project annual expenditure limits for each EG calculated in subparagraph (i) above.

Table 14: Budget Neutrality Expenditure Limit
Eligibility Group	DY 16 (10/1/13 -3/31/15)	DY 17 (4/1/15 - 3/31/16)	Trend Rate	DY 18 (4/1/16 - 3/31/17)	DY 19 (4/1/17 - 3/31/18)	DY 20 (4/1/18 - 3/31/19)	DY 21 (4/1/19 - 3/31/20)	DY 22 (4/1/20- 3/31/21)
TANF Child	$756.70	756.7	0.046	791.51	827.92	866	907.77	952.07
TANF Adult	$1,027.04	1027.04	0.049	1077.36	1130.15	1185.53	1243.62	1304.56
SSI 0 through- 64	$2,646.00	2646	0.042	2757.13	2872.93	2993.59	3129.73	3274.21
SSI 65 and above	$1,625.00	1625	0.043	1694.88	1767.76	1843.77	1923.05	2005.74
Non-Duals 18- 64	$9,396.90	9396.9	0.042	9791.57	10202.82	10631.34	11077.86	11543.13
Non-Duals 65+	$8,403.92	8706.46	0.036	8706.46	9019.89	9344.61	9681.02	10029.54
MLTC Adult Age 18-64 Duals	$4,105.37	4105.37	0.0119	4154.22	4203.66	4253.68	4304.3	4355.52
MLTC Age 65+ Duals	$5,053.44	5053.44	0.0325	5217.68	5387.25	5562.34	5743.12	5929.77

The overall budget neutrality expenditure limit for the demonstration period is the sum of the annual budget neutrality expenditure limits calculated in subparagraph (a)(iv) above for each year. The federal share of the overall budget neutrality expenditure limit represents the maximum amount of FFP that the state may receive for expenditures on behalf of demonstration populations and expenditures described in Section X during the demonstration period.

Savings Phase-out. Each DY, the net variance between the without-waiver cost and actual with-waiver cost will be reduced for selected Medicaid population-based EGs. The reduced variance, to be calculated as a percentage of the total variance, will be used in place of the total variance to determine overall budget neutrality for the demonstration. (Equivalently, the difference between the total variance and reduced variance could be subtracted from the without-waiver cost estimate.) The formula for calculating the reduced variance is, reduced variance equals total variance times applicable percentage. The percentages for each EG and DY are determined based how long the associated population has been enrolled in managed care subject to this demonstration; lower percentages are for longer established managed care populations. The EGs affected by this provision and the applicable percentages are shown in the Table 15 below, except that if the total variance for an EG in a DY is negative, the applicable percentage is 100 percent.

Table 15: Savings Phase Out
Eligibility Group	DY 18 (4/1/16 - 3/31/17)	DY 19 (4/1/17 - 3/31/18)	DY 20 (4/1/18 - 3/31/19)	DY 21 (4/1/19 - 3/31/20)	DY 22 (4/1/20 - 3/31/21)
TANF Child	0.25	0.25	0.25	0.25	0.25
TANF Adult	0.25	0.25	0.25	0.25	0.25
SSI 0 through-64	0.5	0.4	0.3	0.25	0.25
SSI 65 and above	0.5	0.4	0.3	0.25	0.25
Non-Duals 18-64	1	1	0.9	0.8	0.7
Non-Duals 65+	1	1	0.9	0.8	0.7
MLTC Adult Age 18-64 Duals	1	1	0.9	0.8	0.7
MLTC Age 65+ Duals	1	1	0.9	0.8	0.7

Monitoring of New Adult Group Spending and Opportunity to Adjust Projections. For each demonstration year, a separate annual budget limit for the new adult group will be calculated as the product of the trended monthly per person cost times the actual number of eligible/member months as reported to CMS by the state under the guidelines set forth in Section X. The per capita cost estimates for the new adult group are listed in Table 16 below.

Table 16: Per Capita Cost Estimates for the New Adult Group
MEG	DY 16	DY 17	Trend	DY 18	DY 19	DY 20	DY 21	DY 22
New Adult Group	$722.5 7	722.57	0.042	752.92	784.54	817.49	851.82	887.6

If the state´s experience of the take up rate for the new Adult Group and other factors that affect the costs of this population indicates that the new Adult Group PMPM limit described above may underestimate the actual costs of Medical Assistance for the new Adult Group, the state has the opportunity to submit an adjustment to the PMPM limit, along with detailed expenditure data to justify this, for CMS review without submitting an amendment pursuant to Section II. To ensure timely adjustments to the PMPM limit for a demonstration year, the revised projection must be submitted to CMS for approval no later than October 1 in the year the adjustment is to take place.
The budget limit for the new adult group is calculated by taking the PMPM cost projections for the above group in each demonstration year, times the number of eligible member months for that group and demonstration year, and adding the products together across demonstration years. The federal share of the budget neutrality cap is obtained by multiplying total computable budget neutrality cap by the federal share.
The state will not be allowed to obtain budget neutrality "savings" from this population.
If total FFP reported by the state for the new Adult Group should exceed the federal share of FFP for the budget limit for the new Adult Group by more than 3 percent following each demonstration year, the state must submit a corrective action plan to CMS for approval.

Hypothetical Budget Neutrality. When expenditure authority is provided for coverage of populations or services that the state could have otherwise provided through its Medicaid state plan or other title XIX authority (such as a waiver under section 1915 of the Act), CMS considers these expenditures to be "hypothetical;" that is, the expenditures would have been eligible to receive FFP elsewhere in the Medicaid program. For these hypothetical expenditures, CMS makes adjustments to the budget neutrality test which effectively treats these expenditures as if they were for approved Medicaid state plan services. Hypothetical expenditures, therefore, do not necessitate savings to offset the otherwise allowable services. This approach reflects CMS´s current view that states should not have to "pay for," with demonstration savings, costs that could have been otherwise eligible for FFP under a Medicaid state plan or other title XIX authority; however, when evaluating budget neutrality, CMS does not offset non-hypothetical expenditures with projected or accrued savings from hypothetical expenditures. That is, savings are not generated from a hypothetical population or service. To allow for hypothetical expenditures, while preventing them from resulting in savings, CMS currently applies a separate, independent Hypothetical Budget Neutrality Tests, which subject hypothetical expenditures to pre-determined limits to which the state and CMS agree, and that CMS approves, as a part of this demonstration approval. If the state´s WW hypothetical spending exceeds the supplemental test´s expenditure limit, the state agrees (as a condition of CMS approval) to offset that excess spending by savings elsewhere in the demonstration or to refund the FFP to CMS.

Hypothetical Budget Neutrality Test 1: Fo1 Children. The table below identifies the MEGs that are used for Hypothetical Budget Neutrality Test 1. MEGs that are designated "WOW Only" or "Both" are the components used to calculate the budget neutrality expenditure limit. The Composite Federal Share for the Hypothetical Budget Neutrality Test is calculated based on all MEGs indicated as "WW Only" or "Both." MEGs that are indicated as "WW Only" or "Both" are counted as expenditures against this budget neutrality expenditure limit. Any expenditures in excess of the limit from Hypothetical Budget Neutrality Test are counted as WW expenditures under the Main Budget Neutrality Test.

Table 17: Hypothetical Budget Neutrality Test
MEG	PC or Agg*	WOW Only, WW Only, or Both	BASE YEAR [SFY 2017]	TREND	DY 21	DY 22
Fo1 Children	PC	Both	6787.67	0.012	7049.19	$7.131.67

Calculating the Federal Medical Assistance Percentage (FMAP) for Continuous Eligibility for the Adult Group. CMS anticipates that states that adopt continuous eligibility for adults would experience a 2 percent increase in enrollment. Based on this estimate, CMS has determined that 97.4 percent of the member months for newly eligibility in the Adult Group will be made at the enhanced FMAP rate and 2.6 percent will be matched at the regular FMAP rate.
State Reporting for the FMAP Adjustment. Newly eligible individuals in the Adult Group shall be claimed at the enhanced FMAP rate. The state must make an adjustment in the CMS-64W that accounts for the proportion of member months in which beneficiaries are enrolled due to continuous eligibility and could have been disenrolled due to excess income in absence of continuous eligibility (i.e. 2.6 percent). For the purposes of budget neutrality, the members for the Adult Group within the 2.6 percent of the population described in this STC will be treated as a hypothetical population. The state is not subject to use their budget neutrality savings towards providing continuous eligibility for this population.
Future Adjustments to the Budget Neutrality Expenditure Limit. CMS reserves theright to adjust the budget neutrality expenditure limit to be consistent with enforcement of impermissible provider payments, health care related taxes, new federal statutes, or policy interpretations implemented through letters, memoranda, or regulations with respect to the provision of services covered under the MRT demonstration.
Enforcement of Budget Neutrality. CMS shall enforce the budget neutrality agreement over the life of the demonstration extension, which for this purpose will be from April 1, 2016 through March 31, 2021. The budget neutrality test for the demonstration extension may incorporate net savings from the immediately prior demonstration period of October 1, 2011 through March 31, 2016 (including temporary extensions starting January 2015), but not from any earlier approval period. To incorporate savings from the October 1, 2011 through March 31, 2016 approval period, New York must provide CMS a certified and audited final assessment of budget neutrality for that period in which demonstration expenditures totals are consistent with the amounts reported by the state on the CMS-64 report (as summarized in the C Report).
Exceeding Budget Neutrality. If at the end of this demonstration period the overall budget neutrality expenditure limit has been exceeded, the excess federal funds must be returned to CMS. If the demonstration is terminated prior to the end of the budget neutrality agreement, an evaluation of this provision shall be based on the time elapsed through the termination date.
Budget Neutrality Monitoring Tool. The state will provide CMS with quarterly budget neutrality status updates using the Budget Neutrality Monitoring Tool provided through the Performance Metrics Database and Analytics (PMDA) system. The tool incorporates the "Schedule C Report" for comparing demonstration´s actual expenditures to the budget neutrality expenditure limits described in Section XI. CMS will provide technical assistance, upon request.

|top of section| |top of page|

XI. EVALUATION OF THE DEMONSTRATION

Required Evaluations. All evaluations must comply with the evaluation standards set forth in Section XI (2) and in and in 42 CFR §431.424.
1. Demonstration evaluation. On or before January 31, 2017, the state must submit to CMS for approval a draft design for the demonstration evaluation. At a minimum, the draft design must include a discussion of the goals, objectives, and hypotheses, with consideration of the beneficiaries, providers, plans, market areas, and/or expenditures specific to each of the programs. A separate design should be developed for each program, with sufficient methodological detail to determine scientific rigor, including its ability to demonstrate program effects, a pan for statistical analysis, and a description of the data sources to be used for each program.
  
  To obtain public comment for inclusion into the evaluation design, a draft will be posted to the NYSDOH Web site by December 20, 2016, with comments requested from the public by January 10, 2016 to allow for incorporation of comments received prior to submission of the draft design to CMS.
  
  The demonstration evaluation covers the overall demonstration and should include following domains of focus:
  1. MLTC
  2. MMMC
  3. Individuals Moved from Institutional Settings to Community Settings for Long Term Services and Supports
  4. Temporary Assistance to Needy Families
  5. Twelve-Month Continuous Eligibility Period
  6. Express Lanes Eligibility
  The evaluation of DSRIP, HARP, Children´s Design and the Self-Direction Pilot are to be conducted separately and are described elsewhere in the document. Specific evaluation questions are listed in Attachment O. The state may revise research questions with approval from CMS. Attachment O will be updated after the evaluation design for the Children´s Design is approved.
2. DSRIP evaluation. The DSRIP evaluation must follow all requirements as specified in Section VII above, as well as requirements noted in STC 2 of this section.
3. HARP evaluation. The state must respond to CMS comments on the draft evaluation design within 60 days of receipt of comments (see Attachment H). At a minimum, the evaluation of BH integration must examine the impact of HARPs on use of care and health outcomes for individuals eligible to receive BH HCBS benefits, the factors associated with individuals electing to or declining to enroll in HARPs, the cost effectiveness of HARPs, and the consequences of targeting availability of BH HCBS to a more narrowly defined population than the criteria in the state plan. Other research questions are listed in Attachment O. With approval from CMS, the state may change the research questions.
4. Self-Direction Pilot Evaluation. The state shall submit a draft evaluation design to CMS no later than March 31, 2018, including, but not limited to data that the state proposes to be used to evaluate the self-direction pilot (see Attachment F). A draft evaluation report is due six months prior to the end of the pilot. At a minimum, the evaluation must include:
  1. Measures of mental and physical health at baseline and after pilot program participation, consistent with the standards established in STC 2(c), 2(d), 2(e), and 2(f) of this section
  2. Utilization: hospitalizations, emergency visits, and primary care utilization
  3. Functioning: measures of work or school participation, food insecurity, and housing
  4. Quality of Life: life satisfaction, hope, community inclusion, and empowerment
  5. Cost: cost of behavioral health and other healthcare services
5. Children´s Design Evaluation Design. The state must submit a draft evaluation design to CMS 180 days after the approval of the amendment. The design should comply with all requirements identified in sections 2.b.iii, 2.c., and 2.d. below.
Core Evaluation Requirements. The following are requirements of all evaluations under the demonstration.
1. Cooperation with Federal Evaluators. Should HHS undertake an evaluation of any component of the demonstration, the state shall cooperate, to the greatest extent possible, fully with CMS or the evaluator selected by HHS; in addition, the state shall submit the required data to HHS or its contractor. Requests from HHS for information and data shall be made in a timely manner and provide the state with an adequate timeframe to provide the information as agreed to by CMS and the state.
2. Standards for Evaluation Design.
  1. The state shall engage the public in the development of its evaluation design. Each demonstration evaluation described in STC 1 of this section will meet the prevailing standards of scientific and academic rigor, as appropriate and feasible for each aspect of the evaluation, including standards for the evaluation design, conduct, and interpretation and reporting of findings. The demonstration evaluation will use the best available data; use controls and adjustments for and reporting of the limitations of data and their effects on results; and discuss the generalizability of results.
    
    The state shall acquire an independent entity to conduct the evaluation. The evaluation design must describe the state´s process to contract with an independent entity, ensuring no conflict of interest. The design, including the budget and adequacy of approach, is subject to CMS approval. A justification of the costs may be required by CMS if the estimates provided do not appear to sufficiently cover the costs of the design or if CMS finds that the design is not sufficiently developed.
    
    The evaluation design shall incorporate an interim and summative evaluation for each program, and will discuss the following requirements as they pertain to each:
    1. the scientific rigor of the analysis;
    2. a discussion of the goals, objectives and specific hypotheses that are to be tested;
    3. specific performance and outcomes measures used to evaluate the impact of each program;
    4. how the analysis will support a determination of cost effectiveness;
    5. a strategy to utilize data, including identification of existing data sources for the evaluation of each program, data collection as needed, sampling methodology, and statistical analysis;
    6. the potential effect of other initiatives and demonstration program interactions with those initiatives; and
    7. How the evaluation activities and reporting will be developed and maintained.
  2. CMS Response to Draft Evaluation Design. Within 30 days of receiving the draft evaluation design from the state, CMS will provide a response including any changes to be made to the evaluation design prior to final approval.
  3. Preparation of Final Evaluation Design. Within 60 days of receiving CMS´s response, the state will submit the final draft of the evaluation design, addressing the comments from CMS.
3. Evaluation Design Components. The Evaluation Design to be submitted to CMS by the state shall include the following core components to be approved by CMS:
  1. Research questions and hypotheses. This includes a statement of the specific research questions and testable hypotheses that address the goals of each of the programs.
  2. Study design. The design will include a description of the study design (e.g., cohort, controlled before-and-after studies, interrupted time series, case- control, etc.) specific to each of the programs, including a rationale for the design selected, with consideration to the potential confounding effects of other statewide health care reform initiatives. The discussion will include a proposed baseline and approach to comparison, as applicable. The discussion will also include an approach to benchmarking and should consider applicability of national and state standards. The application of sensitivity analyses as appropriate shall be considered.
  3. Performance Measures: This includes identification, for each hypothesis, of quantitative and/or qualitative process and/or outcome measures that adequately assesses the effectiveness of each of the programs with respect to enrollment, beneficiary characteristics, health status, and quality and cost of care. Nationally recognized measures should be used where appropriate. Measures will be clearly stated and described, with the numerator and dominator clearly defined. To the extent possible, the state will incorporate comparisons to national data and/or measure sets by pulling nationally recognized metrics such as from the National Quality Forum, Center for Medicare and Medicaid Innovation, meaningful use under HIT, and the Medicaid Core Adult sets.
  4. Data Collection: This discussion shall include: A description of the data sources; the frequency and timing of any data collection to be conducted; and the method of data collection. The following shall be considered and included as appropriate:
    1. Medicaid encounter and claims data in Transformed Medicaid Statistical Information System (TMSIS);
    2. enrollment data;
    3. EHR data, where available;
    4. semiannual financial and other reporting data;
    5. managed care contracting data;
    6. consumer and provider surveys; and
    7. other data needed to support performance measurement.
  5. Assurances Needed to Obtain Data: The design report will discuss the state´s arrangements to assure needed data to support the evaluation design are available.
  6. Data Analysis: This includes a detailed discussion of the method of data analysis, including appropriate statistical methods that will allow for program effects to be isolated from other initiatives occurring in the state, to the extent possible. The level of analysis may be at the beneficiary, provider, health plan, and program level, as appropriate, for further depth and to glean potential non-equivalent effects on different sub-groups. Sensitivity analyses shall be used when appropriate. Qualitative analysis methods shall also be described, if applicable.
4. Reporting Requirements
  1. Interim Evaluation Report. The state must submit an interim evaluation report as part of the state´s request for any future renewal of the demonstration.
  2. Final Summative Evaluation Report. The Final Summative Evaluation Report shall include the following core components:
    1. Executive Summary. This includes a concise summary of the goals of the Demonstration; the evaluation questions and hypotheses tested; and key findings including whether the evaluators find the demonstration to be budget neutral and cost effective, and policy implications.
    2. Demonstration Description. This includes a description of the Demonstration programmatic goals and strategies, particularly how they relate to budget neutrality and cost effectiveness.
    3. Study Design. This includes a discussion of the evaluation design employed including research questions and hypotheses; type of study design; impacted populations and stakeholders; data sources; and data collection; analysis techniques, including controls or adjustments for differences in comparison groups, controls for other interventions in the state and any sensitivity analyses, and limitations of the study.
    4. Discussion of Findings and Conclusions. This includes a summary of the key findings and outcomes, particularly a discussion of cost effectiveness, as well as implementation successes, challenges, and lessons learned.
    5. Policy Implications. This includes an interpretation of the conclusions; the impact of the demonstration within the health delivery system in the state; the implications for state and federal health policy; and the potential for successful demonstration strategies to be replicated in other state Medicaid programs.
    6. Interactions with Other State Initiatives. This includes a discussion of this demonstration within an overall Medicaid context and long range planning and includes interrelations of the demonstration with other aspects of the state´s Medicaid program, and interactions with other Medicaid waivers and other federal awards affecting service delivery, health outcomes and the cost of care under Medicaid.
  3. State Presentations for CMS. The state will present to and participate in a discussion with CMS on the final design plan at post approval. The state will present on its interim evaluation report that is described to in STC 2 of this section. The state will present on its summative evaluation in conjunction with STC 2 of this section.
  4. Electronic Submission of Reports. The state shall submit all required plans and reports using the process stipulated by CMS, if applicable.
  5. Public Access. The state shall post the final approved Evaluation Design, Interim Evaluation Report, and Summative Evaluation Report on the State Medicaid website within 30 days of approval by CMS.

|top of section| |top of page|

XII. SCHEDULE OF DELIVERABLES FOR DEMONSTRATION

DSRIP Section	Subject	Subsection Deliverable Description	Due/Completed
1.	IAAF	Report of Payments	Completed
9.	Pre-Implementation Activities	Comprehensive Quality Strategy (STC VI. .ss 11.) Attachments I and J updated	Completed Completed
10	Proposal and Project plan review	Independent Assessor scoring and NYS approval of PPS Project Plan Applications	Completed
11.	PPS project Monitoring	Quarterly Progress Reports Annual Learning Collaboratives Midpoint Assessment Annual PPS Site Visits PPS Performance Database (MAPP)	Ongoing Ongoing Completed Ongoing Ongoing
12.	Financial reporting	Quarterly reporting of funding sources for DSRIP Payments	Ongoing
14.	Statewide Accountability	Annual Statewide Performance Measures for 4 Milestones	DY3 completed; DY4 and DY5
17.	Improved Internal Controls	Management Control Document	Completed
18.	DSRIP Transparency	Public hearings twice a year	Ongoing
19. and 20.	Submission of Independent Evaluation Design	Draft and Final DSRIP Independent Evaluation Designs	Completed
23.	Independent Evaluation due dates	Interim Evaluation Report 2019 Summative Evaluation Report 2020	Interim due 90 days after DY4 Summative Preliminary report due 180 days after DY5
26.	Public Access	Posting of Evaluation Design, Interim Evaluation Report, and Summative Evaluation Report within 30 days of CMS approval.	Ongoing
27.	CMS Notification	After CMS approval of IE Summative Evaluation Report, public release notification to CMS of the report(s) or related journal articles is required for 24 months. CMS has 30 days to review and comment.	2022 - 2023
32.	DSRIP Implementation Monitoring	Operational Protocol - Appendix K	Completed
34.	DSRIP Quarterly reports to CMS		Ongoing
37.	Medicaid Managed Care DSRIP Contracting Plan	Annual VBP Roadmap	Ongoing

STC Section	1115 Deliverable	Due Date	Frequency
VIII	Q1 Operational Report	Due to CMS February 28	Ongoing
VIII	Q2 Operational Report	Due to CMS May 31	Ongoing
VIII	Q3 Operational Report	Due to CMS August 31	Ongoing
VIII	Q4 Operational Report	Due to CMS December 31	Ongoing
III	Post Award Public Forum	Annual	Ongoing
XI	Children´s Design Evaluation	180 days after approval of the amendment

|top of section| |top of page|

ATTACHMENT A

Mainstream Medicaid Managed Care (including HIV SNP and HARP) Benefits

Demonstration-Only Services
Inpatient and outpatient hospital services
Clinic services including Rural Health Clinic and Federally Qualified Health Center services
Laboratory and Radiology services
Home health services
Early Periodic Screening, Diagnosis, and Treatment services (for individuals under age 21 only)
Family planning services and supplies
Physicians services including nurse practitioner and nurse midwife services
Dental services
Physical and occupational therapy
Speech, hearing, and language therapy
Prescription drugs, over-the-counter drugs, and medical supplies including second generation long acting injectable antipsychotics and naltrexone extended release suspension under both the pharmacy and medical benefit
Durable Medical Equipment (DME), including prosthetic and orthotic devices, hearing aids, and prescription shoes
Vision care services, including eyeglasses
Intermediate Care Facilities for individuals with intellectual disabilities (ICF-IID)
Nursing facility services, including short term or rehabilitative services and permanent placement (Permanent placement is not covered by HARPs)
Personal care services
Consumer Directed Personal Assistance Services
Medical Social Services for persons transitioning from the LTHHCP who received the service under the LTHHCP (non-state plan service)
Home Delivered Meals for persons transitioning from the LTHHCP who received the service under the LTHHCP (non- state plan service)

Hospice care services
TB-related services
Inpatient and outpatient behavioral health services (mental health and chemical dependence services)
Emergency medical services, including emergency transportation
Adult Day Health Care and AIDS Adult Day Health Care
Personal Emergency Response Services (PERS)
Renal dialysis
Home and Community Based Services waivers (HCBS) including the Children´s HCBS waiver

Non-emergency transportation ⁴
Experimental or investigational treatment (covered on a case-by-case basis)
Health Home Care Coordination and Management
Residential Addiction Services only for services not covered in the State Plan

Crisis Intervention (after 1/1/2020 this will be only for individuals over the age of 21)
Licensed Behavioral Health Practitioner Services (for over age 21)

The state plan and demonstration behavioral health benefits below are being included in the MMMC plans.

	For Individuals 21 and Older	Applicable to Children/ Youth under age 21
State Plan Inpatient and Outpatient Behavioral Health Services in MMMCs, excluding rehabilitation services for residents of community residences	X	X
Clinic: Medically supervised outpatient withdrawal	X	X
Clinic: OASAS outpatient and opioid treatment program (OTP) services	X	X
Rehabilitation: OASAS outpatient rehabilitation programs	X	X
Clinic: Licensed clinic services (OMH services)	X	X
Outpatient Hospital: Comprehensive psychiatric emergency program including Extended Observation Bed (EOB)	X	X
Clinic: Continuing day treatment	X	X()
Clinic: Partial hospitalization	X	X (7/1/2019)
Rehabilitation: Personalized Recovery Oriented Services	X	X()
Rehabilitation: OASAS Chemical Dependence Inpatient Rehabilitation Treatment	X	X
Rehabilitation: Assertive Community Treatment	X	X()

Inpatient Hospital: Medically Managed detoxification (hospital based)	X	X
Inpatient Hospital: Medically supervised inpatient detoxification	X	X
Inpatient hospital: Inpatient treatment	X	X
Inpatient Hospital: Inpatient psychiatric services	X	X
Rehabilitation: Services for residents of community residences Note: these services are currently excluded from the behavioral health integration and will be phased into MMMC via contract amendments at a later date.	X	X
Physician: Psychiatric Services	X	X
Other Licensed Practitioner: Psychologist	X	X
Rehabilitation: Outpatient and residential addiction services (#16-004)	X	X
Outpatient hospital: OASAS Outpatient clinic	X	X
Outpatient hospital: OASAS Rehabilitation programs	X	X
Outpatient hospital: OASAS Opioid Treatment Program Services	X	X
Clinic: OMH designated clinics for Children with Diagnosis of Serious Emotional Disturbance (SED) (effective 7/1/2019)		X
EPSDT Community Psychiatric Support and Treatment		X
EPSDT Other Licensed Practitioner/Licensed Behavioral Health practitioner (LBHP)		X
EPSDT Psychosocial Rehabilitation Supports		X
EPSDT Family Peer Support
EPSDT Crisis Intervention services EPSDT Youth Peer Support and Training.		X
EPSDT VFCA per diem/Residential Supports and Services.		X

|top of section| |top of page|

ATTACHMENT B

Managed Long Term Care Benefits (both State Plan and LTSS benefits in the MLTC Plan Contract

Home Health Care*

Medical Social Services

Adult Day Health Care

Personal Care

Durable Medical Equipment**

Non-emergent Transportation

Podiatry

Dental

Optometry/Eyeglasses

Outpatient Rehabilitation PT, OT, SP

Audiology/Hearing Aids

Respiratory Therapy

Private Duty Nursing

Nutrition

Skilled Nursing Facilities***

Social Day Care

Home Delivered/Congregate Meals

Social and Environmental Supports

PERS (Personal Emergency Response Service)

*Home Health Services including Nursing, Home Health Aide, Physical Therapy (PT), Occupational Therapy (OT), Speech Pathology (SP)
**DME including Medical/Surgical, Hearing Aid Batteries, Prosthetic, Orthotics and Orthopedic Footwear
*** Limited to three months LTNHS in the Partially Capitated MLTC Plan

|top of section| |top of page|

ATTACHMENT C

Home and Community-Based Services Expansion Program Benefits

Assistive Technology (including personal emergency response system)

Community Integration Counseling and Services

Community Transition Services

Congregate/Home Delivered Meals

Environmental Modifications

Home and Community Support Services

Home Maintenance

Home Visits by Medical Personnel

Independent Living Skills Training

Intensive Behavioral Programs

Medical Social Services

Moving Assistance

Nutritional Counseling/Education

Peer Mentoring

Positive Behavioral Interventions

Respiratory Therapy

Respite Care/Services

Service Coordination

Social Day Care (including transportation)

Structured Day Program

Substance Abuse Programs

Transportation

Wellness Counseling Services

All HCBS Expansion program participants may not receive all benefits listed above. An individual participant´s access to the benefits below may vary based on the individual´s similarity to an individual determined eligible for and enrolled in the NHTD or TBI 1915(c) waiver program.

Home and community-based services (HCBS) must be provided in a setting that includes the following qualities:

The setting is integrated in and supports full access of individuals receiving Medicaid HCBS to the greater community, including opportunities to seek employment and work in competitive integrated settings, engage in community life, control personal resources, and receive services in the community, to the same degree of access as individuals not receiving Medicaid HCBS.
The setting is selected by the individual from among setting options including non- disability specific settings and an option for a private unit in a residential setting. The setting options are identified and documented in the person-centered service plan and are based on the individual´s needs, preferences, and, for residential settings, resources available for room and board.
Ensures an individual´s rights of privacy, dignity and respect, and freedom from coercion and restraint.
Optimizes, but does not regiment, individual initiative, autonomy, and independence in making life choices, including but not limited to, daily activities, physical environment, and with whom to interact.
Facilitates individual choice regarding services and supports, and who provides them.
In a provider-owned or controlled residential setting, in addition to the qualities specified above, the following additional conditions must be met:

The unit or dwelling is a specific physical place that can be owned, rented, or occupied under a legally enforceable agreement by the individual receiving services, and the individual has, at a minimum, the same responsibilities and protections from eviction that tenants have under the landlord/tenant law of the State, county, city, or other designated entity. For settings in which landlord tenant laws do not apply, the State must ensure that a lease, residency agreement or other form of written agreement will be in place for each HCBS participant, and that the document provides protections that address eviction processes and appeals comparable to those provided under the jurisdiction´s landlord tenant law.
Each individual has privacy in their sleeping or living unit:
- Units have entrance doors lockable by the individual, with only appropriate staff having keys to doors.
- Individuals sharing units have a choice of roommates in that setting.
- Individuals have the freedom to furnish and decorate their sleeping or living units within the lease or other agreement.
Individuals have the freedom and support to control their own schedules and activities and have access to food at any time.
Individuals are able to have visitors of their choosing at any time.
The setting is physically accessible to the individual.
Any modification of the additional conditions specified in items 1 through 4 above, must be supported by a specific assessed need and justified in the person- centered service plan. The following requirements must be documented in the person-centered service plan:
- Identify a specific and individualized assessed need.
- Document the positive interventions and supports used prior to any modifications to the person- centered service plan.
- Document less intrusive methods of meeting the need that have been tried but did not work.
  - Include a clear description of the condition that is directly proportionate to the specific assessed need.
  - Include regular collection and review of data to measure the ongoing effectiveness of the modification.
  - Include established time limits for periodic reviews to determine if the modification is still necessary or can be terminated.
  - Include the informed consent of the individual.
  - Include an assurance that interventions and supports will cause no harm to the individual.

Settings that are not Home and Community-Based:

For 1115 demonstrations that furnish HCBS services, settings that are not home and community- based are defined at §441.301(c)(5) as follows:

A nursing facility;
An institution for mental diseases;
An intermediate care facility for individuals with intellectual disabilities;
A hospital; or
Any other locations that have qualities of an institutional setting, as determined by the Secretary.

|top of section| |top of page|

ATTACHMENT D

Behavioral Health Home and Community Based Services in HARPS and HIV SNPs

Behavioral Health HCBS
BH HCBS Assessment BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment
Rehabilitation Psychosocial Rehabilitation Community Psychiatric Support and Treatment (CPST)
Empowerment Services-Peer Supports
Habilitation Services
Respite Short-term Crisis Respite Intensive Crisis Respite
Non-medical transportation
Family Support and Training
Employment Supports that are within the scope of HCBS that could be offered under the state plan that are described in a CMS-approved protocol
Education Support Services

*BH HCBS settings must adhere to the same HCBS setting qualities as listed in Attachment C.

|top of section| |top of page|

ATTACHMENT E

Quarterly Operational Report Format

Under Section XIII STC 4, the state is required to submit quarterly reports to CMS. The purpose of the quarterly report is to inform CMS of significant demonstration activity from the time of approval through completion of the demonstration. The reports are due to CMS 60 days after the end of each quarter (except for the report due for the quarter ending on September 30 of each demonstration year, which can be incorporated into the annual report required under Section IX).

The following report guidelines are intended as a framework and can be modified when agreed upon by CMS and the state. A complete quarterly progress report must include an updated budget neutrality monitoring workbook.

NARRATIVE REPORT FORMAT:

Title

Partnership Plan

Section 1115 Quarterly Report

Demonstration/Quarter Reporting Period: Introduction:

Information describing the goal of the demonstration, what it does, and key dates of approval/operation. (This should be the same for each report.)

Enrollment Information:

Please complete the following table that outlines all enrollment activity under the demonstration. The state should indicate "N/A" where appropriate. If there was no activity under a particular enrollment category, the state should indicate that by "0". Please note any changes in enrollment that fluctuate 10 percent or more over the previous quarter as well as the same quarter in the prior demonstration year.

Enrollment Counts

Note: Enrollment counts should be person counts, not participant months

Demonstration Populations (as hard coded in the CMS-64)	Current Enrollees (to date)	No. Voluntary Disenrolled in current Quarter	No. Involuntary Disenrolled in current Quarter
Population 1 - TANF Child under age1 through age 20 in mandatory MC counties
Population 2 - TANF Adults aged 21 through 64 in mandatory MC counties as of 10/1/06
Adult Group in MMMC
Population 9 - HCBS Expansion participants
Population 10 - MLTC Adults 18 through 64 - Duals
Population 11 - MLTC Adults age 65 and above - Duals

Total enrollment in each MCO by month

Voluntary Disenrollments:

Cumulative Number of Voluntary Disenrollments within Current Demonstration Year
- Number of Opt-outs for all HARP enrollees and reasons
- Number of Voluntary enrollments into HARPs
- Number of re-enrollments into HARPs
Reasons for Voluntary Disenrollments

Involuntary Disenrollments:

Cumulative Number of Involuntary Disenrollments within Current Demonstration Year
Reasons for Involuntary Disenrollments

Enrollment Information for Specific Sub-populations:

Enrollees in the HCBS Expansion program
Enrollees in the HIV SNP
Enrollees in the HARPs

Program Operations

Outreach/Innovative Activities: Summarize outreach activities and/or promising practices for the current quarter.

Operational/Policy Developments/Issues: Identify all significant program developments/issues/problems that have occurred in the current quarter, including, but not limited to, approval and contracting with new plans, benefit changes, and legislative activity. Also include any anticipated activities or program changes related to health care delivery, benefits, enrollment, grievances, quality of care, access, and other operational issues.

Update on Progress and Activities related to Quality Demonstrations and Clinic Uncompensated Care Funding: Identify all activities relating to the implementation of these programs, including but not limited to:

Release of solicitations and selection of awardees for the quality demonstrations;
An explanation of grants, contracts or other financial arrangements entered into for purposes of implementing the quality demonstrations of this demonstration; and
Progress of grantees in meeting the milestones identified in these STCs and any award documents.

Consumer Issues: A summary of the types of complaints or problems consumers identified about the program in the current quarter. Include any trends discovered, the resolution of complaints, and any actions taken or to be taken to prevent other occurrences, this should be broken out to show the number of LTSS complaints vs. all other categories identified. Also discuss feedback, issues or concerns received from the Medicaid Managed Care Advisory Review Panel (MMCARP), advocates and county officials.

Quality Assurance/Monitoring Activity: Identify any quality assurance/monitoring activity in current quarter.

Managed Long Term Care Program: Identify all significant program developments, issues, or problems that have occurred in the current quarter.

Home and Community-Based Services Expansion Program: For the quarter ending March 31 each year, attach a copy of the CMS-372 report completed in accordance with Appendix A of the approved Long-Term Home Health Care, the Nursing Home Transition and Diversion, and the Traumatic Brain Injury 1915(c) waivers.

Demonstration Evaluation: Discuss progress of evaluation implementation.

Financial/Budget Neutrality Developments/Issues: Provide information on:

Quality demonstration and clinic uncompensated care expenditures - to whom and when
Designated State Health Programs - amount of FFP claimed for the quarter

Enclosures/Attachments: Identify by title any attachments along with a brief description of what information the document contains.

State Contact(s): Identify individuals by name, title, mailing address, phone, fax, and email address that CMS may contact should any questions arise.

|top of section| |top of page|

ATTACHMENT F

Self-Directed Care Pilot

Overview

Background

Self-Directed Care

Self-directed care (SDC) gives the authority to the individual of using public dollars to purchase services and/or to employ service providers. By providing greater autonomy and choice, SDC can more flexibly match the needs of individuals for health care and related services. The ultimate goal of a better match between individual needs and services is to enhance progress toward recovery goals and improve health and stability in the community. In the U.S. and internationally SDC programs have been implemented extensively for populations including older adults, persons with physical disabilities, and persons with intellectual or developmental disabilities1. More recently, SDC programs for persons with behavioral health needs have been tried in a number of states including Florida, Texas, Oregon, Pennsylvania, Michigan, and Utah.

Research findings for self-directed care programs overall have found increased satisfaction, better outcomes, and cost neutrality (if not cost savings) compared to comparison groups. In the demonstration phase of the national Cash and Counseling program, a randomized control trial in three states found that elderly and disabled Medicaid recipients who self-directed personal assistance services had more satisfaction, fewer unmet needs and comparable or better outcomes than a control group receiving traditional agency-directed personal assistance services2,3. For mental health SDC a randomized control trial in Texas found that SDC participants had reduced symptoms and higher-levels of self-esteem and self-perceived recovery than the control group4. In both studies overall costs were similar for the SDC and control groups although the categories of cost were different: the SDC groups spent less on nursing care or inpatient services and more on personal assistance services and outpatient services than the comparison groups5, 6.

New York State Context

In August 2015, the Centers for Medicare and Medicaid Services (CMS) approved New York State´s request to implement Medicaid Managed Care (MMC) Health and Recovery Plans (HARPs) to integrate physical, behavioral health, and behavioral health (BH) home and community-based services (HCBS) for Medicaid enrollees with diagnosed severe mental illness (SMI) and/or substance use disorders (SUD). Under this 1115 waiver demonstration, HARPs are a separate coverage product that is targeted to Medicaid enrollees who meet need-based criteria for SMI and/or SUD established by the state. HIV Special Needs Plans (HIV SNPs) under MMC will also offer behavioral health HCBS services to eligible individuals meeting targeting, risk, and functional needs criteria.

SDC Pilot Program

Included under the 1115 waiver demonstration is a pilot program of Self-Directed Care for individuals with behavioral health needs. The pilot program will offer opportunities for self-direction in terms of service choice and payment for individuals in NYS who are eligible for the HARP benefit package and BH HCBS services. Two agencies, one in New York City and one outside New York City, have been chosen as sites for the SDC pilot. Additional sites may be added. The agencies will be responsible for recruiting and enrolling participants. The expected number of participants is 200 HARP enrolled and HCBS eligible individuals for the two sites, but may increase to 600 as additional sites are added. Each SDC participant will select a support broker who will work with the individual to identify recovery goals and assist in the creation and implementation of a budget to purchase those goods and services required to meet the recovery goals. Support brokers will be hired, trained and supervised at the participating agency sites. Support brokers will work with a fiscal intermediary who will provide training, support and monitoring for the authorization and purchasing of goods and services.

Pilot Evaluation

New York State will conduct an evaluation of the SDC pilot program using an external evaluator. The overall purpose of the SDC pilot evaluation is to provide policy makers and other stake-holders information related to the viability and effectiveness of the SDC program in NYS for the HARP behavioral health population, and to that end the evaluation will address the following pilot program goals: (1) Implementation of a viable and effective Self-Directed Care program for HARP enrolled/BH HCBS eligible individuals throughout New York State; (2) Improvement in recovery, health, behavioral health, and social functioning for SDC participants; and (3) Maintenance of Medicaid cost neutrality overall and reduction of behavioral health inpatient and crisis service utilization and cost for SDC participants. The evaluation plan will be finalized in an agreement with the independent evaluator. The evaluation will address the following questions to assess attainment of SDC pilot goals.

Goal 1: Implementation of a viable and effective Self-Directed Care program for HARP enrolled/HCBS eligible individuals throughout New York State

What are the characteristics of SDC participants and how do they compare to the larger HARP and HCBS eligible population?
What was the experience of HARP enrolled/HCBS eligible individuals participating in the SDC pilot program in relation to satisfaction with the SDC program and its impact on their recovery, quality of life, and benefit from health and behavioral health services?
What was the experience of non-participant stake-holders in the SDC pilot program (e.g., Support Brokers, pilot site agency staff, State program development/oversight staff, fiscal intermediary) in relation to SDC implementation including State oversight and contracting, fiscal policies and procedures, hiring of SDC staff, recruitment and work with participants, and coordination with the fiscal intermediary?
What were the facilitators and challenges to SDC pilot implementation and how would they impact state-wide roll-out?

Goal 2: Improvement in recovery, health, behavioral health, social functioning and satisfaction with care for SDC participants

Do HARP members have improved quality of life after participating in SDC?
Do HARP members show improved indicators of health, behavioral health and wellness after participating in SDC?
Do HARP members show improvement in education and employment after participating in SDC?
Do HARP members show improvement in community tenure (i.e. maintaining stable long-term independence in the community) after participating in SDC?
Do HARP members show improvement in social connectedness after participating in SDC?
Do HARP members report increased satisfaction with health and behavioral health services after participating in SDC?

Goal 3: Maintenance of Medicaid cost neutrality overall and reduction of behavioral health inpatient and crisis service utilization and cost for SDC participants

Does participation in SDC result in increased use and cost of outpatient behavioral health services and primary care?
Does participation in SDC result in decreased use and cost of behavioral health inpatient, emergency department and crisis services?
How does participation in SDC impact overall Medicaid spending?

Evaluation Framework

New York State will propose to the external evaluator that the evaluation of the SDC pilot program consist of two components: (1) a process evaluation of the implementation of the SDC pilot with the purpose of determining the viability of behavioral health SDC in New York State and assessing factors that will facilitate or challenge state-wide roll-out for HARP enrollees; and (2) an outcome evaluation to examine the impact of SDC on participant health, behavioral health, and quality of life as well as any impact on Medicaid spending.

Process Evaluation

It is expected that the Process Evaluation will be used to address the research questions relating to implementation of the program (specifically questions 1 through 4 listed under Goal 1 above). It will be suggested to the external evaluator that researchers will utilize qualitative methodologies to examine the perspectives of a variety of pilot participants including SDC participants, Support Brokers and pilot site agency leadership, Advisory Council members, and fiscal intermediary and Office of Mental Health program staff. The purpose of this evaluation is to assess the context and process of implementation of the pilot program and identify facilitators and barriers that could impact eventual implementation of a program for behavioral health Self- Directed Care throughout New York State.

Outcome Evaluation

It is expected that the Outcome Evaluation will be used to address the research questions relating to improvement in SDC participant recovery, quality of life, health and behavioral health, and satisfaction with care (specifically questions 1 through 6 under Goal 2 above). In addition, the Outcome Evaluation is expected to address the research questions on Medicaid service utilization and cost (questions 1 through 3 under Goal 3 above). The final design of the outcome evaluation will be agreed upon with the external evaluator. It is expected, however, that the design of the outcome evaluation will be quasi-experimental. Eligibility criteria for SDC participants includes Medicaid enrollment, HARP enrollment and eligibility for HCBS services. A comparison group would likely consist of Medicaid and HARP enrolled and HCBS eligible individuals served in locations where Self-Directed Care pilot programs are not available. Propensity score matching would be used to identify a comparison group comprised of Medicaid/HARP/HCBS eligible individuals who live in areas similar to the locations of the SDC sites and who are similar to the SDC participant group on important covariates. The comparison group would also allow the external evaluator to assess SDC program effects separately from the effects of other Medicaid Redesign initiatives implemented concurrently in New York State.

Evaluation Timeframe

It should be noted that this evaluation plan is conceived as approximately concurrent with the pilot demonstration program (see Evaluation Timeline in Table E below). If the evaluation were conducted at the end of the pilot demonstration program, there should be no impact on the Outcome Evaluation. However, the process evaluation of SDC pilot implementation may be impacted by the constraint of retroactively collecting qualitative data on implementation and participant perception of SDC.

Figure 1 shows a logic model of the SDC Pilot Demonstration showing expected resources, preliminary activities, implementation and intermediate outcomes, and long- term outcomes. The logic model provides a framework for both components of the evaluation. Data for the process evaluation of the implementation will come primarily from documents, site visits, interviews and focus groups. Data to inform the outcome evaluation will come from several sources. The Community Mental Health (CMH) Screen is conducted annually for all HARP enrolled/HCBS eligible individuals including SDC participants. This instrument is based on the InterRAI Community Mental Health Assessment, and gathers information about demographics, treatment history, housing, judicial system involvement, employment, education, risk behaviors, functional status, adverse life events, and social relationships. The HARP Perception of Care Survey will also be gathered annually from SDC participants and contains questions about quality of life and perception of care. The data from these two sources will be used to measure outcomes under Goal 2. Medicaid claims and encounter data will be used to measure changes in patterns of health and behavioral health service utilization and cost that address the questions under Goal 3. More detail on proposed evaluation methods and data sources are presented in the sections below.

Figure 1: SDC Pilot Logic Model

Evaluation Methods

It will be suggested to the external evaluator that for the process evaluation of SDC program implementation the primary method would be qualitative analyses of data from interviews, focus groups and documentation. For the outcome evaluation, it will be suggested that at least three analytic approaches be used. To gain a preliminary understanding of the characteristics of SDC participants, comparison group members and the larger HARP and HCBS eligible population, and to assess any differences in sub-groups (e.g., women, urban residents) descriptive statistics with corresponding graphical illustrations would be used. Assessment of outcomes over time for SDC participants (and in some domains for the comparison group) would be conducted using Generalized Linear Modeling with mixed effects (GLMM). GLMM enables multivariate modeling on different types of outcome variables including rates (e.g., outpatient service use), non-normal distributions (e.g., cost), and categorical or indicator variables (e.g., arrested in past year) as well as normally distributed continuous outcomes. Random effects could be incorporated in the models on two levels: for persons within areas/site and for change over time within persons. Incorporating random effects allows for the accurate modeling of heterogeneity and correlation within both the SDC population and comparison group. Difference-in-Difference (DD) analyses could also be conducted to compare change over time between the two groups. A DD analysis assesses whether the relationship between trends over time for two groups prior to a cut-off point changes after the cut-off point; the assumption is that without the intervention the relationship between the trends for the two groups would remain the same. In this case, the intervention is the Self-Directed Care pilot program, cut- off point is enrollment in the SDC pilot program, and patterns over time will be assessed for variables such as rates of behavioral health inpatient use or overall Medicaid spending. Table A below relates each Research Question to these methods. The specific methods are discussed in more detail below.

Comparison Group (PSM Group)

It will be suggested to the external evaluator that using Propensity Score Matching (see below), the comparison group be derived using the following approach. Comparison group members are required to be HARP enrolled and HCBS eligible, which by definition means that they have been administered the CMH screen and should be re- assessed using the CMH screen annually. The pool of individuals who have been assessed using the CMH screen state-wide is currently over 20,000. It will be suggested to the external evaluator that areas with similar features to the areas of the SDC site populations first be selected; for example, 10 other areas within New York City or other large urban areas like Buffalo or Rochester would be selected for matching to the NYC SDC location, and 10 areas of small cities would be selected for matching to the Newburgh SDC location. The number of areas selected could be increased if necessary, to get a sufficient pool for the next step. In the next step, Propensity Score Matching would be used to identify a comparison group matched to SDC participants using the CMH screen data and Medicaid claims data. As described below, the strategy would result in a larger (1:n) but analytically matched comparison group with covariate distributions balanced between the SDC group and the PSM comparison group.

An important aspect of the use of a comparison group is to control for the effects of other Medicaid Redesign initiatives implemented concurrently with SDC. It will be proposed to the external evaluator that the areas of the SDC sites and the other areas chosen for the comparison group be assessed for the presence of other initiatives and that these be factored into the balancing of the SDC and comparison groups on an area level. The comparison group would be used to partially address most of the research questions under Goals 2 and 3 using either GLMM or Difference-in-Difference described below. The exceptions are Research Questions 1 and 6 under Goal 2 as these rely solely on indicators contained in the HARP PCS. As the HARP PCS for non-SDC participants is based on annual random sampling it would not be used as a basis for the PSM comparison group but would be used to descriptively compare the larger HARP enrolled population to SDC participants. Table A presents the Research Questions, the proposed methods for addressing them, and whether they will involve group comparisons. This is followed by detailed descriptions of both the quantitative and qualitative methods.

Table A. Methods to Address Research Questions
Goal RQ#	Research Question	Quant./ Qual.	Method(s) (Data Sources)	Group Comparisons
1.1	What are the characteristics of SDC participants and how do they compare to the larger HARP and HCBS eligible population?	Quant.	-Descriptive statistics (CMH; HARP PCS; Medicaid)	-Larger HARP -PSM Group
1.2	What was the experience of HARP enrolled/HCBS eligible individuals participating in the SDC pilot program in relation to satisfaction with the SDC program and its impact on their recovery, quality of life, and benefit from health and behavioral health service?	Qual.	-Qualitative analyses (Participant E3Focus Groups)	NA
1.3	What was the experience of non-participant stake-holders in the SDC pilot program in relation to SDC implementation including State oversight and contracting, fiscal policies and procedures, hiring of SDC staff, recruitment and work with participants, and coordination with the fiscal intermediary?	Qual.	-Qualitative analyses (Documentation; Key Informant Interviews)	NA
1.4	What were the facilitators and challenges to SDC pilot implementation and how would they impact state-wide roll- out?	Qual.	-Qualitative analyses (Documentation; Key Informant Interviews)	NA
2.1	Do HARP members have improved quality of life after participating in SDC?	Quant.	Descriptive statistics -GLMM (HARP PCS)	None
2.2	Do HARP members show improved indicators of health, behavioral health and wellness after participating in SDC?	Quant.	-Descriptive statistics -GLMM (CMH, HARP PCS)	None
2.3	Do HARP members show improvement in education and employment after participating in SDC?	Quant.	-Descriptive statistics -GLMM (CMH, HARP PCS)	-PSM Group (CMH Only)
2.4	Do HARP members show improvement in community tenure after participating in SDC?	Quant.	-Descriptive statistics -GLMM (CMH, HARP PCS)	-PSM Group (CMH Only)
2.5	Do HARP members show improvement in social connectedness after participating in SDC?	Quant.	-Descriptive statistics -GLMM (CMH)	-PSM Group
2.6	Do HARP members report increased satisfaction with health and behavioral health services after participating in SDC?	Quant.	-Descriptive statistics -GLMM (HARP PCS)	None
3.1	Does participation in SDC result in increased use and cost of outpatient behavioral health services and primary care?	Quant.	-Descriptive statistics -DD (Medicaid Claims)	-PSM Group
3.2	Does participation in SDC result in decreased use and cost of behavioral health inpatient, emergency department and crisis services?	Quant.	-Descriptive statistics -DD (Medicaid Claims)	-PSM Group
3.3	How does participation in SDC impact overall Medicaid spending?	Quant.	-Descriptive statistics -DD (Medicaid Claims)	-PSM Group

Quantitative Methods

Quantitative Method I. Descriptive Statistics
The external evaluator will be asked to use descriptive statistics including frequencies, measures of central tendency (means, medians), and distributions (histograms, boxplots) to describe the characteristics of SDC participants, comparison group members, and HARP and HCBS eligible individuals more generally. To describe univariate differences or similarities between the SDC and comparison groups or between sub-populations of interest (e.g., based on site, gender, diagnosis), chi- square tests, t-tests or ANOVAs could be conducted depending on variable type. To describe simple differences between time periods (pre to post SDC) paired sample t-tests could be used. Bonferroni adjustments for multiple tests can be applied to the threshold p- value as necessary. Non-parametric tests might be used for measures that do not follow distributional assumptions.

Quantitative Method II: Longitudinal Mixed Effect Regression Method
The primary analytic approach suggested to assess change in the SDC participant group would be Generalized Linear Mixed Modeling (GLMM). GLMM can address the potential heterogeneity in the SDC pilot implementation effect and estimate an average program effect while controlling for important covariates7, 8. This framework has the advantage of separating the effects of time from that of the SDC implementation, accommodating the heterogeneity in the SDC implementation effect, and accounting for serial correlations within individuals (resulting from repeated measurements). Random effects could be included on one or two levels depending on the model and use of the comparison group. For all the models, change over time would be allowed to vary across individuals. This has the advantage that different numbers and times of measurements across individuals can be used; it also accurately accounts for correlation between measurements within individuals. These models could be used for HARP PCS data as well as CMH screen data for the SDC participant group. Random effects could also be used on the area/site level described in the section on the Comparison Group above. Individuals would be allowed to vary within areas to more accurately assess area level effects and to be able to identify SDC program effects apart from effects that may result from differences in areas (e.g., large urban versus small city; additional service initiatives). These models would be used with the PSM Comparison Group but limited to CMH screen or Medicaid claims data. As with implementation longitudinal data, the outcome metrics such as employment, enrollment in formal education, social relationships, social strengths, and behavioral health service utilization may vary considerably over time due to a strong temporal trend before and/or after program implementation. Explanatory risk factors including homelessness, criminal justice involvement (arrest history, incarceration history), alcohol use, drug use, chronic physical health conditions, and traumatic life events would likely vary considerably over time. The GLMM framework helps determine the amount of variability that may be due to temporal trend and the amount due to the new program implementation. The GLMM was chosen because it accounts for the intrinsic differences among individuals, the variability in program impact on individuals, and the correlation potentially induced by collecting data on the same individuals over time. GLMM could also usefully incorporate the PSM comparison group to look at differences over time in outcomes between SDC and the comparison group with the ability to more accurately model differences in persons by area. This would enable detection of program effects by separately comparing the two program site areas with similar areas in NYS.

Quantitative Method III. Difference in Difference Analysis
The primary method suggested to the external evaluator to assess differences in service use, cost and outcomes between the SDC participant group and the quasi- experimental comparison group would be a Difference in Difference (DD) analysis. This approach or strategy accounts for any secular trend/changes in the outcome metrics (it eliminates fixed differences not related to program implementation), with remaining significant differences attributable to the impact of program implementation9. The study groups would be prepared by match-pairing individuals using propensity scores derived from logistic regression based on selected demographic, clinical and social indicators, and health care utilization characteristics (see Quantitative Method IV). The outcome metrics, health care costs per member per month (PMPM) and service use rates, such as hospital admission rates, will be measured over two consecutive periods. Periods of two years prior and two years following program enrollment could be assessed as a preliminary examination of changes in trends. Additionally, periods of four years before and four years after program enrollment could be calculated for a total duration of eight years. Changes in outcome metrics from prior measurement periods to post measurement periods would be compared. Although the approximate measurement periods for two years are pre-period (January 1, 2016 - December 31, 2017) and post-period (January 1, 2018 - December 31, 2019), the actual trends will be based on SDC participant enrollment. For example, for a participant whose enrollment was on June 1, 2018 their last pre-period month would be May 2018 and their first post-period month would be June 2018. Their n matches in the PSM comparison group would be assigned the same pre and post periods. Averages over years would be calculated from the PMPM rates.

Figure 3: Idealized representation of DD Method

Quantitative Method IV: Propensity Score Matching
Propensity score matching is a technique developed to mimic randomization in observational studies like the SDC pilot evaluation10. A propensity score is the probability that an individual would be assigned to the treatment (SDC) versus comparison group conditioned on a set of observed covariates, such as demographics, diagnosis, service utilization history, and other factors. An advantage to propensity score matching is that a large set of potentially confounding covariates can be included without a loss of observations. This method would be applied in the design phase with application for a variety of causal models which may be selected. The propensity scores will be estimated using logistic regression, with the outcome being SDC participation, and predictors being derived from an array of demographic, clinical and social indicator constructs. The potential confounders will be selected a priori based on subject matter knowledge and in consultation with subject matter experts. Matching will also be done on timing of assessments. A greedy matching algorithm with an appropriate matching ratio of SDC participants to not SDC participants (1:n) will be used to create a matched analytic cohort based on the estimated propensity score10, 11. Balance in covariate distribution between SDC participants and not SDC participants in the matched analytic cohort will be assessed with weighted standardized difference12. The matched cohorts will be used for the quantitative methods indicated above as suggested in Table A.

Consumer Survey
The broader evaluation of the HARP Managed Care enrollment program has developed a member survey, the HARP Perception of Care Survey (HARP PCS), designed to measure experience with care, perception of care and perception of quality of life. Although members enrolled in HARPs and BH HCBS eligible members enrolled in HIV SNPs are being surveyed annually through a random sampling, all SDC participants in the pilot program will be asked to complete the survey annually.

During the development of the HARP PCS, several validated instruments intended to assess consumer perception of the performance of health plans and behavioral health services were reviewed. The HARP PCS was derived from those instruments. Specifically, questions were drawn from the Experience of Care and Health Outcomes (ECHO) Survey, the Mental Health Statistics Improvement Program (MHSIP) Survey, the Personal Wellbeing Index adult version (PWI-A), and the Maryland Outcomes Measurement System. NYS OMH also formulated questions for pertinent topic areas where none could be found in existing instruments. The majority of questions address domains of member experience such as accessibility of services, quality of services, and appropriateness of care, wellness, quality of life, and social connectedness. Additionally, a set of socio-demographic questions are included which will allow examination of disparities.

The HARP PCS was piloted by NYS OMH in the fall of 2016 with 8 NYS OMH (4) and OASAS (4) funded behavioral health programs. Peers and staff at the programs received training on survey administration from OMH. Feedback was gathered from pilot participants about the length of the survey, clarity of the questions, and relevance of the questions. Results from the pilot were analyzed and the final version of the survey developed. Initial administration to random samples of HARP enrollees was conducted in 2017 and will be continued annually. The survey is being implemented using two random samplings of HARP enrollees. One random sample selects service providers who serve at least 15 HARP members in mental health or substance use disorder specialty services; all HARP members receiving the service are surveyed. A second random sample uses direct mailing to HARP members. Over 3000 HARP members were asked to complete the survey in 2017.The survey consists of 61 questions found in Appendix C.

Qualitative Methods
The final plan for the process evaluation will be determined together with the external evaluator. It will be suggested to the external researchers that the process evaluation address Goal 1 through collection of documentation, administrative data, and qualitative data from key informant interviews and focus groups. Documentation would comprise program specification, policy and related documents developed by the Office of Mental Health, SDC Advisory Council, fiscal and administrative entities, and pilot site agencies. Topics might include descriptions of administrative and fiscal intermediaries and pilot site agencies, how they were selected, and their operations; structure, membership and meeting minutes of the SDC Advisory Council; eligibility criteria and recruitment strategies; credentialing, hiring, training and supervision of support brokers; budget allocations and financial rules including authorized and prohibited goods and services; and other areas. Administrative data routinely collected from the fiscal and administrative intermediaries and the pilot agencies could also be used to describe ongoing processes between participants, support brokers, and administrative bodies.

For example, the process of participants working with support brokers to develop budgets based on recovery goals, requesting and receiving approval and funds from the administrative and fiscal intermediaries, and documenting final purchases is being recorded in an application with data that can be made available to the external evaluator.

It will be suggested that interviews be held with key personnel from OMH Bureau of Program and Policy Development; SDC Advisory Council; and the fiscal intermediary. It will be suggested that site visits to each pilot site be scheduled within the first nine months from start-up and annually thereafter. It will be proposed that focus groups, which often lead to expanded discussion on mutual topics, be scheduled with at least a subset of SDC participants depending on the numbers enrolled13. At a minimum, 1 to 3 focus groups would be scheduled annually at each site involving 25 to 30 participants. Site agencies would be asked to help recruit participants to focus groups and the external evaluator would be asked to provide a gift card for participants attending the groups. Focus group topics would be expected to include participant perceptions about the process of developing recovery plans and budgets; relationships between participants and support brokers; satisfaction with health and behavioral health services; and SDC impact on participant recovery and quality of life. It will be suggested that interviews also be scheduled with all support brokers, and leadership and supervisory staff at the pilot site agencies. Topics would include relationships with administrative and fiscal intermediaries; credentialing, hiring, training and supervision of support brokers; budget allocations and financial rules including authorized and prohibited goods and services; process of recovery plan and budget development and purchasing of goods and services; relationships between SDC participants, Support Brokers and other staff; and facilitators and challenges of pilot program implementation. Interviews and focus groups would be conducted using semi-structured protocols to allow for data collection on pre- established topic areas and openness to other topic areas of potential interest to the evaluation.

Qualitative Analysis Method
The qualitative data analysis method will be finalized by the external evaluator. One suggested approach would be for the external researchers to follow a framework described by Bradley, Curry, & Devers14 that has been effectively used in health services research. This involves preliminary review of the data using a grounded theory approach (i.e. without predetermined categories) performed to identify emergent themes. A coding structure is then established through an iterative process that labels concepts, relationships between concepts, and, if applicable, evaluative participant perspectives (i.e., statements that are positive, negative, or indifferent to their experiences or observations). Where appropriate (e.g., for interview data) the coding structure also captures respondent characteristics (e.g., age, sex, support broker or position or role in organization) and setting (e.g., pilot site, region). Responses are then re-reviewed independently by at least two researchers, applying the finalized coding structure. Coding discrepancies between reviewers are subsequently resolved through discussion to achieve consensus for the final coding of the data. Coded data is analyzed and interpreted to identify major concept domains and themes.

Integration of Quantitative and Qualitative Methods
It will be suggested to the external evaluator that findings from quantitative and qualitative analyses be integrated in order to refine and deepen the results from the different methods. For example, qualitative information from participant focus groups could be combined with quantitative findings on change indicators (Goal 2) to gain a more nuanced understanding of participant outcomes. In addition, barriers and facilitators of SDC implementation identified through the qualitative data and methods of the process evaluation could be combined with quantitative findings derived from the two pilot sites to gain an understanding of whether there are elements critical to effective implementation. This approach will be particularly important if additional sites are added.

Evaluation Tools

Goal 1: Implementation of a viable and effective Self-Directed Care program for HARP enrolled/HCBS eligible individuals throughout New York State

Evaluation Questions

What are the characteristics of SDC participants and how do they compare to the larger HARP and HCBS eligible population?
What was the experience of HARP enrolled/HCBS eligible individuals participating in the SDC pilot program in relation to satisfaction with the SDC program and its impact on their recovery, quality of life, and benefit from health and behavioral health services?
What was the experience of non-participant stake-holders in the SDC pilot program (e.g., Support Brokers, pilot site agency staff, State program development/oversight staff, fiscal intermediary) in relation to SDC implementation including State oversight and contracting, fiscal policies and procedures, hiring of SDC staff, recruitment and work with participants, and coordination with the fiscal intermediary?
What were the facilitators and challenges to SDC pilot implementation and how would they impact state-wide roll-out?

To address Goal 1, we would suggest that the external evaluator use Quantitative method I (Descriptive Statistics) to address question 1 by describing the characteristics and service utilization patterns of SDC participants and how they compare to the larger HARP enrolled/HCBS eligible population. The remaining questions under Goal 1 would be addressed at the discretion of the external evaluator using qualitative methods such as those described above. Suggested measures, data sources, and methods are listed below in the Evaluation Tool for Goal 1 (Table B).

Table B: Evaluation Tool for Goal 1
Q #	Implementation Indicator	Measure	Data Source	Related Expectation	Possible Methodologies
Q1	SDC participant enrollment	Count SDC participants stratified by demographic, clinical, health and functional characteristics	Pilot site enrollment data CMH Screen data HARP PCS data Medicaid claims data	Members of HARP/HCBS population will be enrolled for participation in SDC at the two pilot sites	Descriptive analysis of Pilot site enrollment data Descriptive analysis of CMH Screen, HARP PCS and Medicaid claims data comparing SDC enrollees to larger HARP/HCBS population
Q2	SDC participant recovery, quality of life, health and behavioral health services	Describe participant perspectives on SDC program, staff and process; impacts on their recovery, quality of life, health and behavioral health; satisfaction; with services	Transcripts of SDC participant focus groups	Participants will gain experience with budgeting and using funds to meet recovery goals with resulting improvement in satisfaction with services, recovery, quality of life, and health/ behavioral health	Qualitative analysis of themes and concepts derived from transcripts of focus groups
Q3	State oversight and contracting	Describe program polices regarding the selection, agreements made and ongoing monitoring of SDC sites and fiscal intermediary	OMH administrative documentation OMH administrative staff interviews	OMH administrative staff will develop selection criteria, contract deliverables and procedures for ongoing monitoring for both pilot site agencies and the fiscal intermediary	Description of the OMH policies regarding SDC program implementation Qualitative analysis of themes and concepts from interviews
Q3	Fiscal policies and procedures	Describe program policies regarding participant eligibility criteria, budgeting/use of funds, conflict of interest, and complaint/ incident handling	OMH administrative documentation OMH administrative staff interviews Pilot site staff interviews	OMH administrative staff will develop fiscal policy and oversee fiscal intermediary and pilot site implementation	Description of the OMH policies regarding SDC program implementation and fiscal policy Qualitative analysis of themes and concepts from interviews
Q3	SDC support broker and supervisory staff hiring and training	Describe support broker and supervisory staff demographics, credentials, training, supervision and their perspectives on the pilot program and their relationship with participants and fiscal and state oversight	Pilot site documentation on hiring, training and supervising of support brokers Transcripts from interviews with support brokers, pilot site agency leadership/supervisory, fiscal intermediary and state oversight staff	Support brokers will be hired, trained and supervised by pilot sites and will interact with SDC participants and supervisory, fiscal intermediary and state oversight to facilitate SDC among participants	Description of documentation regarding the hiring, training and supervision of support brokers for each site Qualitative analysis of themes and concepts derived from interviews
Q3	SDC participant recruitment, enrollment and program participation	Describe pilot site agencies process for recruiting participants, educating participants about what SDC is and how they can participate, enrolling participants and facilitating ongoing participation	Pilot site administrative documents Pilot site staff interviews SDC participant focus groups	Pilot sites will work within OMH administrative policy to recruit, enroll, and facilitate ongoing participation in SDC	Description of the pilot site policies regarding SDC program implementation Qualitative analysis of themes and concepts from interviews and focus groups
Q3	Fiscal intermediary practices and coordination	Describe fiscal intermediary´s policy and infrastructure for providing payments, monitoring payments and supporting customers	Fiscal intermediary administrative and technical documents Interviews with fiscal intermediary staff, pilot site staff, state oversight staff	Fiscal intermediary will develop a web-based system for entering, approving and monitoring participant spending and will provide customer service to support brokers and SDC participants	Description of the fiscal intermediary´s process for payments, monitoring and assisting support brokers and participants Qualitative analysis of themes and concepts from interviews
Q4	Facilitators and challenges to SDC pilot implementation	Identify and describe facilitators and challenges to the implementation of the SDC pilot program	Interviews with state oversight, fiscal intermediary, pilot site agency staff Focus groups with participants	State oversight, pilot site agencies, and SDC participants will encounter both opportunities and barriers in the SDC process	Qualitative analysis of themes and concepts from interviews and focus groups

Goal 2: Improvement in recovery, health, behavioral health, social functioning and satisfaction with care for SDC participants

Evaluation Questions

Do HARP members have improved quality of life after participating in SDC?
Do HARP members show improved indicators of health, behavioral health and wellness after participating in SDC?
Do HARP members show improvement in education and employment after participating in SDC?
Do HARP members show improvement in community tenure (i.e. maintaining stable long-term independence in the community) after participating in SDC?
Do HARP members show improvement in social connectedness after participating in SDC?
Do HARP members report increased satisfaction with health and behavioral health services after participating in SDC?

To address Goal 2, we would propose that the external evaluator assess changes in outcomes for SDC participants between baseline and multiple follow up points over the four years of the pilot program (January 1, 2018-March 31, 2021) using data from the Community Mental Health (CMH) Screen and HARP PCS. We would suggest using GLMM models (Quantitative Method II) that allow time points to vary both in number and spacing, and also adjust for correlation between measures taken at different time points for an individual. This approach will assess average trends on outcome measures derived from the CMH Screen and HARP PCS for SDC participants while controlling for possible confounding factors. Data from the PSM comparison group could be included to examine differences for HARP members participating in SDC versus those who are not, on Research Questions 2-5 using data from CMH. HARP PCS data, which Research Questions 1 and 6 rely upon, is not available for comparison group analyses. The Evaluation Tool for Goal 2 (Table C) presents outcome indicators, measures, data sources, hypotheses and methods for each question.

Table C: Evaluation Tool for Goal 2
Q #	Implementatio n Indicator	Measure	Data Source	Related Expectation	Possible Methodologies
Q1	Participant quality of life	Life satisfaction scale Quality of life scale	HARP PCS	Quality of life will improve between baseline and three year and subsequent follow- up for SDC participants	GLMM
Q2	Participant behavioral health	Tobacco use Alcohol use Illegal drug use Misuse of prescription medications Difficulty due to substance use Reduced ideation/acts of harm to self/others	CMH Screen HARP PCS	Indicators of behavioral health will improve between baseline and three year and subsequent follow-up for SDC participants	GLMM
Q2	Participant physical health	Health status Difficulty due to physical Health	CMH Screen HARP PCS	Health indicators will improve between baseline and three year and subsequent follow-up for SDC participants	GLMM
Q3	Participant employment and participation in education	Employment status Hours worked in competitive Employment Educational status enrollment in Educational program	CMH Screen HARP PCS	Participation in employment and/or educational activities will increase between baseline and three year and subsequent follow-up for SDC participants	GLMM
Q4	Participant community tenure and stability	Residential status/housing stability Arrest, incarceration, other legal involvement AOT order Functional independence	CMH Screen HARP PCS	Stability in the community will improve between baseline and three year and subsequent follow-up for SDC participants	GLMM
Q5	Participant social connection	Social relationship strengths Level of social activity	CMH Screen	Social connected-ness will increase between baseline and three year and subsequent follow-up for SDC participants	GLMM
Q6	Participant satisfaction with care	Quality of Care Helpfulness of Services	HARP PCS	Satisfaction with care for behavioral health services will improve between baseline and three year and subsequent follow-up for SDC participants	GLMM

Goal 3: Maintenance of Medicaid cost neutrality overall and reduction of behavioral health inpatient and crisis service utilization and cost for SDC participants

Evaluation Questions

Does participation in SDC result in increased use and cost of outpatient behavioral health services and primary care?
Does participation in SDC result in decreased use and cost of behavioral health inpatient, emergency department and crisis services?
How does participation in SDC impact overall Medicaid spending?

To address Goal 3, we would propose a more rigorous approach to identify change in Medicaid service utilization and spending patterns using a Difference-in-Difference analysis (Quantitative Method III). The DD analysis would employ the quasi- experimental comparison group derived using Propensity Score Matching (Quantitative Method IV). The DD analysis can assess how change in service use and cost for SDC participants from the pre-period before SDC participation to the post-period compares to patterns in the same timeframes for the comparison group. The Evaluation Tool for Goal 3 (Table D) presents outcomes, measures, data sources, hypotheses and methods for each question.

Table D: Evaluation Tool for Goal 3
Q #	Implementatio n Indicator	Measure	Data Source	Related Expectation	Possible Methodologies
Q1	Participant use of outpatient behavioral health services	Claims for behavioral health outpatient services	Medicaid Claims and Encounters	Outpatient behavioral health service use will increase between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q1	Participant use of primary care	Claims for primary care visits	Medicaid Claims and Encounters	Use of primary care will increase between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q2	Behavioral health inpatient stays	Rates of admissions and days for behavioral health inpatient stays	Medicaid Claims and Encounters NYS OMH State Psychiatric Center records (MHARS)	Inpatient stays for behavioral health will decrease between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q2	Use of emergency department and behavioral health crisis services	Rates of behavioral health emergency department use Rates of non-behavioral health ED use Rates of behavioral health crisis service use	Medicaid Claims and Encounters	Emergency department and behavioral health crisis service use will decrease between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q3	Spending on behavioral health outpatient services	Cost per member per month of Behavioral health outpatient services	Medicaid Claims and Encounters	Spending on behavioral health outpatient services (including non-traditional services) will increase between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q3	Spending on primary care	Cost per member per month of primary care	Medicaid Claims and Encounters	Spending on primary care will increase between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q3	Spending on ED, behavioral health inpatient and crisis service use	Cost per member per month of ED use, and behavioral health inpatient and crisis services	Medicaid Claims and Encounters	Spending on ED and behavioral health inpatient and crisis service use will decrease between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference
Q3	Overall Medicaid spending	Overall Medicaid cost per member per month	Medicaid Claims and Encounters	Overall Medicaid spending will stay the same between baseline and three year and subsequent follow-up for SDC participants	Difference in Difference

Evaluation Timeline
Table E presents a suggested timeline of Evaluation activities and deliverables for the external evaluator.

Table E. Suggested Evaluation Timeline
Evaluation Activity	2019		2020		2021		2022
	Q1-2	Q3-4	Q1-2	Q3-4	Q1-2	Q3-4	Q1-2	Q3-4
Qualitative Data
Collect Documentation	x	x	x
Conduct Participant Focus Groups	x		x		x
Conduct Key Informant Interviews		x		x		x
Quantitative Data
Administer HARP Survey (to SDC)	x		x		x
Prepare Comparison Group (PSM)			x	x
Prepare CMH Data			x
Prepare Medicaid Claims Data			x
Prepare HARP PCS Data			x
Data Analyses
Qualitative Analyses				x	x	x	x
Descriptive Analyses				x	x
GLMM					x	x	x
Difference-in-Difference					x	x	x
Integrate Qualitative & Quantitative						x	x
Reporting and Dissemination
Preliminary Descriptive Report						x
Final Report								x
Presentations						x	x	x

References

An Environmental Scan of Self-Direction in Behavioral Health: Summary of Major Findings. Chestnut Hill, MA: National Resource Center for Participant-Directed Services, 2013. Available online here.
Cook, J.A., Russell, C., Grey, D.D., and Jonikas, J.A. (2008). A self-directed care model for mental health recovery. Psychiatric Services 59(6), 600-602.
Lepidus-Carlson, B., Foster, L., Dale, S.B., and Brown, R. (2007). Effects of Cash and Counseling on personal care and well-being. Health Services Research 42(1), 467-487.
Cook, J.A., Shore, S.E., Burke-Miller, J.K., Jonikas, J.A., Ferrara, M., Colegrove, S., Norris, W.T., Ruckdeschel, B., Batteiger, A.P., Ohrtman, M., Grey, D.D., and Hicks, M.E. (2010). Participatory action research to establish self-directed care for mental health recovery in Texas. Psychiatric Rehabilitation Journal 36(2), 137-144.
Dale, S., Brown, R., Phillips, B., Schore, J., and Leipidus-Carlson, B. (2003). The effects of Cash and Counseling on personal care services and Medicaid costs in Arkansas. Health Affairs Web Exclusive, W3-566-575.
Cook, J.A., Burke-Miller, J.K., Jonikas, J.A. et al. (2015). Preliminary results of the Texas Self-Directed Care Program randomized controlled trial. Chicago, IL: University of Illinois at Chicago Center on Self-Directed Recovery and Integrated Health Care.
Diggle, P.J.; Heagerty, P.; Liang, K-Y.; Zeger, SL. Analysis of Longitudinal Data. New York: Oxford University Press; 2002.
Tooze JA, Grunwald GK, Jones RH. Analysis of repeated measures data with clumping at zero. Statistical Methods in Medical Research. 2002; 11:341-355. [PubMed: 12197301]
Jeffrey S. Harman, Christy H. Lemak, Mona Al-Amin, Allyson G. Hall, and Robert Paul Duncan, Changes in Per Member Per Month Expenditures after Implementation of Florida´s Medicaid Reform Demonstration, Health Services Research 2010.01226
Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med 2007; 26(4): 734-753. DOI: 10.1002/sim.2580
Macro %gmatch. Available here. Accessed September 15, 2011.
Austin PC. Assessing balance in measured baseline covariates when using many-to-one matching on the propensity-score. Pharmacoepidemiol Drug Saf 2008; 17(12): 1218-1225. DOI: 10.1002/pds.1674
Kaplowitz, M.D., and Hoehn, J.P. (2001). Do focus groups and individual interviews reveal the same information for natural resource valuation? Ecological Economics 36, 237-247.
Bradley, A.H., Curry, L.A., and Devers, K.J. (2007). Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Services Research, 42(4), 1758-1772.

|top of section| |top of page|

Appendix A
Data Sources

Pilot Site Enrollment Data
OMH has designed a secure web application for use by SDC Participants and Support Brokers to develop and manage SDC budgets based on personal recovery plans and goals. Data from this application includes SDC enrollment information by site and recovery goal-related expenditures. The application data can be linked to Medicaid claims data.

Medicaid Claims
This database contains billing records for health care services, including pharmacy, for approximately million individuals enrolled in Medicaid in a given year. Also included are data on Medicaid enrollment status, diagnoses and provider associated with the billed services. The Medicaid claims database is updated on a monthly basis to include additional claims and modifications to existing claims. Medicaid claims database will receive data from all managed care plans providing services to the demonstration population. Given the claims processing, there is a 6-month lag in the availability of complete and finalized Medicaid claims data, where data for a given year are considered final by June 30th of the following year.

Community Mental Health (CMH) Screen
The Uniform Assessment System contains CMH Screen data on HARP eligible individuals enrolled in HARPs or HIV SNPS. Data include patient functional status, living situation, employment, education, health status, cognitive functioning, substance use, harm to self and others, stress and trauma and social relations. Data are a mix of self- reported information and information that is available to assessors through the care management process. HCBS eligibility requires an annual re-assessment using the CMH screen. This applies to both SDC enrollees and the PSM comparison group.

HARP Perception of Care Survey
The HARP Perception of Care Survey (HARP PCS) will be administered to all SDC participants annually. For non-SDC HARP members enrolled in HARP or HIV-SNP plans, a random sample of members is surveyed annually to measure perception of care and quality of life outcomes. The survey instrument was piloted in late 2016. The final instrument consists of 61 questions (see Appendix C). The survey is being implemented using two random samplings of HARP enrollees by product line for HARPs and HIV SNPs. One random sample selects service providers who serve at least 15 HARP members in mental health or substance use disorder specialty services; all HARP members receiving the service are surveyed. A second random sample uses direct mailing to HARP members. Over 3000 HARP members were given the survey in 2017. Specific survey domains include Perception of Outcomes, Daily Functioning, Access to Services, Appropriateness of Services, Social Connectedness, and Quality of Life. Findings will be examined for change in BH services satisfaction levels over time. Data will be self-reported and from a sample of HARP members. The experiences of the survey respondent population may be different than those of non-respondents with respect to their health care services. Therefore, data users should consider the potential for non-response bias when interpreting HARP PCS results.

NYS OMH Psychiatric Center Records
OMH maintains the Mental Health Automated Records System (MHARS) for episodes of inpatient, residential, and outpatient care in New York State Psychiatric Centers. This data will be used to identify psychiatric inpatient stays not included in Medicaid claims data.

|top of section| |top of page|

Appendix B

Community Mental Health Screen

(see Attachment on Medicaid.gov for complete Appendix B)

|top of section| |top of page|

Appendix C

Perception of Care Survey for Medicaid Managed Care Members

Please tell us about your experience with your Medicaid Managed Care plan, the care you receive(d) from providers, and your perception of your own health and well-being.
We´re asking about the behavioral health services covered in your plan. Behavioral health means mental health and/or substance use disorder.

☒ We want to know about your experience with behavioral health services like counseling, rehabilitation, inpatient treatment, emergency/crisis services, or medicine for mental health or substance use conditions.

PART I: YOUR BEHAVIORAL HEALTH SERVICES

Did you receive behavioral health services in the last 12-months? ☐ Yes ☐ No
In the last 12-months, did you receive any treatment, counseling, or medicine for:
1. Emotional or mental illness? ☐ Yes ☐ No
2. Alcohol use? ☐ Yes ☐ No
3. Drug use? ☐ Yes ☐ No
4. Tobacco use? ☐ Yes ☐ No
Are you currently receiving behavioral health services? ☐ No ☐ Yes → If Yes, Go To Question 5

Please select the ONE main reason why you are no longer receiving behavioral health services.

☐ a. I no longer needed treatment because the problem that led to treatment was addressed.

☐ b. Treatment was not working as well as expected, so I stopped treatment.

☐ c. Treatment was no longer possible due to problems with transportation.

☐ d. Treatment was no longer possible due to problems paying for treatment.

☐ e. Treatment was no longer possible due to problems with finding time for treatment.

☐ f. Other reason(s) (please explain):

If you have not received behavioral health services in the past 12-months, skip to Part 3.

PART 2: ACCESS and QUALITY OF CARE
The next questions are about all the behavioral health services you got in the last 12-months that were covered by your Medicaid Managed Care plan.

☒ Please consider those services when answering the questions below.
☒ Please do NOT comment here about services that are NOT covered by your healthcare plan (e.g., self-help groups).
☒ If you have not received behavioral health services in the past 12-months, skip to Part 3.

In the last 12-months…	Never	Sometimes	Usually	Always	Not Applicable
5. How often did the people you went to for counseling or treatment explain things in a way you could understand?	○	○	○	○	○
6. How often did the people you went to for treatment treat you with respect and kindness?	○	○	○	○	○
7. How often did you get services at days/times that were convenient to you?	○	○	○	○	○
8. How often did you get services where you needed them?	○	○	○	○	○
9. How often did you get the services you needed as soon as you wanted?	○	○	○	○	○
10. How often did the people you went to for counseling or treatment spend enough time with you?	○	○	○	○	○
11. How often did you feel safe when you were with the people you went to for counseling or treatment?	○	○	○	○	○
12. How often did the people you went to for treatment listen carefully to you?	○	○	○	○	○
13. How often were you involved as much as you wanted in your treatment?	○	○	○	○	○
14. How often were the people you went to for treatment sensitive to your cultural background (race, religion, language, etc.)	○	○	○	○	○
15. How often did the people you went to for treatment tell you what medication side effects to watch for?	○	○	○	○	○
16. How often were the accommodations (for example wheelchair accessibility) you need to obtain services available?	○	○	○	○	○

In the last 12-months, how much were you helped by the counseling or treatment you got?

☐ Not at all ☐ Somewhat ☐ Very Much

The following questions are about services that you might receive through your healthcare plan. For each of the services listed below that you received in the past 12-months, please tell us how helpful the services were.

Services you might receive	If you received this service in the past 12-months, how helpful was the service?
Services you might receive	Very Helpful	Somewhat Helpful	Not at All Helpful	I did not receive this service
18. A Health Home care manager who coordinates your medical, behavioral health, and social service needs	○	○	○	○
19. Peer support services (support and help provided by people who have experienced mental illness and/or substance use disorder)	○	○	○	○
20. Assistance with returning to school or a training program	○	○	○	○
21. Assistance with finding or maintaining a job	○	○	○	○
22. Assistance with transportation other than medical transportation	○	○	○	○
23. Help with finding housing or better housing	○	○	○	○
24. Help in pursuing friendships and personal interests	○	○	○	○
25. Help in figuring out my finances, including getting any benefits I may be entitled to	○	○	○	○
26. Family support and training	○	○	○	○
27. Crisis respite services; i.e., residential care for 7 days or less, during a behavioral health crisis	○	○	○	○
28. Help with developing a crisis or relapse prevention plan	○	○	○	○

PART 3: HEALTH, WELLNESS, AND QUALITY OF LIFE
The next questions are about your health.

During the past 4 weeks, how much difficulty did you have doing your daily work, both at home and away from home, because of your physical health? (Please select one)

Could not do physical activities ☐

Have you used tobacco (e.g., cigarettes, e-cigarettes, pipes, cigars, smokeless or chewed tobacco) in the past 12-months?
☐ Yes ☐ No ☐ Prefer not to answer

	Yes	No	Not Applicable
31. Have you experienced any difficulties as a result of your tobacco use in the last 12-months (e.g., health, social, legal, or financial problems)?	○	○	○
32. Have you experienced any difficulties as a result of your alcohol use in the last 12-months (e.g., personal/family conflict, job instability, legal problems, and/or injuries)?	○	○	○
33. Have you experienced any difficulties as a result of your drug use in the last 12-months (e.g., personal/family conflict, job instability, legal problems, and/or injuries)?	○	○	○

The next group of questions ask about how satisfied you feel, using a zero to 10 scale. Zero means you feel no satisfaction at all. 10 means you feel completely satisfied. The middle of the scale is 5, which means you are neither happy nor sad.

How satisfied are you with……?	0	1	2	3	4	5	6	7	8	9	10
34. the things you have? Like the money you have and the things you own?	○	○	○	○	○	○	○	○	○	○	○
35. your health?	○	○	○	○	○	○	○	○	○	○	○
36. what you are achieving in life?	○	○	○	○	○	○	○	○	○	○	○
37. your personal relationships?	○	○	○	○	○	○	○	○	○	○	○
38. how safe you feel?	○	○	○	○	○	○	○	○	○	○	○
39. feeling part of your community?	○	○	○	○	○	○	○	○	○	○	○
40. how things will be later on in your life?	○	○	○	○	○	○	○	○	○	○	○

Please tell us if you Strongly Agree, Agree, are Neutral, Disagree, or Strongly Disagree with each statement below.

Strongly Agree	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
41. I am aware of community supports available to me.	○	○	○	○	○
42. My living situation feels like home to me.	○	○	○	○	○
43. I have access to reliable transportation.	○	○	○	○	○
44. I have trusted people I can turn to for help.	○	○	○	○	○
45. I have at least one close relationship.	○	○	○	○	○
46. I am involved in meaningful productive activities.	○	○	○	○	○

PART 4: BACKGROUND INFORMATION
The following information is collected to help ensure that services meet the needs of all individuals. Please do not share your name. Please check the boxes and fill in the blanks as applicable.

What is your age? _________
What sex were you assigned at birth, on your original birth certificate? ☐ Female ☐ Male ☐ Unknown
Current gender identity - How do you describe yourself? (check one) ☐ Female ☐ Male ☐ Transgender
☐ Do not identify as female, male, or transgender ☐ Prefer not to answer
How would you describe your sexual orientation? ☐ Heterosexual or Straight ☐ Homosexual, gay or lesbian
☐ Bisexual ☐ Other ☐ Not sure ☐ Prefer not to answer
In what language do you prefer to communicate with your health care providers?
☐ English ☐ Spanish ☐ Other (please specify)
In what language do you prefer to read things about your health care?
☐English ☐ Spanish ☐ Other (please specify)
Are you of Hispanic/Latino Origin? ☐ Yes, Hispanic or Latino ☐ No, not Hispanic or Latino
What is your race? (Select all that apply)
☐ White ☐ American Indian/Alaska Native ☐ Asian
☐ Black/African American ☐ Native Hawaiian/Other Pacific Islander ☐ Other
What is your highest level of education completed?
☐ Less than High School ☐ High School diploma or GED ☐ Business or technical school
☐ Some college, no degree ☐ College degree or higher
Are you currently enrolled in school? ☐ Yes ☐ No
Are you currently enrolled in a job training program? ☐ Yes ☐ No
Have you been employed in the past 12-months? ☐ Yes, I am currently employed ☐ Yes, but I am not currently employed
Please indicate whether the following things affect your ability to work or your decisions about working.

Select all that apply to you.

a. Lack of good jobs	○
b. Concern about losing benefits (e.g., Medicaid, etc.)	○
c. Lack of transportation	○
d. Physical health condition	○
e. Mental health condition	○
f. Arrest history	○
g. Lack of job training / education	○
h. Medication side effects	○
i. Workplace attitudes about mental illness and/or substance use problems	○
j. Retired and no longer looking for work	○

Have you been arrested in the past 12-months? ☐ Yes ☐ No
Have you experienced any difficulties with your housing over the past 12-months (e.g., 3 or more moves, having no permanent address, being homeless, living in a shelter)?
☐ Yes ☐ No

THANK YOU FOR COMPLETING THE SURVEY

|top of section| |top of page|

ATTACHMENT G

Mandatory Managed Long Term Care/Care Coordination Model (CCM)

Mandatory Population: Dual eligible, age 21 and over, receiving community-based long-term care services for over 120 days, excluding the following:

Nursing Home Transition and Diversion waiver participants;
Traumatic Brain Injury waiver participants;
Assisted Living Program participants; and
Dual eligible that do not require community-based long-term care services.

Voluntary Population: Dual eligible, age 18 through 20, in need of community-based long-term care services for over 120 days and assessed as nursing home eligible. Non-dual eligible age 18 and older assessed as nursing home eligible and in need of community-based long-term care services for over 120 days.

The following requires CMS approval to initiate and reflects the enrollment of the mandatory population only.

I. Phase I and II: New York City and the suburbs

July 1, 2012 - Any new dual eligible case new to service, fitting the mandatory definition in any New York City County will be identified for enrollment and referred to the Enrollment

Broker for action.

Enrollment Broker will provide with educational material, a list of plans/CCMs, and answer questions and provide assistance contacting a plan if requested.
Plan/CCM will conduct assessment to determine if eligible for community-based long-term care.
Plan/CCM transmits enrollment to Enrollment Broker.

In addition, the following identifies the enrollment plan for cases already receiving care. Enrollment will be phased in by service type by borough by zip code in batches. People will be given 60 days to choose a plan according to the following schedule.

July 1, 2012: Begin personal care cases in New York County

August 1, 2012: Continue personal care cases in New York County

September 2012: Continue personal care cases in New York County and begin personal care in Bronx County; and begin consumer directed personal assistance program cases in New York and Bronx counties

October 2012: Continue personal care and consumer directed personal assistance program cases in New York and Bronx counties and begin Kings County

November 2012: Continue personal care and consumer directed personal assistance program cases in New York, Bronx and Kings Counties

December 2012: Continue personal care and consumer directed personal assistance program cases in New York, Bronx and Kings Counties and begin Queens and Richmond counties

January 2013: Continue personal care and consumer directed personal assistance program citywide.

February 2013 (and until all people in service are enrolled): Personal care, consumer directed personal assistance program, citywide

March 2013: Personal care, consumer directed personal assistance program, adult day health care, home health care over 120 days citywide

March 2013: Personal care, consumer directed personal assistance program, adult day health care, home health care over 120 days in Nassau, Suffolk and Westchester counties

April 2013: Personal care, consumer directed personal assistance program, adult day health care, home health care over 120 days and long-term home health care program citywide

April 2013: Personal care, consumer directed personal assistance program, adult day health care, home health care over 120 days and long-term home health care program in Nassau, Suffolk and Westchester Counties

II. Phase III: Rockland and Orange Counties

June 2013: Dually eligible community-based long-term care service recipients in these additional counties as capacity

III. Phase IV: Albany, Erie, Onondaga and Monroe Counties

Fall 2013: Dually eligible community-based long-term care service recipients in these additional counties as capacity

IV. Phase V: Other Counties with capacity

Spring 2014: Dually eligible community-based long-term care service recipients in these additional counties as capacity

V. Phase VI:

Previously excluded dual eligible groups contingent upon development of appropriate program models:

Nursing Home Transition and Diversion waiver participants;
Traumatic Brain Injury waiver participants;
Assisted Living Program participants; and
Dual eligible that do not require community-based long-term care services.

|top of section| |top of page|

ATTACHMENT H

HARP Evaluation Plan

In 2015, the State amended its current 1115 waiver demonstration to enable qualified Managed Care Organizations (MCOs) throughout the State to comprehensively manage Behavioral Health (BH) benefits for eligible recipients. These benefits will be met in the following ways:

Mainstream Medicaid Managed Care (MMC) Plans: All adult recipients who are eligible for Medicaid Managed Care (excludes Medicare recipients and certain other populations), will receive the full medical and BH benefit through managed care. Plans began to cover expanded BH benefits in October 1, 2015. The expanded benefit includes services which the MMC plans previously managed for the non-SSI population (Psychiatric inpatient and Psychiatric clinic services), services that were covered only via the Medicaid Fee For Service (FFS) program (ACT, PROS, IPRT, SUD Inpatient and Clinic, Partial Hospitalization, CPEP, Opioid treatment, Outpatient chemical dependence rehabilitation), and new services (licensed behavioral health practitioner and behavioral health crisis intervention services).

Also effective October 1, 2015 consumers enrolled in a MMC whose BH benefit was covered under FFS Medicaid through SSI will begin receiving these benefits through the MMC plan.
Health and Recovery Plans (HARPs) and HIV Special Needs Plans (HIV SNPs): Adults enrolled in Medicaid and 21 years or older meeting the serious mental illness (SMI) and/or SUD targeting criteria and risk factors (see Appendix A) were passively enrolled into HARPs following the same timeline as the MMC behavioral health integration. These specialty lines of business operated by the qualified mainstream MCOs (MMMC) are also available statewide. Individuals meeting the HARP eligibility criteria who are already enrolled in an HIV SNP remained enrolled in their current plan but will receive the enhanced benefits of a HARP. In addition, HARPs and HIV SNPs will arrange for access to a benefit package of Home and Community Based Services (HCBS) for members who meet defined functional needs criteria. HARPs and HIV SNPs will work with Health Homes, or other State designated entities, to develop a person-centered care plan and provide care management for all services within the care plan, including HCBS.

The Behavioral Health demonstration was phased in with New York City (NYC) transitioning starting in October 2015 and rest of state (ROS) in July 2016 for adult enrollees (ages 21 to 64). Behavioral Health Home and Community Based Services were offered beginning in January 2016 in NYC and in October 2016 for ROS. The aims of the

New York BH demonstration are to improve the NYS Medicaid BH population´s health care quality, costs, and outcomes and to realize transformation of the BH system from an inpatient focused system to a recovery focused outpatient system. New York will conduct a multi-method, comprehensive statewide evaluation using an independent evaluator to document the impact of both the Mainstream Managed Care carve-in of behavioral health specialty services and the HARP implementation on health care service delivery, quality, health outcomes, and cost effectiveness of the HARP. In addition, program components that posed particular successes or challenges for implementation and outcomes for this population will also be examined. The broad goals of the New York HARP evaluation are to assess the impact of the demonstration on: 8) Improvement in health and behavioral health outcomes for adults in Mainstream Medicaid Managed Care whose behavioral health care was previously carved out in a fee for service payment arrangement; 9) Improvement in health, behavioral health and social functioning outcomes for HARP enrollees and 10) Improvement in recovery, social functioning and community integration for individuals in the HARP meeting HCBS eligibility criteria.

Toward these goals, the following evaluation questions will be addressed:

Goal 8: Improve health and behavioral health outcomes for adults in Mainstream Medicaid Managed Care (MMMC) with behavioral health conditions

To what extent are MMMC enrollees with behavioral health conditions accessing community-based behavioral specialty services1, including ACT, PROS, and first episode psychosis programs?
To what extent are MMMC enrollees with behavioral health conditions accessing primary care, preventive services, or integrated health/behavioral health care?

Goal 9: Improve health, behavioral health and social functioning outcomes for HARP enrollees

How has enrollment in HARP plans increased over the length of the demonstration?
What factors are associated with individuals choosing to opt out of HARP plans?
What are the demographic, social, functional and clinical characteristics of the HARP* population? Are they changing over time?
What are the educational and employment characteristics of the HARP* population? Are they changing over time?
To what extent are HARP* enrollees accessing primary care?
To what extent are HARP* enrollees accessing community-based behavioral specialty services?
To what extent are HARP enrollees accessing community-based health care or integrated health/behavioral health care?
To what extent is HARP quality of care improving, especially related to HEDIS®/QARR measures of health monitoring, prevention, and management of chronic health conditions?
To what extent are HARP* enrollees experiences with care and access to health and behavioral health services positive?
To what extent are HARP* enrollees satisfied with the wellness and recovery orientation, cultural sensitivity and their degree of social connectedness?
To what extent are HARPs cost effective? What are the PMPM cost of inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox and emergency room services for the HARP population? Are these costs decreasing over time?

Goal 10: Develop Home and Community Based services focused on recovery, social functioning, and community integration for individuals in HARP meeting eligibility criteria

Access to Care: How many HARP enrollees become eligible to receive Home and Community Based Services? How many HCBS eligible enrollees go on to receive Home and Community Based Services?
Access to care: What are the consequences of targeting availability of BH HCBS to a more narrowly defined population as compared to the HARP eligibility criteria in the State Plan?
Costs: What are the PMPM costs of BH HCBS for HARP* enrollees who receive services?

Evaluation Framework

New York will conduct an end of demonstration period evaluation of the HARP. The proposed evaluation is a multi-method and robust statewide plan designed to examine the impact of the behavioral health demonstration on health care service delivery, quality, health outcomes, and cost effectiveness of the HARP, as well as to determine program components that posed particular successes or challenges for implementation and outcomes. The evaluation plan would be finalized in an agreement with an independent evaluator.
Figure 1 shows a logic model depicting the BH demonstration in NYS which identifies the expected short-term activities, and intermediate and long-term program outcomes and provides a guiding framework for the evaluation. Although intermediate outcomes are expected, these will be formally evaluated at the end of the demonstration. The evaluation will use quantitative methods to assess program outcomes statewide and by region (NYC and ROS) and will also track outcomes over time. Some outcomes will also be compared across plan type (e.g., MCO Mainstream, MCO HARP and MCH HIV SNP levels). Survey methods will be used to assess consumer experience with care and consumer perception of care. Qualitative methods will be used to provide context for the quantitative and survey findings, as well as to obtain insights on HARP program functioning and effectiveness from administrative, provider, and patient perspectives. Evaluation methods and data sources (Appendix D) are detailed in sections to follow.

Figure 1: NYS Logic Model

Resources

DOH/OMH/OASAS funds

State Agency (NYSDOH/OMH/OASAS) Staff Resources

CMS Resources

Activities

MMC Carve-In of BH services

MCO HARP Product lines established

New BH services developed

Health Home linkage to HARP Established

HARP BH HCBS Eligibility Assessments and Plans of care developed

HARP BH HCBS Services Established

Health Care System Transformation

Increased number of providers delivering integrated health care services
Improved management of co- occurring physical and behavioral health care needs

Recovery Improvements

Social outcomes, functioning and recovery are assessed on a regular basis for HARP
Eligible members access BH HCBS services
Improved or consistent high level of satisfaction consumer experience with care

Improved Care & coordination

Improved care transitions (e.g. discharge follow-up)
Improved screening and treatment of BH in primary care settings
Reduction in inpatient and ER utilization
Increased access to Crisis services
Increased access to BH specialty and evidence-based services (ACT, PROS, FEP, MAT)

Reduced Health Care Costs & Recovery

Reduced Medicaid expenditures
Improved chronic physical health problem profile for behavioral health population
Costs shift from high cost IP and ER services to community-based services
Improved profiles on social determinants of health including employment, housing, education, social connectedness for HARP members
Improved quality of life for HARP members

Clinical Improvements

Behavioral Health
Chronic Health Conditions

Quantitative Method Approach

A variety of quantitative analytic methods may be utilized by the independent evaluator to assess the BH demonstration in NYS Medicaid. Pre and post quasi-experimental design methods may include interrupted time series and difference in difference. Causal model designs will be applied in pre-design phases to develop comparable groups where applicable and feasible. Longitudinal mixed effect regression methods will be used to examine individual outcomes over time for the HARP population. Multiple analysis of variance and chi-square comparisons will be applied to compare population and acuity characteristics of the HARP qualified populations who are enrolled in HARPS, HIV SNPs and MMMC plans by NYC and ROS on an annual basis. Survey methods and qualitative methods will be used to collect consumer input on the demonstration. Data available within the New York Department of Health and Office of Mental Health as specified below will be utilized for these analyses. The specific outcomes, measures, data sources and hypotheses related to the above indicated questions are detailed in subsequent sections. Note that depending on the goal and question addressed, "enrolled" may mean enrolled in a Mainstream Medicaid Managed Care Plan which includes the Behavioral Health Carve-In, enrolled in a HARP Plan, or enrolled in a HARP plan and eligible for BH HCBS. The following are potential methodologies that may be used in the independent evaluation, but the independent evaluation may also use additional methodologies as needed.

Quantitative Method I - Interrupted Time Series

Evaluation Approach I will involve a pre/post analysis of "enrolled" members using an interrupted time series design. An interrupted time series design1 is proposed to test hypotheses in assessing the BH demonstration and HARP´s statewide impact. This is a quasi-experimental design in which summary measures of the outcome variable are taken at equal time intervals over a period prior to program implementation (independent variable), followed by a series of measurements at the same intervals over a period following program implementation, as shown in the idealized illustration in Figure 2.

Figure 2: Pre- and Post-Intervention Comparison of Outcome Variable using Interrupted Time Series Design.

This design allows for the primary objective of evaluating trends/trajectory of outcome metrics such as cost before and after program implementation. The methods used in this design allows for a clear display of the monthly outcome variable trend overtime, changes in outcome variable trajectory as well as the dependencies or correlations between consecutive monthly measurements.

As with any program implementation analysis, the primary challenge is defining and acquiring groups between which to compare individuals within and without the implementation demonstration i.e. Non-BH or Non-HARP as comparative groups for BH Mainstream and HARP enrolled individuals. This design was chosen in consideration of the fact that non-BH /non-HARP control groups are unlikely to be available, limiting the ability to separate the effects of the BH demonstration from other statewide health care reform initiatives that are ongoing such as DSRIP, the New York Prevention Agenda, the State Health Innovation Plan (SHIP) with the support of the State Innovation Models (SIM) grant, the Affordable Care Act, and other concurrent market forces. Given the limitation resulting from the likely absence of a comparison group, this design is advantageous in that potential confounders (i.e., other health care reform initiatives) are minimized in that they would have to occur contemporaneously with the introduction of the BH demonstration including HARPs in order to exert a confounding effect, which is unlikely, but is recognized as possible nonetheless. This design also has the advantage of accounting for secular trends in the outcome variables to which other non-BH demonstration health reform initiatives would be expected to contribute.

To utilize the strength of this design, a segmented regression2 will be used to analyze the interrupted time series data. This analysis enables the evaluation of changes in the level and trend in the outcome variable from pre- to post-intervention and uses the estimates to test causal hypotheses about the intervention. In the post-intervention period, actual rates for the various metrics for each month will be compared to expected rates, while controlling for characteristics of the patients enrolled in the program, secular trend, serial autocorrelation, and seasonal fluctuation in the outcome variable. Analysis will be limited to patients with continuous Medicaid enrollment for the 12-months prior to the given intervention. Variables included in the regression adjustment will include factors such as prior inpatient, ED, and primary care utilization patterns (frequency), other resource use, diagnostic history, etc.

Quantitative Method II Difference in Difference Design (Dd)

There is the potential to strengthen the above design with respect to causal inference by taking advantage of the phased in approach of the BH demonstration in which NYC implemented 6 months prior to the rest of state implementation. The use of the HARP eligible control group in ROS compared to NYC may be proposed by independent evaluator applicants, however, there are several issues to consider. First, the time lag between NYC and ROS is only 6 months. It is likely that in that first 6 months the system will still be going through many changes in order to be able to provide the new benefit package and to develop the new HARP product lines. It is unlikely that the 6-month time period will be sufficient to be able to identify changes between the two groups. In addition, the use of eligible control group in ROS compared to NYC may be a problem since changes in the health of patients in the ROS might be systematically different from NYC, due to, say, aid (socioeconomic), transportation and housing differences rather than the BH program implementation.

A more robust DD analysis will be performed (depending on data availability) by using eligible individuals who opt out of the HARP (HARP- Opt-Out) as control for those who opt into HARP. This approach or strategy accounts for any secular trend/changes in the outcome metrics (it eliminates fixed differences not related to program implementation), with remaining significant differences attributable to the impact of program implementation3 The study groups will be prepared by match-pairing individuals using propensity scores derived from logistic regression based on selected demographic, clinical and social indicators, and health care utilization characteristics (see Quantitative Method V).The outcome metrics, health care costs per member per month/year (PMPM/Y) and service use rates, such as hospital admit rates measured over two consecutive periods of two years before and two years after program implementation will be calculated (total duration of four years). Changes in outcome metrics from measurement period-1 (2013 - 2015), (2014 - 2016), to measurement period-2 (2016 - 2017), (2017 - 2018), will be compared for NYC and ROS respectively. Also, changes in individuals who are HCBS eligible and opt for HCBS services will be compared to individuals who are HCBS eligible and do not opt for HCBS services using similar match-pairing and DD techniques.

Figure 3: Idealized representation of DD Method

Quantitative Method III Longitudinal Mixed Effect Regression Method

A Generalized Linear Mixed Model (GLMM) will be implemented to address the potential heterogeneity in the program/BH implementation effect and estimate an average program effect while controlling/adjusting for important covariates4,5 The GLMM framework uses a model-based approach to estimate HARP enrolled individual program effects allowing for program/BH implementation random effects.

This framework has the advantage of separating the effects of time from that of the BH implementation, accommodating the heterogeneity in the BH implementation effect, and accounting for serial correlations within individuals (resulting from repeated measurements). As with implementation longitudinal data, the outcome metrics such as employment, enrollment in formal education, social relationships, social strengths, and behavioral health service utilization may vary considerably over time due to a strong temporal trend before and/or after program implementation. Risk factors including homelessness, criminal justice involvement (arrest history, incarceration history), alcohol use, drug use, chronic physical health conditions, and traumatic life events would likely vary considerably over time. The GLMM framework helps determine the amount of variability that may be due to temporal trend and the amount due to the new program implementation. The GLMM was chosen because it accounts for the intrinsic differences among individuals, the variability in program impact on individuals, and the potentially induced correlation by collecting data on the same individuals over time.

Quantitative Method IV Descriptive Statistics: Multiple Analysis of Variance and Chi- square Analysis

Comparisons will be made to examine characteristics of HARP enrollees in NYC and in the ROS in each annual period (10/2015-2020) using descriptive statistical methods for categorical, ordinal or continuous data. Chi-square analysis comparing NYC to ROS as independent samples will be performed for categorical outcome variables. McNemar´s chi-square test will be performed to compare binary outcomes between correlated groups for each region before and after implementation. Similar analysis will be considered for comparing categorical outcome variables for each region year to year.

For continuous outcome variables, ANOVA will be used to test the difference in means score between independent samples from NYC and ROS. The use of repeated measures ANOVA for yearly changes within each region may be proposed by an independent evaluator, however, an important assumption of the repeated measure ANOVA known as sphericity may be violated. Correlations between data in year 1 and year 2 may not be the same as year 2 to year 3 and likewise between year 1 and year 3. This condition of equal correlations from one-year to the other can be a problem given the continuous assignment, and enrollment into HARPs as well as the complexities surrounding the BH implementation. Paired t-test will be used to compare pairs of years and for multiple pair comparisons, say, for measurement of 3 years (comparing year 3 with year 2 and year 3 with year 1) a Bonferroni adjustment will be applied to the threshold p-value.

Quantitative Method V - Propensity Score Matching

Quantitative method V will involve using what is termed propensity or prognostic score matching to control for potential confounding by identifying a comparison group for specific study questions. This method may be used combined with Quantitative Method II to examine the impact of the HARP benefit on health outcomes and to examine the impact of HCBS services on recovery outcomes. A comparison group for the HARP benefit could be members qualified for HARP plans who opted out of the HARP and are enrolled in MMMC. A comparison group to examine HCBS services could be HARP members eligible for HCBS services but receiving only traditional services 6. This method would be applied in the design phase with application for a variety of causal models which may be selected. Using prior utilization and diagnostic information, this approach attempts to identify recipients with similar characteristics during pre and post demonstration period. The method estimates each individual´s conditional probability of being enrolled in HARPs (or HCBS for the assessment cohort). The propensity scores will be estimated using a logistic regression, with the outcome being opting to enroll into HARPs (coded 1 @ HARPS, 0 @ HARP- Opt-Out), opting to receive HCBS Services (coded 1 @ HCBS, 0 @ No HCBS), and predictors being derived from an array of demographic, clinical and social indicator constructs. The potential confounders will be selected a priority based on subject matter knowledge and in consultation with subject matter experts.

A greedy matching algorithm with an appropriate matching ratio of HARPs to HARP-Opt Out (1: n) will be used to create a matched analytic cohort based on the estimated propensity score and other appropriate service use indicator such as the number of psychiatric hospitalization days prior to program implementation7. Balance in covariate distribution between HARPs and HARP-Opt Out (or HCBS and No HCBS) in the matched analytic cohort will be assessed with weighted standardized difference8. The matched cohorts will be used for the quantitative methods indicated above where possible.

Quantitative Method VI - Exponential Smoothing Methods

An exponential smoothing method17will be used to examine the monthly, quarterly and yearly trends of service utilization or program enrollments, and cost of service use where appropriate. In this method, the trend/trajectory of a series of summary measurements of the outcome variable (rate of service use, program enrollments) taken at equal time intervals over a defined period are analyzed using smoothing techniques. Service use or program enrollment projections based on exponential smoothing techniques are weighted averages of past service use or enrollments, with the weights decaying exponentially as the outcome/observations get older. Thus, the more recent the outcome the larger the assigned weight. This allows for reliable examination of monthly, quarterly and yearly trends, as well as future projections of program enrollment or service use. This method allows for a clear display of the monthly service use and cost trend overtime, changes in service use and cost trajectory as well as the intrinsic nature (i.e. the dependence or correlations between consecutive months) of one monthly outcome to the other.

Consumer Survey Approach

The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) survey is administered on a bi-annual basis with Adults enrolled in all Medicaid Managed Care product lines according to the current quality strategy approved by CMS in the 1115 Waiver. Adult members with behavioral health needs are included in the CAHPS® survey, however, oversampling is not implemented to ensure that there is representation of members with behavioral health needs from mainstream product lines. The HARP MMC product lines will be included in the CAHPS® survey in 2018.

In addition, the HARP Perception of Care (HARP PCS) survey was developed by the State to evaluate HARP member perception of and experience with care. Members enrolled in HARPs and BH HCBS eligible members enrolled in HIV SNPs will be surveyed annually to measure experience with care, perception of care and perception of quality of life. This survey was derived from validated instruments intended to assess consumer perception of the performance of health plans and behavioral health services. Specifically, questions were drawn from the Experience of Care and Health Outcomes (ECHO) Survey, the Mental Health Statistics Improvement Program (MHSIP)/ OMH Consumer Assessment of Care Survey (CACS) Survey, the Personal Wellbeing Index adult version (PWI-A), and the Maryland Outcomes Measurement System. NYS OMH also formulated questions for pertinent topic areas where none could be found in existing instruments. The majority of questions address domains of member experience such as accessibility of services, quality of services, and appropriateness of care, wellness, quality of life, and social connectedness. Additionally, a set of socio-demographic questions are included which will allow examination of disparities. The current draft of the survey consists of 78 questions found in Appendix B.

The HARP PCS will be piloted by NYS OMH in the fall of 2016 with a small number of NYS OMH and OASAS programs. Final modifications to the HARP PCS will be completed based on pilot findings in the first quarter of 2017. The first HARP PCS will be implemented in Q4 of 2017.

The HARP PCS pilot will be implemented in 3-5 NYS OMH or OASAS funded programs in Q4 of 2016. Additional survey questions will be included to gather feedback from pilot participants about the length of the survey, clarity of the questions, and relevance of the questions. Surveys will be implemented by the State with the assistance of program administrators at selected programs and administered by non-direct care program staff at the pilot program sites. Participants will complete the surveys on site, with the option of mailing the survey back to OMH individually or in a sealed individual envelope with other respondents. Completed surveys will be processed and summarized by NYS OMH. NYS OMH will also collect survey response rate and administration feedback from program sites. The pilot findings will be used to finalize the HARP PCS instrument for full implementation in 2017.

Qualitative Method

Qualitative methods may include key informant interviews, focus groups, and surveys. Issues to be investigated qualitatively include notable program outcomes and challenges, effectiveness of governance structure and provider linkages, contractual and financial arrangements, changes in the delivery of patient care, the effect of other ongoing health care initiatives (e.g., DSRIP, New York Prevention Agenda, Affordable Care Act) on the BH demonstration, HARP and HCBS services implementation and operation, and patient experience and satisfaction with services. The Independent Evaluator will develop key informant and focus group interviews to address the questions under each objective. Development will include the determination of interview questions with appropriate review and pre-testing to ensure that questions are comprehensive, understandable, and reliable.

The Independent Evaluator will determine a strategy for identifying a range of stakeholders to target for in-depth interviews and focus groups. At a minimum, stake holders would be expected to include HARP enrollees; HARP Managed Care administrators; and HCBS service providers and would reflect variation in region (NYC vs ROS) and other contextual factors (e.g., urban vs rural). Managed Care Plans, providers and state agency offices would be used to facilitate contact and recruitment. Interviews and focus groups will be semi-structured such that questions to be asked will address consistent topics for a given category of respondent (e.g., administrator, provider, enrollee), while at the same time allowing for follow-up questions to probe for more in-depth responses. Modifications in the interview questions will be made as necessary based on responses obtained on early interviews.

Analysis will follow a framework described by Bradley, Curry, & Devers9 that has been effectively used in health services research. Preliminary review of the data using a grounded theory approach (i.e. without predetermined categories) will be performed to identify emergent themes. A coding structure will then be established through an iterative process that labels concepts, relationships between concepts, and evaluative participant perspectives (i.e., statements that are positive, negative, or indifferent to their experiences or observations). The coding structure will also capture respondent characteristics (e.g., age, sex, position or role in organization) and setting (e.g., community-based provider, HARP plan, MMC mainstream plan, NYS region). Responses will then be re-reviewed independently by at least two evaluation staff members, applying the finalized coding structure. Coding discrepancies between reviewers will be resolved through discussion to achieve consensus for the final coding of the data. Coded data will be analyzed and interpreted to identify major concept domains and themes.

Figure 2. Evaluation Tool for the New York State Behavioral Health Partnership Plan Demonstration Evaluation: October 1, 2015 through March 31, 2021

Goal 8: Improve Health and behavioral health outcomes for adults in Mainstream Medicaid Managed Care whose behavioral health care was previously carved out in a fee for service payment arrangement

Evaluation Questions

To what extent are MMC enrollees accessing community-based behavioral specialty services (see Appendix C for a list of specialty services), for example ACT, PROS, and first episode psychosis (FEP) programs?
To what extent are MMC enrollees accessing community-based health care or integrated health/behavioral health care?

The quantitative methods to be used to investigate these two areas are discussed below. The outcomes, measures, data sources and hypotheses to be tested are shown in the Evaluation tool for Goal 8 (Table A) below.

Questions 1 and 2 will utilize a pre-post design with interrupted time series analysis (Quantitative Method I). The proportion of MMC enrollees using any and specific BH specialty services and average units used pre and post (2010-9/2015: 10/2015 to 2020) will be examined. A similar design will be used to examine the proportion of MMC enrollees receiving integrated care in primary care settings and average units used pre and post (2010-9/2015: 10/2015 to 2020). In addition, the percent of MMC enrollees with BH needs with no claim´s history for primary and preventive services in each annual period pre: post (2010-9/2015: 10/2015 to 2020) will be examined. Data from Medicaid claims will be utilized to examine all service patterns.

We expect that the use of BH specialty and integrated care services will be utilized by more individuals and that more units of service will be provided in the post intervention period compared to the pre period. We expect that the proportion of MMC enrollees with BH needs with no claim´s history for primary and preventive services in each annual period pre compared to the post period will decline.

The State recognizes complexity with respect to monitoring the utilization and uptake of treatment and services related to FEP and integrated primary care. Each topic is detailed below with respect to how evaluation questions related to services utilization may be approached by the State and Independent Evaluator.

FEP Services

The State provides evidence-based treatment for FEP using the OnTrackNY (OTNY) Coordinated Specialty Care (CSC) program. This program provides treatment to individuals between the ages of 16 and 30 who have experienced non-affective psychosis for less than two years at the time of admission. Coordinated Specialty Care (CSC) is a multi-disciplinary team approach for delivering evidence-based services to young people experiencing first episode psychosis (FEP) with the goal of improving outcomes by providing early intervention services10. OTNY evolved from the Recovery After an Initial Schizophrenia Episode (RAISE) Connection program, which was developed in partnership with the NYS OMH as part of the National Institute of Mental Health (NIMH)-funded RAISE Implementation and Evaluation Study (RAISE-IES). OTNY is considered to be an evidence-based program model. The program currently has the capacity to serve 800 individuals per year across the state. Based on the incidence of schizophrenia (10 per 100,000) we expect to have 2000 new cases per year. Based on the current sample of patients served in OTNY we estimate that approximately 50% would be enrolled in Medicaid. It is notable that OTNY is a new program and will have limited enrollment prior to 2015. In addition, OTNY will be expanding across the state through the demonstration period.

It is notable that the current system for identification of FEP is driven primarily by provider referrals with MMMC plans assisting where possible. The State is working with MMMC plans on to develop a referral and tracking methodology for these enrollees with priority given to OTNY program enrollment. In addition, the State is still developing a system in which FEP individuals can become eligible for HARP enrollment in 2017. The State anticipates that over the course of the Demonstration period that identification, tracking and monitoring related to FEP will become more robust.

At the same time, the State is working to develop a Medicaid claims-based algorithm which will be tested in collaboration with MMMC plans to develop capacity to identify incident cases of FEP using claims and potentially EHR data. This methodology is emergent at this time. It is anticipated that this method could be used to capture a measure of duration of untreated psychosis to validate the accuracy of first episode occurrence and to understand if providers and plans are improving timely access to treatment.

The State anticipates that over the course of the Demonstration period that the identification of incident cases of FEP will become more robust. Using this algorithm, the State plans to identify Medicaid recipients meeting potential FEP criteria to examine the rate of identification of FEP in the MMC population over the 2015-2020 period and the duration of untreated psychosis. The Independent Evaluator will be able to take advantage of the methods and technologies developed over the demonstration for the external evaluation at the end of the Demonstration.

The OTNY data system provides a unique opportunity for the State and Independent Evaluator to identify the MMMC or HARP enrollees who are receiving OTNY services. Outcomes including rates of engagement, hospitalization and school and work participation are monitored via the OTNY data system. Outcomes related to Medicaid service utilization for emergency, inpatient, outpatient and HCBS services can also be monitored. FEP utilization will be captured from the OTNY data system. The proportion of MMC enrollees receiving an evidence-based treatment for first episode psychosis will be tracked using the OTNY data system.

We expect to see identification of FEP and utilization of the FEP programs to increase over the course of the demonstration. FEP is not currently a billable Medicaid service in NYS although NYS MMC plans are required to offer FEP as a plan benefit. It is anticipated that during the Demonstration period FEP will become a billable Medicaid service and utilization will be monitored using Medicaid claims in the future.

Integrated Behavioral Health Care

Provision of integrated behavioral health care programs is an integral part of the DSRIP Medicaid system re-design. Currently the State has 3 options for Behavioral Health Integration under DSRIP13 (Goal 3ai). In the NYS implementation of DSRIP every PPS chose model 1 and some combination of the other two models, but there is not a minimum number of program sites operating selected models. As a result, the level of penetration of the model within the PPS is not readily determined.

Model 1: Bringing BH services in to a PCMH or APC primary care practice. Performance provider systems work in partnership with behavioral health providers to offer behavioral health services on site. Providers implement a preventative screening (PHQ-9, SBIRT) to identify unmet behavioral health needs. If/when screenings are positive, provider refers patient to behavioral health provider for further evaluation and/or treatment.
Model 2: Bringing a Primary Care Provider to a BH clinic. Performance provider systems identify behavioral health service sites interested in providing primary care services on location. Provider then works with behavioral health provider to identify community needs, develop a structure for integration requirements and develop evidence-based standards of care.
Model 3: Implementing the IMPACT model (Collaborative Care) in a primary care practice. The IMPACT Model employs a collaborative team of professionals with complementary skills to fully integrate behavioral health treatment into primary care. This team includes a depression care manager, a primary care provider (PCP) and a consulting psychiatrist. The patient´s PCP works with the care manager to develop and implement a stepped care treatment plan and consults with the psychiatrist to change course of treatment for patients who do not improve after 10-12 weeks. In over 80 randomized controlled studies, IMPACT has shown to improve PHQ-9 scores by >50% in 12-months.

The DSRIP metrics for integrated care are process metrics related to implementation of the elements of the chosen model. These process metrics rely on EHR and other reporting requirements specific to the State oversight of DSRIP. Population level outcomes would also be expected from having these services available to the Medicaid population. DSRIP outcomes such as reduction of ER utilization and hospital readmissions on a PPS level would be expected to improve over the course of the demonstration. At this time, the State does not have the ability to identify the receipt of integrated behavioral health care using Medicaid claims data. The State will examine how the DSRIP findings can be used by the Independent Evaluator to determine the penetration and impact of integration models on the MMMC population.

Table A: Evaluation tool for Goal 8
Q #	Outcome	Measure	Data Source	Related Hypotheses	Possible Methodologies
1	Improve access to behavioral health care specialty services (See Appendix C for specialty services)	Proportion of enrollees using any and specific BH specialty services and average units used pre and post (2010- 9/2015: 10/2015 to 2020)	Medicaid Claims; OnTrack NY Client records	Utilization of BH specialty services will increase in the MMC population	Pre-post design with interrupted time series analysis
1	Improve identification of and access to care for First Episode Psychosis patients	Percent of MMC population identified as having first episode psychosis in each annual period from baseline (2015 to 2020); Proportion of MMC population utilizing evidence-based care for First Episode Psychosis in each annual period (2015 to 2020).	Medicaid Claims; OnTrack NY Client records	Identification of First episode psychosis will increase; utilization of evidence-based care for first episode psychosis will increase; Duration of untreated psychosis will decrease	Pre-post design with interrupted time series analysis
2	Improve access to primary and preventive services	Percent of MMC BH population enrolled for entire prior 12-months with no claims history for primary and preventive services in each annual period pre: post (2010-9/2015: 10/2015 to 2020)	Medicaid Claims	Percent of MMC BH members without primary care utilization will decline	Pre-post design with interrupted time series analysis

Goal 9: Improve health, behavioral health and social functioning outcomes for adults in the HARP

The quantitative methods to be used to investigate the eleven questions related to this goal are discussed below. The outcomes, measures, data sources and hypotheses to be tested are shown in the Evaluation tool for Goal 9 (Table C) below. HARP refers to HARP enrollees in HARP or HIV SNP plans.

HARP enrollees

How has enrollment in HARP plans increased over the length of the demonstration?
What factors are associated with individuals choosing to opt out of HARP plans?
What are the demographic, social, functional and clinical characteristics of the HARP population? Are they changing over time?
What are the educational and employment characteristics of the HARP population? Are they changing over time?
To what extent are HARP enrollees accessing primary care?
To what extent are HARP enrollees accessing community-based behavioral specialty services?
To what extent are HARP enrollees accessing community-based health care or integrated health/behavioral health care?
To what extent is HARP quality of care improving, especially related to HEDIS®/QARR measures of health monitoring, prevention, and management of chronic health conditions?
To what extent are HARP enrollee experiences with care and access to health and behavioral health services positive?
To what extent are HARP enrollees satisfied with the cultural sensitivity of BH providers and their wellness, recovery, and degree of social connectedness?
To what extent are HARPs cost effective? What are the PMPM cost of inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox and emergency room services for the HARP population? Are these costs decreasing over time?

Evaluation Questions

Q1. How has enrollment in HARP plans increased over the length of the demonstration?

HARP plan enrollment will be assessed within the context of overall program enrollment. To assess the impact of HARP roll-out, the evaluation will examine how many HARP-eligible members are enrolled in each annual period in each MMC, HARP or HIV SNP. It is important to note that for this measure, there is no pre- implementation comparison or other group comparison possible. Quantitative Method IV will be used to monitor year to year comparisons in NYC and in the ROS in each annual period for the period 10/2015 to 12/2020 and reported at the end of the demonstration period. It is expected that enrollment in HARP plans will increase over the length of the demonstration as new members are identified and original members opt to remain in the HARP or HIV SNP plans rather than joining a MMC mainstream plan. We expect that the majority of HARP eligibles will enroll in HARP or HIV SNP plans rather than in MMMC plans.

Medicaid enrollment data will be used for this analysis. Medicaid enrollment data are available lagged by one month. It should be noted that the first 9 months of the implementation include only NYC plans with the rest of NYS beginning 7/2016.

Q2. What factors are associated with individuals electing to or declining to enroll in HARP plans?

The demographic (age, gender, race, residential region), diagnostic (Dx) (MH Dx, SUD Dx, Dual Dx) and acute BH service utilization (BH inpatient (IP), SUD IP detox, SUD IP rehabilitation) characteristics of HARP-eligible members who are enrolled in each annual period in MMC, HARP or HIV SNPs will be compared (Quantitative Method IV). Demographic characteristics will be categorical, diagnostic characteristics dichotomous (y/n) and BH service utilization will be characterized as number of episodes in a year or number of days utilized for each service type per year. Comparisons will be made using chi-square analysis and Anova as appropriate according to data type (Quantitative Method IV).

We hypothesize that HARP eligible members who opt out may be younger and less behaviorally acute than those who remain enrolled in HARP/HIV SNP.

Medicaid enrollment and claims data will be used for this analysis. Medicaid enrollment data are lagged by one month. Medicaid claims data is lagged by 6- months. It should be noted that the first 9 months of the implementation include only NYC plans with the rest of NYS beginning 7/2016.

In addition, the qualitative reasons members have for opting back into MMMC is being collected by the State to assess reasons for opting out of the HARP. The data collected include a categorical list of reasons for declining and allow for open ended response by enrollees. The data are summarized on a weekly basis for NYC and ROS. The reasons for opting out will be monitored over time and cumulated by year 10/2015 to 12/2020. It is important to note that these data are not available on an individual member basis. Data are collected by the enrollment broker in the NY Medicaid Choice Enrollment data system; however, no recipient identifier is retained with the data.

Q3. What are the demographic, social, functional and clinical characteristics of the HARP population? Are they changing over time?

Q4. What are the educational and employment characteristics of the HARP population? Are they changing over time?

Questions 3 and 4 examine the detailed socio-demographic data which will be available for HARP enrollees in HARP and HIV-SNP plans via the BH HCBS Eligibility Brief Assessment and BH HCBS Full Assessment. These assessments are derived from the interRAI Community Mental Health Assessment®14. The BH HCBS Eligibility Brief Assessment is required annually for all HARP enrollees and HARP eligible HIV SNP enrollees. For screened individuals who meet BH HCBS eligibility criteria a BH HCBS Full Assessment is completed and repeated annually. As such, this detailed information will be available for HARP/HIV SNP members but are not available for HARP eligible members who opt out and return to MMC mainstream plans.

Two analytic approaches are recommended to be applied to these data to examine the above questions: population level year by year comparisons (Quantitative Method IV) and individual level analysis of change over time (Quantitative Method III). First, population characteristics will be examined in each annual period at the end of the demonstration (10/2015-2020) for HARP enrollees in HARP and HIV-SNP plans in NYC and ROS. Characteristics examined include socio-demographic, clinical, and recovery related measures including education, employment, social network, risk factors, home environment, social relationships, criminal justice involvement, top health diagnoses, behavioral diagnoses, behavioral health symptoms, substance related practices and behavioral health services accessed (Please refer to Appendix E for the BH HCBS Eligibility Brief Assessment and Appendix F for the BH HCBS Full Assessment used in the demonstration). These indicators will be coded as categorical, ordinal or continuous variables as appropriate for analysis. Comparisons using Quantitative Method IV include descriptive statistical methods (e.g., ANOVA, Chi- square) for categorical, ordinal or continuous data. It is expected that the distribution of the measured risk factors and protective factors for this population will shift toward fewer risk factors and greater protective factors. Regional (NYC vs ROS) differences in improvements may be observed. Specifically, higher rates of educational and employment attainment will be observed among HARP enrollees over time as the program matures. Paired t-test will be used to compare pairs of years and for multiple pair comparisons, say, for measurement of 3 years (comparing year 3 with year 2 and year 3 with year 1) a Bonferroni adjustment will be applied to the threshold p-value.

Individual level change may be examined using longitudinal data analytic methods (Quantitative Method III). Individuals will have repeated BH HCBS Eligibility Brief Assessments and BH HCBS Full Assessments completed. Longitudinal change in risk and protective factors identified above will be examined to determine change trajectories using multivariable mixed effects regression methods (Quantitative Method III). Fixed effects will be identified including age, gender and race/ethnicity and time. Random effects will include risk and protective factor level at each annual time point.

It is important to note that for these questions, there is no pre-implementation comparison group available. The risk and protective, employment and education data collected via the BH HCBS Eligibility Brief Assessment and BH HCBS Full Assessment as part of this demonstration are not available prior to the demonstration so no pre-post comparison can be made. For some analyses, assessment data may be matched to enrollment and services data in the Medicaid data mart. Each assessment includes Medicaid Id so matching between the assessment data and Medicaid data will not be a barrier. In addition, since the HARP demonstration applies to ages 21-64, we do not anticipate the age structure of the eligible population to change. However, this will be examined to determine if changes in the population age structure may be impacting the analysis.

Q5. To what extent are HARP enrollees accessing primary care?

Pre-post approaches (Quantitative Method II) could be used to assess access to primary care among HARP eligible pre-implementation compared to HARP enrolled in HARP and HIV SNP plans post-implementation. The unit of analysis will be rate of primary or preventive care visits measured as members receiving one or more primary or preventive care visits in a year (e.g., the use of evaluation and management CPT codes or well visit codes by primary care physicians) from Medicaid claims data. We anticipate that HARP enrollees will access primary and preventive care at greater rates in comparison to HARP eligible populations prior to the demonstration. Changes in use of primary care and preventive care from measurement period-1 (2013 - 2015), (2014 -2016) to measurement period-2 (2016 - 2017), (2017 - 2018), and afterwards (2019-2020) will be compared for NYC and ROS respectively. Comparable members during the pre and post periods may be selected using the HARP population algorithm and propensity score matching techniques (Quantitative Method V). Medicaid claims will be utilized for these analyses.

Q6. To what extent are HARP enrollees accessing community-based behavioral specialty services?

Pre-post approaches (Quantitative Methods I or/and II) could be used to assess access to community-based behavioral specialty services (see Appendix C for list) among HARP eligible pre-implementation compared to HARP enrolled in HARP and HIV SNP plans post. The unit of analysis will be rates at which members use community-based behavioral health specialty services in a month/quarter and within the year. This will be measured as the proportion of members receiving one or more community-based behavioral health specialty service in each service category in a month/quarter and within the year. We anticipate that HARP enrollees will access community-based behavioral health specialty services at greater rates in comparison to HARP eligible populations prior to the demonstration. Changes in use of behavioral health specialty services from measurement period-1 (2013 - 2015), (2014 -2016) to measurement period-2 (2016 - 2017), (2017 - 2018), (2019-2020) will be compared for NYC and ROS respectively. Analysis evaluating the monthly/quarterly utilization trends of community-based behavioral health specialty services using Quantitative Method I may be limited to only HARP enrollees receiving HCBS services. Comparable members during the pre and post periods may be selected using the HARP population algorithm and propensity score matching techniques (Quantitative Method V). Analysis evaluating the changes in yearly utilization of community-based behavioral health specialty services in the comparable matched cohort will be conducted using Quantitative Method II. Medicaid claims will be utilized for these analyses.

Q7. To what extent are HARP enrollees accessing Health Homes for care coordination?

Pre-post approaches (Quantitative Method I and II) could be used to assess access to Health Home care coordination among HARP eligible pre-implementation compared to HARP enrolled in HARP and HIV SNP plans post. The measure to be used will be the proportion of HARP enrollees engaged in health homes pre and post measurement period-1 (2013 - 2015), (2014 -2016) to measurement period-2 (2016 - 2017), (2017 - 2018) for NYC and ROS respectively, and subsequently (2019-2020). . We expect that access to care coordination services will increase in terms of health home engagement for HARP members.

Analysis evaluating the monthly/quarterly enrollments in health homes (utilization over time) will be conducted using Quantitative Method I. Analysis evaluating the changes in yearly utilization of health homes in the comparable matched cohort will be conducted using Quantitative Method II. Comparable members during the pre and post periods maybe selected using the HARP population algorithm and propensity score matching techniques (Quantitative Method V).
Medicaid claims will be utilized for these analyses.

Q8. To what extent is HARP quality of care improving? (HEDIS®/QARR measures including health monitoring, prevention, chronic health conditions)

Pre-post approaches (Quantitative Method II) will be used to assess improvements in quality of care related to health monitoring, prevention, chronic health and behavioral health among HARP eligible pre-implementation compared to HARP enrolled in HARP and HIV SNP plans. The measure specifications follow HEDIS® specifications for each measurement year.15 Note that we expect HEDIS® quality of care metrics and value sets to change over the course of the demonstration period. The Independent evaluator will be expected to apply definitions as deemed appropriate. We expect that care quality will improve in the areas of behavioral health, cardiovascular disease, asthma and diabetes (Table B below). Changes in these measures from measurement period-1 (2013 - 2015), (2014 -2016) to measurement period-2 (2016 - 2017), (2017 - 2018), and afterwards (2019-2020) will be compared for NYC and ROS respectively. Comparable members during the pre and post periods will be selected using the HARP population algorithm and propensity score matching techniques (Quantitative Method V). Metrics for these analyses are plan reported as part of the Medicaid quality oversight. These analyses may supplement plan submitted data with Medicaid claims data to enhance rates or may recalculate administratively derived HEDIS® metrics using Medicaid claims so that appropriate pre and post periods can be selected and to allow for identification of appropriate comparison groups.

Table B. Clinical Improvement Outcome Measures
Outcome	HEDIS® Measure Name	Source
Behavioral Health	Antidepressant Medication Management	Claims
	Diabetes Monitoring for People with Diabetes and Schizophrenia	Claims
	Diabetes Screening for People with Schizophrenia/BPD Using Antipsychotic	Claims
	Cardiovascular Monitoring for People with CVD and Schizophrenia	Claims
	Follow-up care for Children Prescribed ADHD Medication	Claims
	Follow-up after Hospitalization for Mental Illness	Claims
	Adherence to Antipsychotic Medications for People with Schizophrenia	Claims
	Initiation and Engagement of Alcohol and Other Drug Dependence Treatment (IET)	Claims
	Follow-up After Emergency Department Visit for Mental Illness (FUM)	Claims
	Follow-up After Emergency Department Visit for Alcohol and Other Drug Dependence (FUA)	Claims
Diabetes	Comprehensive Diabetes Care	Claims
Cardiovascular	Controlling high blood pressure (CBP)	Plan submitted
Asthma	Medication Management for People with Asthma	Claims

Q9. To what extent are HARP enrollees´ experiences with care and access to health and behavioral health services positive?

Q10. To what extent are HARP enrollees satisfied with the cultural sensitivity of BH providers, and their wellness, recovery, and degree of social connectedness?

Question 9 will utilize the Health Plan version of the CAHPS® survey to examine HARP enrollee experience with care and perception of access to health and behavioral health services. We expect that HARP enrollee experience with care and perceived access to health and behavioral health services will improve over time. Quantitative method IV will be used to examine year to year comparisons of the survey responses by NYC and ROS.

The CAHPS® survey will be administered to adults via the EQRO contract in 2017 and 2019. The survey administration will include a random sample of individuals in HARPs. The survey is administered by both mail and telephone and assesses patients´ experiences with health care providers and health plan staff. This includes information on patient experience with access to care, experiences with health care providers and health plan support. Questions specific to behavioral health include: need for mental health or SUD treatment, access to mental health or SUD treatment, satisfaction with mental health or SUD treatment, and self-rating of overall mental health.
Given confidentiality agreements, only de-identified CAHPS® data will be available for use. This limits the ability to make pre-post comparisons. In addition, the survey will not be oversampled in terms of mainstream populations with mental health issues or HARP eligible enrollees in HIV SNP plans. This limitation also applies to current CAHPS® results. Since the BH population is not oversampled it is not possible to examine what the existing reporting patterns are for this sub-population.

Question 10 will utilize the HARP Perception of Care Survey (PCS) (See Appendix B). We expect that HARP enrollee satisfaction with the cultural sensitivity of their behavioral health providers will increase over the length of the demonstration. We also expect that HARP enrollee satisfaction with their wellness, recovery, and degree of social connectedness will improve over the time of the demonstration. Quantitative method IV will be used to examine year to year comparisons of the survey responses by NYC and ROS.

The PCS was developed by NYS with advocate, program and psychiatric research input. The PCS is derived from a number of standardized instruments including: the Experience of Care and Health Outcomes (ECHO) Survey, the Mental Health Statistics Improvement Program (MHSIP) Survey, the Personal Wellbeing Index adult version (PWI-A), and the Maryland Outcomes Measurement System. NYS OMH also formulated questions for pertinent topic areas where none could be found in existing instruments. The PCS is designed to collect experience with behavioral health care in terms of access and perception of quality of life in the areas of health, wellness and social functioning. The PCS will be piloted by NYS in 2016 and will be collected annually on a random sample of HARP members in HARP and HIV-SNP plans starting 2017 through 2020. The annual implementation will be via the EQRO contract.

Since this is a new survey, the State will pilot the instrument and obtain consumer feedback in the fall of 2016. The 2016 pilot will be conducted by NYS OMH and OASAS program staff and will occur in BH specialty program settings including ACT, PROS and OASAS outpatient rehabilitation programs. Medicaid eligible consumers in these settings are expected to be HARP enrolled. Agency consumer affairs liaisons will assist program staff with the survey implementation and to obtain consumer feedback. It is expected that changes will be made to the survey based on the pilot.

HARP members enrolled in HARP or HIV-SNP plans will be surveyed annually starting in 2017. The survey will be implemented by the EQRO using a random sampling methodology of HARP enrollees by product line for HARPs and HIV SNPs. Methods to improve response rate from this representative sample will include reminder calls and mailing.

Measures will be derived at the domain and item levels. Specific survey domains include Perception of Outcomes, Access and Quality of Care, Appropriateness of Services, Social Connectedness, Wellness, and Quality of Life. Demographics are also collected on the form to monitor disparities. Items that will be measured include member´s perception of BH provider´s responsiveness to their cultural background, a seven-item scale measuring satisfaction with quality of life, presence of social support, relationships, and beliefs about health and wellness. In terms of specific measurement methods, satisfaction with quality of life will be measured on a scale from 0 to 10, social connectedness items will be measured on a five item Likert agreement scale, and beliefs about health and wellness will be measured on a four item Likert frequency scale. A draft of the full survey can be found in Appendix B. Data from this survey will allow the State and plans to monitor HARP members´ perception of services and how their behavioral health services affect different areas of their life. Findings will be examined for change in BH services satisfaction levels over time. Surveys will be identified to allow for linking responses to Medicaid claims and other administrative data.

We expect that survey responses will be consistently high and improving over the demonstration time frame. Pre and post comparisons will not be possible given that the PCS survey will be implemented in the 2017-2020 periods with no pre demonstration data collection.

Q11. Costs: To what extent are HARPs cost effective? What are the PMPM cost of acute BH services (e.g. inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox and emergency room services) for the HARP population? Are these costs decreasing over time?

Pre-post approaches (Quantitative Methods I and II) are recommended to be applied to these data to examine the trends and potential changes in costs for care for HARP- eligible members following the implementation of the program. This global assessment could examine whether shifting costs in any of the named service types above are offset elsewhere in the continuum of care (and even where). We expect that costs for HARP enrollees are shifting from acute services to non-acute outpatient-based health and behavioral health services. To assess the potential/expected shifts in cost over time, two separate trend analyses using Quantitative Method I may be conducted to 1) evaluate the PMPM cost trend of acute BH services 2) evaluate the PMPM costs trend of non-acute outpatient services for HARP enrollees pre and post program implementation. In addition, changes in mean annual PMPM cost acute BH services and non-acute outpatient services in the comparable matched cohort will be conducted using Quantitative Method II.

The analyses, PMPM cost of acute and non-acute services as described above will be conducted using data from measurement period-1 (2013 - 2015), (2014 -2016) to measurement period-2 (2016 - 2017), (2017 - 2018), and afterwards through (2019- 2020), for NYC and ROS respectively. Comparable members during the pre and post periods maybe selected using the HARP population algorithm and propensity score matching techniques (Quantitative Method V). Medicaid claims will be utilized for these analyses.

Table C: Evaluation tool for Goal 9
Q #	Outcome	Measure	Data Source	Related Hypotheses	Possible Methods
1	Increase HARP Enrollment	HARP eligible members who in each annual period are in MMC, HARP or HIV SNP (10/2015 -2020)	Medicaid Enrollment	HARP enrollment will increase & the majority of HARP eligibles will enroll in HARP or HIV SNP plans rather than MMC mainstream plans	Year to year comparisons in NYC and in the ROS in each annual period for the period 10/2015 to 12/2020 and reported at the end of the demonstration period (Quantitative Method IV)
2	Describe characteristics of members electing to or declining enrollment in HARP & Reasons for declining enrollment in HARP	Group differences in demographic (age, race, gender), BH service utilization, and diagnostic characteristics of the HARP eligible enrolled members in HARP/HIV-SNP and HARP eligible who opt out for MMMC in each annual period (10/2015-2020). The qualitative reasons for opting out of HARP will be monitored over time and cumulated by year 10/2015 to 12/2020.	Medicaid Claims; Medicaid Choice enrollment data	HARP eligible members who opt out are younger and less behaviorally acute than those who remain enrolled in HARP/HIV SNP	Demographic characteristics, BH service utilization, diagnosis on a year to year basis during the demonstration period. Comparisons will be made using chi-square analysis and Anova as appropriate according to data type (Quantitative Method IV).
3	Compare demographic, social, functional and clinical characteristics of the HARP enrolled population and demographics characteristics of the HARP enrolled compared to HARP eligible population	Year to year comparison (baseline 10/2015-12/2020) of HARP enrollees in terms of social, functional and clinical characteristics in each annual period (10/2015-12/2020) language, risk factors, home environment, social relationships, criminal justice involvement, top health diagnoses, behavioral diagnoses, behavioral health symptoms, substance related practices and behavioral health services accessed. Measures that will be tracked in each annual period are: Percent of HARP enrollees by the following socio-demographic characteristics: age, sex, gender identity, race, ethnicity, preferred language, marital status, education, and sexual orientation Percent of HARP enrollees with the following risk factors: homelessness, criminal justice involvement (arrest history, incarceration history), alcohol use, drug use, chronic conditions, and traumatic life events Percent of HARP enrollees with the following protective factors: employment, enrollment in formal education, social relationships, social strengths, and behavioral health conditions, and traumatic life events Percent of HARP enrollees with the following protective factors: employment, enrollment in formal education, social relationships, social strengths, and behavioral health service utilization	Medicaid claims BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment	On a population level, it is expected that the distribution of the measured risk factors and protective factors for this population will shift toward fewer risk factors and greater protective factors over time as the program matures; Regional (NYC vs ROS) differences in improvements may be observed. On an individual level, trajectories of improvement in risk and protective factors over time will be observed.	Two analytic approaches are recommended to be applied to these data to examine the above questions: population level year by year comparisons (Quantitative Method IV) and individual level analysis of change over time using Quantitative Method III Generalized Linear Mixed Models (GLMM) will be implemented to address the potential heterogeneity in the program/BH implementation effect and estimate an average program effect while controlling/adjusting for important covariates
4	Improve educational and employment characteristics of the HARP population	Year to year comparison (baseline 10/2015- 12/2020) of average HARP beneficiary scores on employment status, employment arrangement, employment compensation, employment supports, enrollment in formal education, and education supports. Measures that will be tracked are: Employment The percentage of members currently employed The percentage of members currently competitively employed The percentage of members employed at least 35 hours per week in the past month The percentage of members employed at or above the minimum wage The percentage of members who prefer change in their employment situation The percentage of members who prefer change in employment supports Education The percentage of members currently enrolled in a formal education program The percentage of members who prefer change in their level of education The percentage of members who prefer a change in educational support services	Medicaid claims BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment	Higher rates of educational and employment attainment will be observed for the HARP enrolled population over time as the program matures; Individual level improvements will be noted	Two analytic approaches are recommended to be applied to these data to examine the above questions: population level year by year comparisons (Quantitative Method IV) and individual level analysis of change over time using Quantitative Method III Generalized Linear Mixed Model (GLMM) will be implemented to address the potential heterogeneity in the program/BH implementation effect and estimate an average program effect while controlling/adjusting for important covariates
5	Improve access to primary and preventive services	Percent of HARP - eligible members in pre period compared with HARP enrolled members in post period with no claims history for primary and preventive services pre and post measurement period-1 (2013 - 2015) to measurement period-2 (2016 -2017), (2017 - 2018), (2019-2020)	Medicaid Claims	Percent of HARP members without primary care access will decline	Quantitative Method II Pre-post design with Difference in difference analysis Quantitative Method II; Quantitative Method V.
6	Improve access to behavioral health care specialty services (See Appendix C for specialty services)	Proportion of HARP enrollees using any and specific BH specialty services pre and post measurement period-1 (2013 - 2015) to measurement period-2 (2016 - 2017), (2017 - 2018), (2019-2020)	Medicaid Claims;	Access to and Utilization of BH specialty services will increase	Quantitative method I Pre- post design with interrupted time series analysis; Quantitative Method II; Quantitative Method V.
7	Increase access to care coordination (health homes)	Proportion of HARP enrollees´ engaged in health homes pre and post measurement period-1 (2013 - 2015) to measurement period-2 (2016 - 2017), (2017 - 2018), (2019-2020)	Medicaid Claims	Access to care coordination services will increase in terms of health home engagement for HARP members	Quantitative method I Pre- post design with interrupted time series analysis; Quantitative Method II; Quantitative Method V.
8	Improve quality of care related to health monitoring, prevention, chronic health and behavioral health (Refer to Table B)	HEDIS®/QARR rates for HARP plans measurement period-1 (2013 - 2015) to measurement period-2 (2016 - 2017), (2017HEDIS®/QARR rates for HARP plans measurement period-1 (2013 - 2015) to measurement period-2 (2016 - 2017), (2017- 2018), (2019-2020)	HEDIS®/QARR Medicaid Claims	HEDIS®/QARR quality profiles for HARP plans will improve over time as the program matures	Comparable members during the pre and post periods will be selected using the HARP population algorithm and propensity score matching techniques (Quantitative Method V). Quantitative method II Pre-post design with using DID analysis
9 & 10	Improve HARP enrollees self-reported experience of care related to access, health, behavioral health and HCBS services; Improve HARP enrollees satisfaction with care in terms of wellness and recovery, social connectedness and cultural sensitivity of services.	Percent of HARP enrollees that were satisfied with access to care, communication and knowledge of Medicaid managed care in each annual period (2017-2020) Measures derived from the CAHPS® survey that will be tracked in 2017 and 2019 are: Percentage of HARP enrollees who report that was easy to get mental health treatment Percentage of HARP enrollees who report that was easy to get SUD treatment Percentage of HARP enrollees who rated their mental health treatment positively Percentage of HARP enrollees who rated their SUD treatment positively Percentage of HARP enrollees who rated items related to communication with health care providers positively Measures that will be derived from the PCS are: Percentage of HARP members who report that their behavioral health care was responsive to their cultural background in each annual period (2017-2020) Percent of HARP members who had a positive overall rating of quality of life in each annual period (2017-2020). Percent of HARP members who had overall positive beliefs about health and wellness in each annual period (2017-2020) Percent of HARP members who rated PCS survey questions in the social connectedness domain positively in each annual period (2017-2020).	CAHPS® Survey HARP Perception of Care Survey	Perception of experience of care and satisfaction with care will improve over time as the program matures. HARP enrollee satisfaction with the cultural sensitivity of their behavioral health providers will increase over the length of the demonstration. HARP enrollee satisfaction with their wellness, recovery, and degree of social connectedness will improve over the time of the demonstration.	Quantitative Method IV Year to year comparisons in NYC and in the ROS in each annual period for the period 10/2015 to 12/2020 and reported at the end of the demonstration period
11	Decrease PMPM cost of inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox and emergency room services for the HARP population	PMPM cost of acute and non-acute services will be conducted using data from measurement period-1 (2013 - 2015) to measurement period-2 (2016 - 2017), (2017 - 2018), (2019-2020), for NYC and ROS respectively.	Medicaid claims	We expect that costs for HARP enrollees are shifting from acute services to non-acute outpatient-based health and behavioral health services.	Analytic Method I Pre-post design with interrupted time series analysis; Quantitative Method II

Goal 10: Develop Home and Community Based services focused on recovery, social functioning, and community integration for individuals in HARPs meeting eligibility criteria

The quantitative methods to be used to investigate the four questions related to this goal are discussed below. The outcomes, measures, data sources and hypotheses to be tested are shown in the Evaluation tool for Goal 10 (Table E) below. HARP refers to HARP enrollees in HARP or HIV SNP plans.

Evaluation Questions

Access to Care: To what extent are HARP enrollees deemed eligible to receive Home and Community Based Services (HCBS)?

Question 1 focuses on examining the HCBS eligibility determinations for HARP members and HARP eligible HIV-SNP members. All HARP and HARP eligible HIV-SNP members will be assessed for HCBS eligibility using the BH HCBS Eligibility Brief Assessment. The BH HCBS Eligibility Brief Assessment is used to identify individuals who may have functional needs and service/support needs that could be addressed by HCBS services. HCBS services are divided into two tiers. Eligibility for Tier 1 services will include a lower threshold for needs than Tier 2 services. Tier I includes peer, employment and/or education supports. Tier 2 includes all Tier 1 BH HCBS services plus additional services as specified in Table D to individuals whose medical need surpasses the need for Tier 1 services. Crisis respite HCBS services are available to all HARP enrollees, regardless of the tier under which they receive services. This includes intensive crisis respite or short-term crisis respite in a dedicated facility. Individuals determined to be HCBS eligible receive a comprehensive assessment using the BH HCBS Full Assessment tool. The BH HCBS Full Assessment is used to develop a client-centered plan of care for the individual.

Behavioral Health Home and Community Based Services were offered beginning in January 2016 in NYC and in October 2016 for ROS.

We expect that 75% of HARP members will be eligible for any HCBS services, 75% of HARP members will be eligible for HCBS Tier 1 services and 70% of HARP members will be eligible for HCBS Tier 2 services. We expect these targets to be achieved by the end of the demonstration. Comparisons will be made to examine characteristics of HARP enrollees deemed eligible in NYC and in the ROS in each annual period (10/2015-2020), and from year to year using descriptive statistical methods for categorical, ordinal or continuous data (Quantitative Method IV). Data from the BH HCBS Eligibility Brief Assessment (demographic, clinical) and from Medicaid claims (plan membership, HCBS eligibility status) will be utilized for these analyses.

It is important to note that for this measure, there is no pre-implementation comparison possible. For Goal 10 Questions 1 and 2 we expect that as the HARP program matures, it would be possible to compare those members eligible for HCBS and those receiving HCBS to those deemed ineligible or eligible but not accessing services. These comparisons could examine any significant differences in term of population demographic characteristics (e.g. age, gender, residential region), plan membership (HARP Plan) and clinical characteristics (e.g., MH Dx, SUD Dx, Dual Dx).

Table D: Behavioral Health HCBS
BH HCBS Assessment BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment
Rehabilitation Psychosocial Rehabilitation Community Psychiatric Support and Treatment (CPST)
Empowerment Services-Peer Supports
Habilitation Services
Respite Short-term Crisis Respite Intensive Crisis Respite
Non-medical transportation
Family Support and Training
Employment Supports Pre-vocational Transitional Employment Intensive Supported Employment On-going Supported Employment
Education Support Services

To what extent are HARP enrollees who are deemed eligible receiving Home and Community Based Services?

The expectation is that the monthly, and annual utilization of HCBS services will increase over the demonstration period. We expect that 75% of HARP members deemed eligible for HCBS services will utilize these services. The monthly and annual rate of utilization of BH HCBS will be examined using (Quantitative Method VI). The unit of analysis will be rates at which HARP enrollees deemed eligible use BH HCBS services in a month and within the year. This will be measured as the proportion of HARP enrollees receiving one or more BH HCBS service in each tier in a month and within the year. Rates will be examined monthly and annually at the statewide, regional and HARP plan levels over the period 2016-2020. Also, average annual percent change in program enrollments or service use or both will be assessed at the statewide, regional levels from year to year starting from 2016 and thereafter. The average annual percent change for the year of assessment will be calculated as the difference in average service use between that year and the prior year divided by the average of the prior year. Data from the BH HCBS Eligibility Brief Assessment (demographic, clinical) and from Medicaid claims (plan membership, HCBS eligibility status) will be utilized for these analyses. Additionally, GLMM (Quantitative Method III) will be used to examine the association between BH HCBS service utilization for those deemed eligible (used versus not, used 6 or more months versus less) controlling for demographic and clinical characteristics, and time.

It is important to note that for this measure, there is no pre-implementation comparison possible. For Question 1 and question 2 we expect that as the HARP program matures, it would be possible to compare those members eligible for HCBS and those receiving HCBS to those deemed ineligible or eligible but not accessing services. These comparisons could examine any significant differences in term of population demographic characteristics (e.g. age, gender, residential region), plan membership (HARP Plan) and clinical characteristics (e.g., MH Dx, SUD Dx, Dual Dx).

To what extent has the demonstration developed provider network capacity to provide behavioral health Home and Community Based Services for HARPs?

This question addresses the need for network adequacy to provide HCBS services. It is important to note that for this measure, there is no pre-implementation comparison possible, but as the HARP program matures, it would be possible to monitor rates of provider participation in HARPs / HIV SNPs and the rate of providers per member population.

The extent to which HCBS service providers are available and contracted with by MMC HARP/HIV SNP plans will be examined. The measures include the number of providers contracted for BH HCBS in MMC HARP plans and the Ratio of BH HCBS providers per 1,000 enrollees. Year to year comparisons for the period 2016-2020 at the statewide, NYC, and ROS, county and HARP plan levels will be conducted (Quantitative Method IV). The Medicaid Managed Care HCBS Provider Network Data System will be used to determine HCBS provider information related to geographic areas served and plan contracts. Medicaid claims will be used to determine HARP enrollment.

A year to year comparison of the number of complaints related to access to HCBS services will be done. Collection of complaints related to HCBS is done through a designated email address which has been available to New York State OMH Providers since October 2015. OMH has designated staff to monitor and manage the mailbox. Designated staff has created an extended tracking system that includes multiple fields. These fields include origin of inquiry, type of inquiry, Primary and Secondary topics, fields for each MCO to indicate if they are part of the inquiry, which NYS region the inquirer is located in, name of the inquirer, and if forwarded to other state agencies. Through this data collection, issues related to HCBS are identified, monitored and remedied.

Monitoring of complaints is coordinated with the New York State Office of Alcoholism and Substance Abuse Services (OASAS) and the New York State Department of Health (DOH). The three agencies meet regularly to identify trends, urgent issues and outstanding emails. NYS OMH is able to generate complaint reports from a linked database. These reports can be created via subject matter, if routed to DOH/OASAS, type of inquiry (complaint, question) and date opened/completed.

Does targeting of BH HCBS more narrowly lead to increased numbers of members without access to appropriate BH care? (What are the consequences of targeting availability of BH HCBS to a more narrowly defined population than the criteria in the State Plan?)

The State will examine how total costs PMPM have increased or decreased following the implementation of HARP and for HARP enrollees with and without BH HCBS access through HARPs using Quantitative Methods I and II. The state will recommend a pre- post design that examines the potential changes in costs for care for HARP-eligible members following the implementation of the program.

We expect that the added costs arising from access to BH HCBS are offset elsewhere in the continuum of care. For example, we expect that costs and utilization of employment, education or peer services will offset hospital costs and utilization over the course of the demonstration.

The outcome metrics, health care costs per member per month/year (PMPM/Y) and service use rates, such as hospital admit rates measured over two consecutive periods of two years before and two years after program implementation will be calculated (total duration of four years). Changes in outcome metrics from measurement period-1 (2013 - 2015), (2014 - 2016), to measurement period-2 (2016 - 2017), (2017 - 2018), will be compared for NYC and ROS respectively. Also, changes in individuals who are HCBS eligible and opt for HCBS services will be compared to individuals who are HCBS eligible and do not opt for HCBS services using similar match-pairing and DD techniques. Specific HCBS service types will also be tested. Changes in individuals who are Tier 1 HCBS eligible and opt for Tier 1 HCBS services will be compared to individuals who are Tier 1 HCBS eligible and do not opt for Tier 1 HCBS services using similar match-pairing and DD techniques. Additionally, changes in individuals who are Tier 2 HCBS eligible and opt for Tier 2 HCBS services will be compared to individuals who are Tier 2 HCBS eligible and do not opt for HCBS services using similar match-pairing and DD techniques.

Table E: Evaluation tool for Goal 10
Q #	Outcome	Measure	Data Source	Related Hypotheses	Possible Methods
1	Access to Care: To what extent are HARP enrollees deemed eligible to receive Home and Community Based Services?	Year to year comparison of statewide, NYC, and ROS rates of percentages of HARP enrollees deemed eligible for any and for specific HCBS services	BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment Medicaid Claims	We expect that 75% of HARP members will be eligible for any HCBS services, 75% of HARP members will be eligible for HCBS Tier 1 services and 70% of HARP members will be eligible for HCBS Tier 2 services by the end of the demonstration	Comparisons will be made to examine characteristics of HARP enrollees deemed eligible in NYC and in the ROS in each annual period (10/2015-2020), and from year to year using descriptive statistical methods for categorical, ordinal or continuous data (Quantitative Method IV).
2	Access to Care: To what extent are HARP enrollees who are deemed eligible receiving Home and Community Based Services?	The monthly and annual rate of utilization of BH HCBS will be examined at the statewide, regional and HARP plan levels over the period 2016-2020. Data from the BH HCBS Eligibility Brief Assessment (demographic, clinical) and from Medicaid claims (plan membership, HCBS eligibility status) will be utilized for these analyses to	Medicaid Claims BH HCBS Eligibility Brief Assessment	We expect PMPM BH HCBS utilization to increase over the course of the demonstration.	Monthly and Yearly rate of utilization of BH HCBS will be examined using Quantitative Method VI and Generalized Linear Mixed Model (GLMM, Quantitative Method III) used to address the potential heterogeneity in BH HCBS service use and estimate an average program effect while controlling/adjusting for important covariates Rates will be examined at the statewide, regional and HARP plan levels over the period 2016-2020
3	To what extent has the demonstra tion developed provider network capacity to provide behavioral health Home and Community Based Services for HARPs?	Year to year comparison of statewide, NYC, and ROS rates of behavioral health home and community-based provider participation in Medicaid managed care plans by county; ratio of BH HCBS providers per 1,000 enrollees; Examine complaints and appeals to determine if plans, providers or members have requested BH HCBS but were	BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment Medicaid Claims Complaints and appeals submitted to the State Medicaid Managed Care HCBS Provider Network Data System	We expect the number and ratio of BH HCBS providers per 1,000 enrollees to increase over the course of the demonstration	Year to year comparisons for the period 2016-2020 at the statewide, NYC, and ROS, county and HARP plan levels will be conducted (Quantitative Method IV).
4	Access to care: What are the consequences of targeting availability of BH HCBS to a more narrowly defined population than the criteria in the State Plan? What are the PMPM costs of BH HCBS for HARP enrollees who receive services?	Outcome metrics, health care costs per member per month/year (PMPM/Y) and service use rates, will be calculated (total duration of four years). Changes in outcome metrics from measurement period-1 (2013 - 2015), (2014 - 2016), to measurement period-2 (2016 - 2017), (2017 - 2018), will be compared for NYC and ROS respectively. Also, changes in individuals who are HCBS eligible and opt for HCBS services will be compared to individuals who are HCBS eligible and do not opt for HCBS services using similar match-pairing and DD techniques.	Medicaid Claims BH HCBS Eligibility Brief Assessment BH HCBS Full Assessment	We expect that the added costs arising from access to BH HCBS will be offset elsewhere in the continuum of care.	Quantitative Methods 1 and 2: The State recommends a pre-post design that examines the potential changes in costs for care for HARP-eligible members following the implementation of the program. The outcome metrics, health care costs per member per month/year (PMPM/Y) and service use rates, such as hospital admit rates measured over two consecutive periods of two years before and two years after program implementation will be calculated (total duration of four years). Changes in outcome metrics from measurement period-1 (2013 - 2015), (2014 - 2016), to measurement period-2 (2016 - 2017), (2017 - 2018), will be compared for NYC and ROS respectively. Also, changes in individuals who are HCBS eligible and opt for HCBS services will be compared to individuals who are HCBS eligible and do not opt for HCBS services using similar match-pairing and DD techniques.

References

Cook, TD, Campbell, DT. (1979). Quasi-experimentation: Design and analysis for field Settings. Boston, MA: Houghton Mifflin Company.
Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics 2002; 27:299-309. [PubMed: 12174032]
Jeffrey S. Harman, Christy H. Lemak, Mona Al-Amin, Allyson G. Hall, and Robert Paul Duncan, Changes in Per Member Per Month Expenditures after Implementation of Florida´s Medicaid Reform Demonstration, Health Services Research 2010.01226
Diggle, PJ.; Heagerty, P.; Liang, K-Y.; Zeger, SL. Analysis of Longitudinal Data. New York: Oxford University Press; 2002.
Tooze JA, Grunwald GK, Jones RH. Analysis of repeated measures data with clumping at zero. Statistical Methods in Medical Research. 2002; 11:341-355. [PubMed: 12197301]
Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med 2007; 26(4): 734-753. DOI: 10.1002/sim.2580
Macro %gmatch. Available here. Accessed September 15, 2011.
Austin PC. Assessing balance in measured baseline covariates when using many- to-one matching on the propensity-score. Pharmacoepidemiol Drug Saf 2008; 17(12): 1218-1225. DOI: 10.1002/pds.1674
Bradley, A.H., Curry, L.A., Devers, K.J. (2007). Qualitative data analysis for health services research: Developing taxonmy, themes, and theory. Health Services Research, 42(4), 1758-1772.
Heinssen RK, Goldstein AB, Azrin ST: Evidence-Based Treatments for First Episode Psychosis: Components of Coordinated Specialty Care. Rockville, Md, National Institute of Mental Health, 2014. Available here.
Dixon LB, Goldman HH, Bennett ME, et al: Implementing coordinated specialty care for early psychosis: the RAISE Connection Program. Psychiatric Services 66:691-698, 2015
Kane JM, Robinson DG, Schooler NR, et al: Comprehensive versus usual community care for first-episode psychosis: 2-year outcomes from the NIMH RAISE Early Treatment Program. American Journal of Psychiatry 173(4): 362- 72, 2016
New York State Department of Health. New York State Delivery System Reform Incentive Payment Program Project Toolkit.
Hirdes JP, Marhaba M, Smith TF, Clyburn L, Mitchell L, Lemick RA, Telegdi NC, Pérez E, Prendergast P, Rabinowitz T, Yamauchi K, Resident Assessment Instrument-Mental Health Group: Development of the resident assessment instrument--mental health (RAI-MH). Hosp Q. 2000, 4: 44-51.
National Committee for Quality Assurance. HEDIS® 2016 Technical Specifications for Health Plans, Volume 2; October 2015.
New York Request for Qualifications for Adult Behavioral Health Benefit Administration: Managed Care Organizations and Health and Recovery Plans. March 21, 2014.
Brown, R.G. (1962), Smoothing, Forecasting and Prediction of Discrete Time Series, New York: Prentice-Hall.

|top of section| |top of page|

Appendix A

HARP Targeting Criteria and Risk Factors

HARPs: Adult Medicaid beneficiaries 21 and over who are eligible for mainstream MCOs are eligible for enrollment in the HARP if they meet either:
1. Target criteria and risk factors as defined below (Individuals meeting these criteria will be identified through quarterly Medicaid data reviews by Plans and/or NY State); or
2. Service system or service provider identification of individuals presenting with serious functional deficits as determined by:
  1. A case review of individual´s usage history to determine if Target Criteria and Risk Factors are met; or
  2. Completion of HARP eligibility screen.
HARP Target Criteria: The State of New York has chosen to define HARP targeting criteria as:
1. Medicaid enrolled individuals 21 and over;
2. SMI/SUD diagnoses;
3. Eligible to be enrolled in Mainstream MCOs;
4. Not Medicaid/Medicare enrolled ("duals");
5. Not participating or enrolled in a program with the Office for People with Developmental Disabilities (OPWDD) (i.e., participating in an OPWDD program).
HARP Risk Factors: For individuals meeting the targeting criteria, the HARP Risk Factor criteria include any of the following:
1. Supplemental Security Income (SSI) individuals who received an "organized"5 MH service in the year prior to enrollment.
2. Non-SSI individuals with three or more months of Assertive Community Treatment (ACT) or Targeted Case Management (TCM), Personalized Recovery Oriented Services (PROS) or prepaid mental health plan (PMHP) services in the year prior to enrollment.
3. SSI and non-SSI individuals with more than 30 days of psychiatric inpatient services in the three years prior to enrollment.
4. SSI and non-SSI individuals with 3 or more psychiatric inpatient admissions in the three years prior to enrollment.
5. SSI and non-SSI individuals discharged from an OMH Psychiatric Center after an inpatient stay greater than 60 days in the year prior to enrollment.
6. SSI and non-SSI individuals with a current or expired Assisted Outpatient Treatment (AOT) order in the five years prior to enrollment.
7. SSI and non-SSI individuals discharged from correctional facilities with a history of inpatient or outpatient behavioral health treatment in the four years prior to enrollment.
8. Residents in OMH funded housing for persons with serious mental illness in any of the three years prior to enrollment.
9. Members with two or more services in an inpatient/outpatient chemical dependence detoxification program within the year prior to enrollment.
10. Members with one inpatient stay with a SUD primary diagnosis within the year prior to enrollment.
11. Members with two or more inpatient hospital admissions with SUD primary diagnosis or members with an inpatient hospital admission for an SUD related medical diagnosis-related group and a secondary diagnosis of SUD within the year prior to enrollment.
12. Members with two or more emergency department (ED) visits with primary substance use diagnosis or primary medical non-substance use that is related to a secondary substance use diagnosis within the year prior to enrollment.
13. Individuals transitioning with a history of involvement in children´s services (e.g., RTF, HCBS, B2H waiver, RSSY).
Behavioral Health Home and Community Based Services (BH HCBS) Service Eligibility and Assessment Process: HARP members who meet Targeting Criteria and Risk Factors as well as Need-Based Criteria (below), will have access to an enhanced benefit package of BH HCBS.
1. Need-based Criteria: Individuals meeting one of the Needs-Based Criteria identified below will be eligible for BH Home and Community Based Services:
  1. An individual with at least "moderate" levels of need as indicated by a State designated score on a tool derived from the interRAI Assessment Suite.
  2. An individual with need for BH HCBS services as indicated by a face to face assessment with the interRAI Assessment Suite and a risk factor of a newly emerged psychotic disorder suggestive of Schizophrenia herein called individuals with First Episode Psychosis (FEP). Individuals with FEP may have minimal service history.
  3. A HARP enrolled individual who either previously met the needs-based criteria above or has one of the needs based historical risk factors identified above; AND who is assessed and found that, but for the provision of BH HCBS for stabilization and maintenance purposes, would decline to prior levels of need (i.e., subsequent medically necessary services and coordination of care for stabilization and maintenance is needed to prevent decline to previous needs- based functioning).6
2. All individuals in the HARP will be evaluated for eligibility for BH HCBS.
  1. Once an individual is enrolled in the HARP, a Health Home (or other State- designated entity) will initiate an independent person-centered planning process to determine a plan of care.
  2. This will include the completion of an evaluation for BH HCBS eligibility.
  3. This process will comply with federal conflict-free case management requirements.
3. Individuals determined eligible for the BH HCBS services based on the brief evaluation using the BH HCBS Eligibility Brief Assessment will receive a conflict-free functional assessment from an appropriately qualified individual.
  1. The assessment determines eligibility for BH HCBS and is used to establish a written, person-centered, individualized plan of care.
  2. Assessments are conducted using a BH HCBS Eligibility Assessment, a tool derived from the interRAI, a standardized clinical and functional assessment tool consistent with the State´s approved Balancing Incentive Payment Program.
4. The results of the functional assessment will be incorporated into the individual´s person-centered plan of care.
5. These plans must be approved by the HARP or their designee.
6. Reassessment of the plan of care (including need for BH HCBS) must be done at least annually; when the individual´s circumstances or needs change significantly; or at the request of the individual. Plans may require more frequent reviews of plans of care to evaluate progress towards goals, determine if goals have been achieved or whether the plan of care requires revision.

|top of section| |top of page|

Appendix B

Perception of Care Survey for Health and Recovery Plan (HARP) Members

Thank you for participating in this survey. Please take a moment to review this page for information and instructions.

Purpose of the Survey
This survey is sponsored by the New York State Offices of Mental Health, Office of Alcoholism and Substance Abuse Services and the Department of Health. NYS recently implemented specialized Medicaid Managed Care plans for individuals with behavioral health needs. The plans are designed to provide a wider array of specialty services, care coordination and assistance with things like employment and education.
According to our records, you´re currently enrolled in __________________________. If you are not enrolled in this plan you do not need to complete the survey.
We´re asking you to answer some questions about your experience with this plan as well as the care you received from providers and your perception of your own health and well-being. Your answers will help us continue to improve services and to identify what is working well in these plans.
This survey is specifically asking about the behavioral health services covered in your plan. This include services like counseling, treatment, inpatient, emergency, crisis or medicine for mental health or substance use issues. Please do NOT comment here about services that are NOT covered by your healthcare plan (e.g., self-help groups).

Voluntary and Confidential

Your participation is voluntary. You may choose to complete this survey or not. The benefits and services you receive will not be affected whether you complete this survey or not. Your responses will remain confidential. Please do NOT write your name anywhere on the form.
Your behavioral health providers will NOT have access to your individual responses.

Part I: BEHAVIORAL HEALTH SERVICES EXPERIENCE

Date (month and year) you last received behavioral health services ______________________
☐ If Unknown, check here
In the last 12-months, did you receive any treatment, counseling, or medicine for:
1. Emotional or mental illness? ☐ Yes ☐ No
2. Personal or family concerns? ☐ Yes ☐ No
3. Alcohol use? ☐ Yes ☐ No
4. Drug use? ☐ Yes ☐ No
5. Tobacco use? ☐ Yes ☐ No
Are you currently receiving behavioral health services? ☐ No ☐ Yes → If Yes, Go To Question 5
Please select the ONE main reason why you are no longer receiving counseling or treatment.

☐ a. I no longer needed treatment because the problem that led to treatment was addressed.

☐ b. Treatment was not working as well as expected, so I stopped treatment with this provider.

☐ c. Treatment was no longer possible due to problems with transportation.

☐ d. Treatment was no longer possible due to problems paying for treatment.

☐ e. Treatment was no longer possible due to problems with finding time for treatment.

☐ f. Other reason(s) (please explain):

If you have not received behavioral health services in the past 12-months, skip to Part 3.

Part 2: ACCESS and QUALITY OF CARE

The next questions are about all the behavioral health services you got in the last 12-months that were covered by your healthcare plan. This include services like counseling, treatment, inpatient, emergency, crisis or medicine for mental health or substance use issues. Please consider those services when answering the questions below. Please do NOT comment here about services that are NOT covered by your healthcare plan (e.g., self-help groups). Respond even if you had only one visit in the last 12-months. If you have not received behavioral health services in the past 12-months, skip to Part 3.

In the last 12-months…	Never	Sometimes	Usually	Always	Source
5. How often did the people you went to for counseling or treatment explain things in a way you could understand?	○	○	○	○	ECHO
6. How often did the people you went to for treatment treat you with respect and kindness?	○	○	○	○	ECHO
7. How often did you get services at days/times that were convenient to you?	○	○	○	○	CACS/MHS IP*
8. How often did you get services where you needed them?	○	○	○	○	ECHO*
9. How often did you get the services you needed as soon as you wanted?	○	○	○	○	ECHO
10. How often did the people you went to for counseling or treatment spend enough time with you?	○	○	○	○	ECHO
11. How often did you feel safe when you were with the people you went to for counseling or treatment?	○	○	○	○	ECHO
12. How often did the people you went to for treatment listen carefully to you?	○	○	○	○	ECHO
13. How often were you involved as much as you wanted in your treatment?	○	○	○	○	ECHO

The following questions are about services that you may or may not receive through your healthcare plan. You would have received an assessment to determine if you were eligible for these services. For each of the services, please indicate whether you received the service from your healthcare plan in the past 12-months, and rate how helpful you found the services.

Possible Services in Your Plan of Care	Received this service in the past 12-months?			If you received this service in the past 12-months, how helpful was the service?
Possible Services in Your Plan of Care	No, I did not need it	No, but I needed it	Yes	Very Helpful	Somewhat Helpful	Not at All Helpful
14. A Health Home care manager who coordinates your medical, behavioral health, and social service needs
15. Peer support services (services provided by people who have experienced mental illness and/or substance use disorder and who work to help others with a mental illness and/or substance use disorder; e.g., recovery support, companionship during a crisis, assistance with self-help tools and helping with transitioning from the hospital to home)	○	○	○	○	○	○
16. Assistance with returning to school or a training program	○	○	○	○	○	○
17. Assistance with finding or maintaining a job
18. Assistance with transportation other than medical transportation	○	○	○	○	○	○
19. Help with finding housing or better housing	○	○	○	○	○	○
20. Help in pursuing friendships and personal interests	○	○	○	○	○	○
21. Help in figuring out my finances, including getting any benefits I may be entitled to	○	○	○	○	○	○
22. Family support and training	○	○	○	○	○	○
23. Crisis respite services; i.e., residential care for 7 days or less, during a behavioral health crisis	○	○	○	○	○	○
24. Help with developing a crisis or relapse prevention plan	○	○	○	○	○	○

a. Does your language, race, religion, ethnic background or culture make any difference in the kind of behavioral health care you need? ☐ Yes ☐ No [proposed RCE transformation item]

b. If yes, in the past 12 months, was the care (services) you received responsive to those needs? ☐ Yes ☐ No
Using any number from 0 to 10, where 0 is the worst counseling or treatment possible and 10 is the best counseling or treatment possible, what number would you use to rate all your counseling or treatment in the last 12-months? [ECHO #28]

0 1 2 3 4 5 6 7 8 9 10
In the last 12-months, how much were you helped by the counseling or treatment you got? [ECHO #29]

☐ Not at all ☐ Somewhat ☐ Very Much
a. In the last 12-months, did you take any prescription medicines as part of your treatment?

☐ Yes ☐ No → If No, Go to Question 29 [ECHO 16]

b. How often were you told what side effects of medicines to watch for? [ECHO 17 modified]

☐ Never ☐ Sometimes ☐ Usually ☐ Always
a. In the last 12-months, have you needed accommodations (for example wheelchair accessibility) in order to obtain services? ☐ Yes → If No, Go to Question 30 [OMH item]

b. How often were accommodations you needed available? ☐ Never ☐ Sometimes ☐ Usually ☐ Always

Part 3: HEALTH, WELLNESS, AND QUALITY OF LIFE

The next questions are about your health. If you are unsure about how to answer a question, please give the best answer you can.

In general, how would you rate your overall mental or emotional health? (Please select one)
[ECHO #30 modified]

☐ Excellent ☐ Very good ☐ Good ☐ Fair ☐ Poor
In general, how would you rate your overall physical health? (Please select one) [ECHO #44 modified]

☐ Excellent ☐ Very good ☐ Good ☐ Fair ☐ Poor
During the past 4 weeks, how much difficulty did you have doing your daily work, both at home and away from home, because of your physical health? (Please select one) [SF8]

☐ None at all ☐ Very little ☐ Somewhat ☐ Quite a lot ☐ Could not do physical activities

The following questions ask about how you are feeling now compared to 12-months ago. Please answer using the scale "Much Better" to "Much Worse."

Compared to 12-months ago, how would you rate…	Much Better	A Little Better	About the Same	A Little Worse	Much Worse	Source
33. your ability to deal with daily problems now?	○	○	○	○	○	ECHO
34. your ability to deal with social situations now?	○	○	○	○	○	ECHO
35. your ability to accomplish the things you want to do now?	○	○	○	○	○	ECHO
36. your problems or symptoms now?	○	○	○	○	○	ECHO

The next group of questions ask about how satisfied you feel, using the Zero to 10 scale. Zero means you feel no satisfaction at all. 10 means you feel completely satisfied. The middle of the scale is 5, which means you are neither happy nor sad. [PWI- A]

How satisfied are you with……?	0	1	2	3	4	5	6	7	8	9	10
37. the things you have? Like the money you have and the things you own?	○	○	○	○	○	○	○	○	○	○	○
38. your health?	○	○	○	○	○	○	○	○	○	○	○
39. what you are achieving in life?	○	○	○	○	○	○	○	○	○	○	○
40. your personal relationships?	○	○	○	○	○	○	○	○	○	○	○
41. how safe you feel?	○	○	○	○	○	○	○	○	○	○	○
42. feeling part of your community?	○	○	○	○	○	○	○	○	○	○	○
43. how things will be later on in your life?	○	○	○	○	○	○	○	○	○	○	○

Following is a list of statements about your attitudes and beliefs about your health and wellness. There are no right or wrong answers. We just want to know what you think about these things. Read each statement and then decide how often you agree with it, from Never to Always.

	Never	Sometimes	Usually	Always	Source
44. I am confident that I can make positive changes in my life	○	○	○	○	MD ARS- SF
45. I am hopeful about the future	○	○	○	○	MD ARS- SF
46. I believe I make good choices in my life	○	○	○	○	MD ARS- SF
47. I am able to set my own goals in life	○	○	○	○	MD ARS- SF
48. I feel accepted as who I am	○	○	○	○	MD ARS- SF
49. I do things that are meaningful to me	○	○	○	○	MD ARS- SF
50. I am able to take care of my needs	○	○	○	○	MD ARS- SF
51. I am able to handle things when they go wrong	○	○	○	○	MD ARS- SF
52. I am able to do things that I want to do	○	○	○	○	MD ARS- SF

	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree	Source
53. I am aware of community supports available to me.	○	○	○	○	○	RCE
54. My living situation feels like home to me.	○	○	○	○	○	RMQ
55. I have access to reliable transportation.	○	○	○	○	○	RCE*
56. I have trusted people I can turn to for help.	○	○	○	○	○	RMQ
57. I have at least one close relationship.	○	○	○	○	○	RMQ
58. I am involved in meaningful productive activities.	○	○	○	○	○	RMQ

59. Do you have comments about the behavioral health services that you received or would like to receive?

_________________________________________________________________________________________________________

Part 4 - Background Information

The following information is collected to help ensure that services meet the needs of all individuals. Please do not share your name. Please check the boxes and fill in the blanks as applicable.

What is your zip code? ________________________________
What is your age? _____________________
What was your sex at birth (on your original birth certificate)? ☐ Female ☐ Male ☐ Other
What is your gender identity? ☐ Female ☐ Male ☐ Other
How would you describe your sexual orientation ☐ Heterosexual or Straight ☐ Homosexual, gay or lesbian ☐ Bisexual ☐ Other ☐ Not sure ☐ Prefer not to answer
In what language do you prefer to receive your health care? ☐ English ☐ Other (please specify)
Are you of Hispanic/Latino Origin?

☐ Yes, Hispanic or Latino ☐No, not Hispanic or Latino
What is your race? (Select all that apply)

☐ White ☐American Indian/Alaska Native ☐Asian ☐ Black/African American ☐Native Hawaiian/Other Pacific Islander ☐Other
Were you born in the United States? ☐ Yes ☐ No
What is your highest level of education completed?

☐ Less than High School ☐ High School diploma or GED ☐ Some college, no degree ☐ College degree or higher ☐ Business or technical school
Are you currently enrolled in school? ☐ Yes ☐ No
Are you currently enrolled in a job training program? ☐ Yes ☐ No
Have you been employed in the past 12-months? ☐ Yes, but I am not currently employed ☐ Yes, I am currently employed ☐ No
Please indicate whether the following things affect your ability to work or your decisions about working. Select all that apply to you.

a. Retired and no longer looking for work	○
a. Lack of good jobs	○
b. Concern about losing benefits (e.g., Medicaid, etc.)	○
c. Lack of transportation	○
d. Physical health condition	○
e. Mental health condition	○
f. Arrest history	○
g. Lack of job training / education	○
h. Medication side effects	○
i. Workplace attitudes about mental illness and/or substance use problems	○

Have you been arrested in the past 12-months? ☐ Yes ☐ No
Have you experienced any difficulties with your housing over the past 12-months (e.g., 3 or more moves, having no permanent address, being homeless, living in a shelter)? ☐ Yes ☐ No

Alcohol and Drugs	Yes	No
17. Do you think you have a problem with alcohol?	○	○
18. Do you think you have a problem with drugs?	○	○
19. Do you think you have a problem with tobacco?	○	○

THANK YOU FOR COMPLETING THE SURVEY.

|top of section| |top of page|

Appendix C - List of Community Based Behavioral Health Specialty Services

The following are the community-based behavioral health specialty services that MMC plans are required to offer in their benefit packages:

ACT
PROS
OMH Outpatient Clinic
Continuing Day Treatment
Partial Hospitalization
OASAS Opioid Treatment Program
OASAS Outpatient Clinic
Treatment for first episode psychosis

|top of section| |top of page|

Appendix D - Data Sources

Medicaid Choice Enrollment Data NY Medicaid Choice Enrollment data
New York´s enrollment broker, New York Medicaid Choice is collecting information to track the HARP enrollment process. The number of announcement, passive enrollment, and opt out acknowledgement letters distributed, number of announcement, passive enrollment, and opt out acknowledgement letters returned, number of members enrolled, number of members who opt out, and reasons for opting out are collected on an ongoing basis.

ONTrack NY Data System for tracking First Episode Psychosis treatment
OnTrackNY teams complete data collection forms to provide information on client outcomes and program functioning/services. Information about individual clients is collected through a Referral tracking form, an Admission form, Follow-up forms (submitted quarterly) and a Discharge form. Team-level information such as staffing, functioning and caseload is collected via a quarterly Program components form.

Referral tracking form: referring organization and relationship to potential client, outcome of referral (eligibility evaluation results, declined or enrolled in OnTrackNY).
Admission form: Demographic information (dob, gender, race, marital status, primary language), Educational background (highest grade, current status of school enrollment), Employment status and history (currently employed or not, job/internship history), Family background (education, employment status, primary language of primary support person), Previous psychiatric treatment (psychiatric hospitalizations and psychotropic medications prescribed), Medical & Substance use history, MIRECC GAF score (symptom, occupational functioning and social functioning scale).
Follow-up form: Current primary diagnosis, Service utilization (met with SEES (Supported Education and Employment Specialist), list of core sessions completed), Current antipsychotic medications and side effects evaluation, Education and employment status during the assessment period, Substance use and behavioral concerns (violent behavior, suicide attempts), MIRECC GAF score
Discharge form: Reason for discharge and post discharge services arranged, Education and employment status, Antipsychotic medications at time of discharge, Staff perspective on client outcomes (whether client´s goals for education/employment/symptom management were met), MIRECC GAF score
Program components form: Staffing (FTE devoted to team), Number of team meetings and % time spent on SEES (Supported Education and Employment Specialist)-related activities, Recruitment and evaluation activities (number of individuals contacted the program, number of individuals who began eligibility evaluation, number of individuals who were determined to be eligible).
The State is working to develop a Medicaid claims-based algorithm which will be tested in collaboration with MMMC plans to develop capacity to identify incident cases of FEP using claims and potentially EHR data. This methodology is emergent at this time. The State anticipates that over the course of the Demonstration period that the identification of incident cases of FEP will become more robust.

Medicaid Managed Care HCBS Provider Network Data System
NYS OMH maintains a database containing information on providers who applied to provide BH HCBS. The database contains provider contact information, provider location, specific service(s) provided, staff qualifications, and funding information. NYS OMH also will collect from MMC plans a list of BH HCBS providers that plans have contracted with.

Medicaid Claims
This database contains billing records for health care services, including pharmacy, for approximately 5.7 million individuals enrolled in Medicaid in a given year. Also included are data on Medicaid enrollment status, diagnoses and provider associated with the billed services. The Medicaid claims database is updated on a monthly basis to include additional claims and modifications to existing claims. Medicaid claims database will receive data from all managed care plans providing services to the demonstration population. Given the claims processing, there is a 6-month lag in the availability of complete and finalized Medicaid claims data, where data for a given year are considered final by June 30th of the following year.

Consumer Assessment of Healthcare Providers and Systems (CAHPS®®)
The Health Plan version of the CAHPS®® survey will be administered to adults by NYSDOH every other year during the BH Demonstration period and will serve as the data source for selected member experience measures. The survey is administered by both mail and telephone and assesses patients´ experiences with health care providers and health plan staff. This includes information on patient experience with access to care, experiences with health care providers and health plan support. The survey includes standardized questionnaires for adults and children. Given confidentiality agreements, only de-identified CAHPS® data will be available for use. Data will be self-reported and from a sample of Medicaid Managed Care members. The experiences of the survey respondent population may be different than those of non-respondents with respect to their health care services. Therefore, data users should consider the potential for non-response bias when interpreting CAHPS results.

BH HCBS Eligibility Brief Assessment and BH HCBS Full Assessment
The Uniform Assessment System contains the BH HCBS Eligibility Brief Assessment and BH HCBS Full Assessment data on HARP eligible individuals enrolled HARPs or HIV SNPS. Data include patient functional status, living situation, employment, education, behavioral health status, health status, cognitive functioning, and care preferences. The assessments include comprehensive sections on mental health state and substance use behaviors, including the following domains: Mental state indicators, Substance use or excessive behaviors, Harm to self and others, Behavior, Cognition, Stress and trauma. In terms of social functioning, the assessments include comprehensive sections on: Cognition, Functional status, Social relations, employment, education and finances, and environmental assessment. Data are a mix of self- reported information and information that is available to assessors through the care management process. Data users should consider the potential for self-reported items to be inaccurate.

HEDIS®/QARR Plan Reported Metrics
MMC plans, HARPs, and HIV SNPs will report HEDIS®/QARR data to NYS DOH annually. To supplement the QARR measurement set, the State will produce Behavioral Health Medicaid Outcome Measures at least annually. These reports will be based on Medicaid claims data and include measures related to inpatient discharge events and also measures related to outpatient care. The State accesses data in the Medicaid Data Mart. Encounter cost data is only available in the OHIP Data Mart. As a result, both Medicaid sources are cited below in Figure 2. The measures will cover both the mental health and substance use disorder populations. Starting in the first year of implementation, metrics will be produced for the HARP* and MMMCO plans. Change over time in the above HEDIS®/QARR and NYS Behavioral Health Medicaid Outcome Measures will be examined.

Where there are gaps in HEDIS®/QARR utilization data, the State will produce service utilization measures. The State will monitor utilization of behavioral health services beginning in the first year of implementation. Monitoring will consist of utilization of services, cost, and encounter volume by behavioral health service. This monitoring will allow the State to determine if services are being provided at an appropriate volume. It is important that the transition of behavioral health services into managed care does not disrupt members´ treatment. These reports will also allow the State to monitor utilization of the new BH HCBS.

HARP Perception of Care Survey
HARP members enrolled in HARP or HIV-SNP plans will be surveyed annually to measure perception of care and quality of life outcomes. The survey will be implemented by the EQRO using a random sampling methodology of HARP enrollees by product line for HARPs and HIV SNPs. The first survey is expected to be piloted in late 2016. The survey instrument will consist of approximately 50 questions and will be mailed to a random sample of eligible HARP members. Methods to improve response rate (e.g., web and mail survey administration, administration by peer advocates, sending reminders) from this representative sample are under review. Demographics will be collected, which will allow HARPs to monitor disparities. Data from this survey will allow the State and plans to monitor HARP members´ perception of services and how their behavioral health services affect different areas of their life. Specific survey domains include Perception of Outcomes, Daily Functioning, Access to Services, Appropriateness of Services, Social Connectedness, and Quality of Life. Findings will be examined for change in BH services satisfaction levels over time. Data will be self-reported and from a sample of HARP members.

The experiences of the survey respondent population may be different than those of non-respondents with respect to their health care services. Therefore, data users should consider the potential for non-response bias when interpreting HARP PCS results.

(see Medicaid.gov for remaining Appendices for this Attachment)

|top of section| |top of page|

ATTACHMENT I

DSRIP Program Funding and Mechanics Protocol

I. Preface

Delivery System Reform Incentive Payment Fund
- On April 14, 2014, the Centers for Medicare and Medicaid Services (CMS) approved New York´s request for an amendment to the New York´s Partnership Plan section 1115(a) Medicaid demonstration extension (hereinafter "demonstration") authorizing the creation of a Delivery System Reform Incentive Payment (DSRIP) Fund. In 2016, the demonstration was extended and renamed the New York Medicaid Redesign Team Demonstration. These protocols are effective through March 31, 2021.
  
  Section IX of the Special Terms and Conditions (STC) describes the general rules and requirements of the Delivery System Reform Incentive Payment (DSRIP) Fund.
DSRIP Strategies Menu and Metrics and Program Funding and Mechanics Protocol
- The DSRIP requirements specified in the STCs are supplemented by two attachments to the STCs. The Program Funding and Mechanics Protocol (this document, Attachment I) describes the State and CMS review process for DSRIP project plans, incentive payment methodologies, reporting requirements, and penalties for missed milestones. The DSRIP Strategies Menu and Metrics (Attachment J) details the specific delivery system improvement activities that are eligible for DSRIP funding.
  
  This version of the DSRIP Program Funding and Mechanics Protocol is approved. In accordance with STC 10.c in section VII, the state may submit modifications to this protocol for CMS review and approval in response to comments received during the post-award comment period and as necessary to implement needed changes to the program as approved by CMS.

II. DSRIP Performing Provider Systems

An entity that is responsible for performing a DSRIP project is called a "Performing Provider System" (PPS). A PPS must meet all requirements described in the STCs, including the safety net definition (described in STC 2 in section VII). This section provides more detail about the specific criteria that a PPS must meet in order to receive DSRIP funding and the process that the state will follow to assure that a PPS meets these standards.

The state will determine the types of providers eligible to participate as a PPS, as described in paragraph (a) below. All providers are required to form coalitions of providers that participate in DSRIP as a single PPS, as described in paragraph (b) below. Outpatient beneficiary populations will be assigned based on the attribution model described in paragraph (c) below.

Assessment of Safety Net Provider Status
The state will use data from DSH audits and other available information to make an assessment of which providers in the state could be eligible for DSRIP funding, consistent with STC 2 in section VII. This list of providers will be submitted to CMS and will be publicly available on the state´s website. A PPS is expected to continue serving a high proportion of Medicaid and uninsured patients throughout the duration of the demonstration, and significant deviation from these standards will be cause to discontinue DSRIP funding for the PPS after the mid-point assessment.
Coalitions
- Eligible major public general hospitals and other safety net providers are encouraged to form coalitions that apply collectively as a single PPS. The state will review each proposed PPS and may require additional connectivity to additional medical, behavioral health, long-term care, developmental disabilities or social service providers as required to build a comprehensive regional performance network. Coalitions will be evaluated on performance on DSRIP milestones collectively as a single PPS. Coalitions are subject to the following conditions:
  1. Coalitions must designate a lead coalition provider who is primarily responsible for ensuring that the coalition meets all requirements of a PPS, including reporting to the state and CMS. In the process of formally approving each PPS, the state shall articulate a set of standards that each lead entity must follow including specific rules on project oversight, performance payment distribution and other required legal and operational obligations of the lead entity.
  2. Coalitions must establish a clear business relationship between the component providers, including a joint budget and funding distribution plan that specifies in advance the methodology for distributing funding to participating providers. The funding distribution plan must comply with all applicable laws and regulations, including, without limitation, the following federal fraud and abuse authorities: the anti-kickback statute (sections 1128B(b)(1) and (2) of the Act); the physician self-referral prohibition (section 1903(s) of the Act); the gainsharing civil monetary penalty (CMP) provisions (sections 1128A(b)(1) and (2) of the Act); and the beneficiary inducement CMP (section 1128A(a)(5) of the Act). CMS approval of a DSRIP plan does not alter the responsibility of each PPS to comply with all federal fraud and abuse requirements of the Medicaid program.
  3. Coalitions must have a plan for reporting, decision-making, change management, and dispute resolution on performance and incentive payments.
  4. Each coalition must in the aggregate meet the minimum outpatient beneficiary requirements specified in paragraph (d) below.
  5. For coalitions that involve public hospitals that are providing Intergovernmental Transfer (IGT) funding for a project, the public entity providing IGT funding will generally be the lead coalition provider for the PPS that is directly using the IGT match. Private safety net providers can also service as coalition leads as provided in paragraph (e) below.
  6. Each coalition must have a data agreement in place to share and manage patient level data on system-wide performance consistent with all relevant HIPAA rules and regulations.
DSRIP Beneficiary Attribution Method
- The goal of DSRIP is to have each PPS responsible for most or all Medicaid beneficiaries in the given geography or medical market area. It is expected that most of the Medicaid beneficiaries (including dually eligible members) in the state will be attributed to a PPS. The possible exceptions are beneficiaries that are primarily being served by providers not participating in a PPS in the region. However, given the comprehensive nature of DSRIP, it is expected that each approved PPS will include all of the major providers of Medicaid services in their region, greatly reducing the number of beneficiaries not attributed to a PPS. A beneficiary will only be attributed to one PPS, based on the methodology described below.
- Each PPS must include a proposed target population, including a specific geography for the overall performing provider effort. In this way, each PPS will be approved for a specific geography, consisting of one or more counties, based on their application and the state´s review. This specific geography will be utilized to form a service area for the purpose of attribution.
- Utilizing the proposed geography, for each DSRIP Project Plan submitted by a given PPS, the department will identify the Medicaid and uninsured beneficiaries´ population (if applicable) that will be attributed to that system prospectively at the start of each measurement year. This prospective attribution denominator for DSRIP year (DY) 1 will be used in valuation for payment purposes without any adjustments applied, except at the midpoint evaluation as specified in section VI.d of this document. The attributed members will be the collective focus for all projects.
- The aim of the attribution process is to help assign DSRIP participants to the best PPS based on the recipient´s current utilization patterns, including assigned care management and primary care provider as well as the geographical appropriateness of that system. This means beneficiaries will be assigned to a PPS, in their region, which includes the providers most responsible for their care (as determined based on visits to primary service types -including PCP - as described below). The attribution logic will test for a plurality of visits within a PPS. Plurality, for DSRIP purposes, means a greater proportion of services as measured in qualifying visits within the PPS than from services outside the PPS.

1. Two Forms of Attribution:

DSRIP Attribution will come in two forms. The first form of attribution will be to initially assign a given cohort of patients to each PPS. This will be a 1 to 1 match between a PPS and each attributable Medicaid and uninsured member (uninsured members will be attached at the aggregate county level based on census data). This first form of attribution will be called Attribution for Initial Valuation. The second form of attribution will be for performance measurement purposes and will be done at the conclusion of each measurement year to create an appropriate group of members for DSRIP performance measurement purposes - this form of Attribution will be called Attribution for Performance Measurement.

Attribution for Initial Valuation

This initial attribution is done for two basic purposes. The first purpose is to create a number of Medicaid and uninsured lives for use in the calculation of potential performance awards as part of the DSRIP valuation process. The second purpose is to create an initial group of Medicaid members only for initial performance benchmark development. Attribution for Initial Valuation will follow a logic flow based first on 1) the type of PPS, 2) the population subcategory the given Medicaid member falls into, and 3) member loyalty.

PPS Type and Attribution:
Three PPS types are recognized for the purpose of attribution, as described below.

Single PPS in a Region - If a PPS is the only PPS approved by the state in a defined region then all the Medicaid members receiving services⁸ in that region will be attributed to that single PPS. As previously promised by the State, the single PPS in a region will also receive all the non-utilizing Medicaid members (i.e., members enrolled in Medicaid but not receiving any Medicaid paid services) residing in their approved region in their attribution. In addition, the single PPS will receive all the uninsured residing in their approved region if they agree to do project 2.d.i targeted to the uninsured.
Multi PPS in Region - Public Hospital Led/Involved - If a PPS that includes a major public hospital in their network (as lead, co-lead, or network partner) is approved in a region where there is at least one other approved PPS, then the public led/involved PPS will receive all utilizing Medicaid members (with the exception of some low utilizing Medicaid members - see below) that get most of their services from the PPS network through the loyalty assignment methodology described below. This public led/involved PPS will also be given the first opportunity to develop a 2.d.i project specifically designed to serve the uninsured in its region. If this public led/involved PPS opts to do that 2.d.i project, they will then also have all the uninsured members residing in their approved region attributed to their PPS for initial valuation. This public led/involved PPS will also receive (for attribution for payment purposes and again only if they do project 2.d.i) a cohort of non-utilizing and low use Medicaid members in the region. Low use members are those that meet a state definition of lower use designed to target members with use patterns that appear to not be coordinated by PCP or care manager during the attribution period (e.g., ED visits with no evidence of PCP access, Inpatient visits with no primary care etc.). All of these low use members may however be included in the attribution denominators for measurement purposes (and baseline data) based on their current access patterns. This cohort of non-utilizing and low utilizing members will be utilized in attribution and valuation for all Public hospital Led/Involved PPSs and any non- public PPSs approved to do the 11th project 2.d.i as discussed below. This non- utilizing and low utilizing cohort will be determined at the conclusion of the DSRIP application review.
Multi PPS - Non Public Involved - If the PPS is approved in a region that contains at least one other PPSs approved for all or part of their approved region (Multi-PPS) and this region does not include a major public hospital as a major partner in their network, then this non-public involved PPS will receive attribution of utilizing Medicaid members that get most of their services from their PPS network in the loyalty assignment methodology described below. This Multi-PPS (non-public) type is only eligible to receive uninsured and a cohort of low/non-utilizing Medicaid members under one of two scenarios - 1) there is no public PPS in the region or 2) there is a public PPS in the region but the public PPS has opted not to do project 2.d.i. If scenario 1 or 2 materializes, the non- public PPS(s) would then have the option to elect to pursue project 2.d.i. If the non-public PPS(s) decides to pursue project 2.d.i, they will then be eligible to receive uninsured and a cohort of low/non-utilizing Medicaid members in their attribution. If a public led/involved PPS is approved in the region and that public PPS opts not to do the project 2.d.i, then the non-public involved PPS(s) in the region will be offered an opportunity to do so. If the non-public PPS(s) selects project 2.d.i, under such circumstances then they will be assigned the uninsured members residing in their approved PPS region in the attribution for initial valuation based on the percentage of Medicaid members assigned to the PPS(s) in the region (e.g., if a given non-public PPS has 60 percent of the region´s Medicaid population attributed, then they will get 60 percent of the uninsured members). So, if no public led PPS exists in the region or the public declines to do the 11th project 2.d.i, the uninsured members will be divided between any non-public PPS(s) (once they opt to do the 11th project 2.d.i) based on the percentage of Medicaid members assigned to the PPS(s) in the region. Also, the cohort of the low/non-utilizing Medicaid population will be attributed to the any non-public PPS(s) using the same method as the uninsured are distributed; again, they will be assigned this population only if they opt to do project 2.d.i.

Table 1. Attribution for Valuation Logic based on PPS Type:
PPS Type	Medicaid Regular ¹ Utilizers Attribution	Medicaid Non/Low Utilizers Attribution	Uninsured Attribution
Single PPS in Region	All in the defined region	All in the defined region	All in the defined region if the PPS opts for project 2.d.i
Multi PPS in Region - Public Led/Involved	Based on attribution loyalty logic	Cohort in the defined region if the PPS opts for project 2.d.i	All in defined region if PPS opts for project 2.d.i
Multi PPS - Non-Public Involved	Based on attribution loyalty logic	None - unless no public PPS in the region or the public PPS opts not to do project 2.d.i In addition, this PPS would need to do project 2.d.i and Non/Low Utilizer attribution will then be based on PPS MA attribution percentage in region.	None - unless no public PPS in the region or the public PPS opts not to do project 2.d.i In addition, this PPS would need to do project 2.d.i and uninsured attribution will then be based on PPS MA attribution percentage in region.

Attribution by Population Subtype

Four mutually exclusive population subcategory groupings have been set up for DSRIP purposes:
1. Developmental Disabilities (OPWDD Service Eligible - Code 95)
2. Long Term Care (Only NH residents)
3. Behavioral Health (SMI/Serious SUD)
4. All Other
Medicaid members will be placed into one of these population subcategories based on a mutually exclusive hierarchy in the order presented above. In other words, the logic will first look for evidence of Developmental Disabilities and if none exists then evidence of Long-Term Care and if none exists then Behavioral Health and if none exists then the member will be assigned to "All Other." So, for example, if the member meets criteria for developmental disabilities and long-term care, they will be assigned to development disabilities as that is first in the hierarchy. Similarly, if a member does not meet criteria for developmental disabilities but does meet criteria for both long-term care and behavioral health, they will be assigned to long-term care.

After a member is assigned to a population subcategory, they will then be assigned to a PPS based on a loyalty algorithm that is specific to their population subcategory. For instance, if they have been assigned to the behavioral health subcategory the algorithm will check first for care management/health home connectivity and if none exists go on to look for residential connectivity and then ambulatory and so on in hierarchical order.

The following graphic helps to illustrate the overall process.

Fig X: Title

It should be noted that the majority of members will be attributed from within the "All Other" category above. It is estimated that over 80 percent of Medicaid members will be attributed from within that category. Further, while some members in the All Other category with multiple chronic illness will be attributed based on their health home care management agency, clearly most of the "All Other" members are going to be attributed to a DSRIP network based on their health plan assigned PCP as most patients are in health plans and many of those members are utilizing their assigned PCP. If a non-health home member in the "All Other" is not utilizing the assigned PCP they will then be attributed based upon the primary care provider or clinic that they see most often for ambulatory care. If no ambulatory care exists, they will then be attributed based upon emergency department and then inpatient use if necessary. Irrespective of the final attribution, each PPS will be required to make ongoing efforts to work with health plans and providers to align care management, PCP and specialty services for all attributed members in such a way to fully leverage existing positive clinical relationships.

The results of the preliminary attribution process above will be shared with the Medicaid Managed Care organizations (MCOs) for their enrolled members. The MCOs will review the state´s attribution logic/results and suggest any needed changes based on more current member utilization information including more recent PCP assignment or specialty service access. In advance of this attribution process the state will share the DSRIP PPS network with the plan to identify any network alignment gaps that may exist so that the DSRIP PPS and the MCOs can work together to align service delivery and plan contracted networks as appropriate.
Attribution by Loyalty
- Utilizing Medicaid Members will be attributed first based on what population subcategory they belong to and second based on the attribution loyalty logic that has been specifically designed for that given subpopulation by the state. Once the PPS network of service providers is finalized each PPS service network will be loaded into the attribution system for recipient loyalty to be assigned based on visit counts to the overall PPS network in each of the above hierarchical population subcategories.
  
  Once the initial attribution is calculated for the purposes of setting DSRIP project values, the PPS network may only be changed with a DSRIP plan modification (as described in section XI.c below). For each of these population subcategories, the algorithm will check the services provided by each provider and accumulate these visits to the PPS the given provider is partnered with. If a recipient is currently outside the PPS´s geographic area, the visits are excluded (e.g., recipient traveling from upstate to NYC for special surgery). Each PPS associated with the matched provider accumulates the total number of visits for each service/provider combination. Adjustments to attribution based on known variables (e.g., recent changes to the recipient´s address) may be made by the state with MCO input if deemed necessary by data. After all visits against all providers are tallied up for a given service type, the methodology finds the PPS with the highest number of visits for the recipient in each service loyalty level as appropriate. If a single provider is in more than one PPS network (e.g., PCP) then the tie breaking method below may be employed for final matching purposes. This overall process will be designed to ensure that the PPS that is the best fit for the recipient is chosen.
Finalizing Match and Ties
If more than one Performing Provider System (PPS) has the highest number of visits based on the highest priority service loyalty types noted, the methodology re-runs the above logic across all Medicaid service types. This process could break a tie if additional visits in other service types cause one PPS to accumulate more visits. So, for instance, Nursing Home residents that are in nursing homes with connectivity to multiple PPSs may be placed based on their utilization of hospital or other services. If, however, this still results in a tie, the methodology will place the recipient in a separate bucket to be assigned at the end of the process. Recipients who have no predominant demonstrated provider utilization pattern will be assigned to a PPS based on a special logic. If the member is not matched from within the Developmental Disabilities, Long Term Care or Behavioral Health population subcategories the PPS in their geographic region will be chosen by first looking to see if the beneficiary has any primary care provider (PCP) assigned by a Medicaid health plan; if the beneficiary has an assigned PCP the beneficiary will be matched to the PPS that has that PCP in their network (a method will be developed to address PCPs that are in more than one PPS). For all population subcategories, if the beneficiary cannot be matched by PCP, then the beneficiary will be assigned to the PPS with the most beneficiaries already assigned (by the visit attribution method) in their specific zip code or other relevant geographic area. Except for beneficiaries who are explicitly excluded because they receive the majority of their services (more than 50%) at providers that are not participating in DSRIP, all beneficiaries will be attributed.

Attribution for Performance Measurement Purposes
- Although the patient populations targeted for PPS measurement are determined as of January 1 (or other date specified) of the measurement year for valuation purposes, patient attribution for PPS quality measurement for domain 2 and 3 metrics will be defined as of the measurement period. This is consistent with the CMS Medicare Shared Savings Program (MSSP), where there is an initial, prospective attribution at the start of the measurement year to determine the populations to be included and a final attribution at the end of the year for evaluation and measurement. Each patient will be assigned to only one PPS for measurement purposes. The patient population attributed for performance measurement will form the basis for quality measurement for all population-based measures (see Measure Specification and Reporting manual) with the appropriate criteria applied for each measure. For episodic-based measures (see Measure Specification and Reporting manual), the initial population attributed to each PPS will be limited to only those members seen for that episode of care within the PPS network during the measurement period. Episode of care refers here to all care provided over a period of time (as defined in the measurement specifications) for a specific condition (e.g. Diabetes - all diabetes care received in a defined time period for those members; HIV- all HIV care received in a defined time period for those members). Since PPS networks are non-binding and members can choose to receive care outside of network, it is necessary to protect patient confidentiality for certain highly sensitive medical conditions, as well as, ensure medical records are available to the PPS network for all hybrid measures. For institutional-based measures (see Measure Specification and Reporting manual) the population for quality measurement will represent the population within that facility.
Minimum Outpatient Service Level
- PPSs must have a minimum of 5,000 attributed Medicaid beneficiaries a year in outpatient settings.
Performing Provider System Relation to IGT Entities
- Intergovernmental transfer (IGT) entities are entities that are eligible to contribute allowable governmental funds or other non-federal funds for use by the state for the non-federal share of DSRIP payments for a PPS. They include government-owned Hospitals and other government entities such as counties.
- The non-federal share of DSRIP payments to providers will be funded through the use of intergovernmental transfers (IGTs) from government owned or operated major public hospitals or their sponsoring government entity or other affiliated governmental entity, consistent with applicable federal law and regulations. Such IGTs will not be represented on any financial statement by the public hospital as a cost of patient care, overhead, tax, or administrative cost; instead it shall be reflected as a transfer to the state government. For purposes of this section, the major public hospital or their affiliated government or governmental entity are deemed to be one and the same.
- No portion of a DSRIP payment paid to a private PPS may be redirected to the public entity that is supplying IGTs to finance the non-federal share of such payments. Also, no private provider that is included in a coalition of providers that includes public providers can transfer DSRIP funds to those public providers for the purposes of funding the non-federal share of the administrative activities. Nothing herein precludes or restricts such private provider from making payments to a public provider for services performed or provided by the public provider including DSRIP related services.
- The state encourages public and private providers to collaborate where appropriate and will work with PPSs to clarify the flow of IGT funding to avoid impermissible provider donations.

III. Projects, Metrics, and Metric Targets

Projects
- PPSs will design and implement at least five and no more than eleven DSRIP projects, selected from the Strategies Menu and Metrics (Attachment J). Each project will be based on a particular strategy from Attachment J and will be developed to be responsive to community needs and the goal of system transformation, as defined by the objectives in STC 6 in section VII.
- All the DSRIP projects for a PPS will be part of the PPS´s overall DSRIP Project Plan.
- There are projects described in Attachment J that are grouped into different strategies, such as behavioral health, within each Domain (System Transformation Projects (Domain 2), Clinical Improvement Projects (Domain 3), and Population-wide Projects (Domain 4). For each strategy, there is a set of metrics that the PPS will be responsible for if they do any one of the projects within that strategy.
- As described in Attachment J, PPSs will select at least two system transformation projects (including one project to create integrated delivery systems as well as another project from either the care coordination or connecting settings strategies list), two clinical improvement projects (including a behavioral health project), and one population-wide project. The selection of all projects must be based on the community needs assessment of the baseline data and the target population selected by the PPS. PPSs may choose additional projects as appropriate.
Metrics
- In order to measure progress towards achieving each objective, each project must include metrics in all four of the following domains. PPSs will report on these metrics in their semi- annual reports (described in VII.a below) and will receive DSRIP payment for achievement of these milestones (based on the mechanism described in VII.a below).
  1. Overall project progress metrics (Domain 1)
  2. System transformation metrics (Domain 2)
  3. Clinical improvement metrics (Domain 3)
  4. Population-wide metrics (Domain 4)
PPSs that exceed their metrics and achieve high performance by exceeding a preset higher benchmark for reducing avoidable hospitalizations or for meeting certain higher performance targets for their assigned behavioral health population will be eligible for additional DSRIP funds from the high-performance fund, described in paragraph VIII.c. below.

The Strategies Menu and Metrics (Attachment J) describes the specific metrics that will be used to assess performance under each domain and specifies which metrics are pay-for-reporting and which are pay-for-performance. Additional measure specifications, including the process for addressing small number issues is described in the Measure Specification and Reporting Manual supplement to Attachment J. NYS DOH, with CMS approval, will develop a procedure for replacing measures which are retired by the measure steward due to issues, such as guideline changes. Additionally, the procedure will address methods for handling specification changes which affect the ability to trend from previous results.

As described in STC 12.e. in section VIII the state or CMS may add domain 1 metrics to a project prospectively in order to address implementation concerns with at-risk projects.
Metric Targets
- All PPSs must have a target for all pay-for-performance metrics, which will be used to determine whether or not the performance target for the metric was achieved. Statewide performance targets should be based on the top decile of performance for state or national data, or an alternative method approved by CMS. NY DSRIP goals for metrics may be based on NYS Medicaid results (preferred source) or national data where possible and on DSRIP DY1 results for metrics where state or national data are unavailable.
  
  Annual improvement targets for PPS metrics will be established using the methodology of reducing the gap to the goal by 10%. The PPS baseline data will be established as soon as complete data is available for the baseline period with the necessary minimum thresholds met (as specified in the Measure Specification and Reporting manual) and will be used as the foundation to determine the gap to goal to set the annual improvement target.
  
  For example, if the baseline data for a measure is 52 percent and the goal is 90 percent, the gap to the goal is 38. The target for the project´s first year of performance would be 3.8 percent increase in the result (target 55.8 percent). Each subsequent year would continue to be set with a target using the most recent year´s data. For example, should a PPS meet or exceed the first year´s target of 55.8 percent, the next annual target would be 10 percent of the new gap to the goal. This will account for smaller gains in subsequent years as performance improves toward the goal or measurement ceiling.
  
  The PPS will know the annual performance target to be achieved at the beginning of the current DY and the method for determining the annual performance target will remain the same throughout the DSRIP years.
  
  In general, a PPS that achieve their target for the DY will be considered to have reached the annual milestone for the metric, and a PPS that achieve 20 percent gap to goal or the statewide performance goal for the high-performance metrics listed in Attachment J may be eligible for additional payment for high performance. If more frequent reporting (more than annual) of metric results are required for projects, the reported results for payment should be based on a standard twelve-month period.

IV. DSRIP Project Plan Requirements

Project Plan Development Process
- The proposed project plans should be developed in collaboration with community stakeholders and responsive to community needs. PPSs have the option to seek DSRIP design grants described in STC 10.a in section VIII.
- According to a timeline developed by the state and CMS that aligns with the DSRIP deliverables schedule outlined in STC 1 in section VIII PPSs must submit a final DSRIP Project Plan to the state for review with a complete budget and all other items described below, consistent with the requirements in STC 7 in section VII.
- It is expected that the transformational nature of the activities to be undertaken in these projects will require a strict adherence to disciplined project management. The DSRIP Project Plan must provide evidence that the PPS has a clear understanding of the needs of the service area (based on objective data specific to the service area as well as community input), that the project will address these needs in a significant manner, that the PPS understands the metrics that will need to be monitored and the methodology that will be used to do such, and that the PPS has internal and/or external resources that will be available for project management and the required rapid cycle improvements inherently needed in these projects.
Organization of DSRIP Project Plan
- DSRIP Project Plans must be submitted in a structured format agreed upon by the state and CMS. At a minimum, the plan shall include the following sections:
  1. DSRIP Face Sheet
    
    This face sheet will list the documents included within the package and include the applicant´s name and a brief (no more than 1000 word) executive summary of the submitted project.
  2. Provider Demographics
    1. Name, address, senior level person responsible for the DSRIP project and to whom all correspondence should be addressed.
    2. The name of providers and their identification numbers participating in the project plan, including the lead provider in the case of a coalition.
    3. Definition of service area (according to the specifications in the DSRIP Strategies Menu and Metrics) and a discussion of how the providers in the coalition relate to (or inform) the service area definition. As further described in the DSRIP Strategies Menu and Metrics, PPSs are accountable for improving the quality of care for all Medicaid and low-income uninsured beneficiaries in their service area as defined in the DSRIP Member Attribution Method above.
    4. Identification as a safety net provider with documentation supporting that identification as described in paragraph II.a above.
    5. Current patient population including demographic information, payer mix to document qualification as described in paragraph II.c above.
  3. Identification of Provider Overarching Goals
    - The PPS will need to identify its goals for the project, as well as how the project contributes to achieving the overall goals (defined in STC 6 in section VII) to create and sustain an integrated, high performing health care delivery system that can effectively and efficiently meet the needs of Medicaid beneficiaries in their local communities by improving care, improving health and reducing costs. More specifically, the PPS should demonstrate how the project will engage in system transformation (including linking across settings, ensuring appropriate capacity, and taking responsibility for a population), as demonstrated by achievement of avoidable events [including addressing behavioral health]. The PPS will need to demonstrate that it has a governance strategy that ensures that participating providers work together as a "system" and not as a series of loosely aligned providers nominally committed to the same goal. Plans to progressively move from a loosely organized network of affiliated entities to an actual Integrated Delivery System must be evident in the goals.
    - The PPS will need to provide objective data-driven evidence that this is a relevant goal for the PPS and its service area. The PPS must demonstrate that all relevant Domain 3 metrics for the projects selected align with community needs and that these areas have room for improvement. With the exception of behavioral health Domain 3 measures, for which the following will not apply, if the PPS´s performance on the most recent available data (as specified in the Metric Specification Guide supplement to Attachment J) for the majority of any chosen Domain 3 metric set is within 10 percentage points or 1.5 standard deviations to the high-performance goal described in section III.c above (whichever is greater) the project would not be approved.
  4. Identification of Provider Project to meet identified goals
    - Includes a brief rationale for project choice and summary (including citations) of existing evidence showing that project can lead to improvement on goals of project. Logic models such as driver diagrams may be helpful to demonstrate how the elements of the project all contribute to the central goals. Further information will be provided in the detailed assessment provided in (5) and must include all relevant domains outlined in the Strategies Menu and Metrics.
  5. Performance Assessment
    1. Current community health needs (population demographics, types and numbers of providers and services, cost profile, designation as Health Professional Shortage Area, mortality and morbidity statistics, and health disparities). Population demographics reflecting community health needs should include those who are institutionalized as well as those involved in the criminal justice systems. The assessment will also include a discussion of a designated list of public health concerns determined by the state, including behavioral health. The selection of these concerns should be supported by baseline data on current performance on targeted health indicators and quality metrics. A review of the social determinants of health and assessment of disabilities consistent with Olmstead should be included in such a community assessment as well. Needs assessment should include a review of non-English languages spoken in the PPS catchment area to guide development of multi-language outreach and education material.
    2. Evidence of regional planning including names of partners involved in the proposed project (in addition to any coalition members in the PPS in accordance with the process described in paragraph II.b above). The assessment will also include a detailed analysis of issues causing poor performance in the project area including but not limited to patient co-morbidities, patient characteristics, social system support, system capacity for primary care and disease management, and institutional issues such as finances, confounders to health care system improvement including fragmentation of services, competition, and assessment of regional planning issues.
    3. Comprehensive workforce strategy - this strategy will identify all workforce implications - including employment levels, wages and benefits, and distribution of skills - and present a plan for how workers will be trained and deployed to meet patient needs in the new delivery system based on the performance assessment of community health needs, and how the strengths of current workforce will be leveraged to the maximum degree possible under current state law and regulations.
    4. Review of Financial stability - A complete review of the financial condition of the PPS Lead provider, including a review of financial records and a narrative on the PPS plan to monitor the financial sustainability of all financially challenged safety net and public providers in the PPS.
    5. Evidence of public input into the project including consumer engagement. This should include documentation of collaboration with local departments of public health, public stakeholders and consumers. In addition, the provider will need to document how there will be ongoing engagement with the community stakeholders, including active participation in any regional health planning activities currently underway in their community. Applicants will need to include workers and their representatives in the planning and implementation of their overall project with particular emphasis on the comprehensive workforce strategy. The PPS must specifically include evidence of consumer engagement in their needs assessment and planning process. The state may require PPSs to maintain a website including contact information, overview of public comment opportunities, results of public processes, application materials, and required reporting.
  6. Work Plan Development
    - In this section the provider will provide an initial high-level work plan in a state- approved format using the domains of milestones identified in the DSRIP Strategies Menu and Metrics.
      1. Project progress milestones (Domain 1)
      2. System transformation and financial stability milestones (Domain 2)
      3. Clinical improvement milestones (Domain 3)
      4. Population-wide Milestones (Domain 4)
      The PPS will need to document their plans to address and implement the project including each of the confounders identified in the Performance Assessment section. This should include resources available to complete the project. The time frame for the work plan will be five years. It is expected that no more than the first two years will be utilized to implement major system changes related to the project. In addition, it is expected that improvements in outcome metrics will begin to occur in that first two-year period.
  7. Rapid cycle evaluation
    - The plan must include an approach to rapid cycle evaluation that informs the system of progress in a timely fashion, and how that information will be consumed by the system to drive transformation and who will be accountable for results, including the organizational structure and process to oversee and manage this process. The plan must also indicate how it will tie into the state´s requirement to report to CMS on a rapid cycle basis.
  8. Establishment of Milestones and Metrics
    - A section of the work plan must provide documentation of the monitoring strategy for the project including significant milestones and associated metrics, as specified in the DSRIP Strategies Menu and Metrics.
  9. Budget
    - PPSs must provide a detailed budget for all 5 years of their DSRIP project. For PPSs that were awarded HEAL grants, a detailed budget report along with a description of the similarities or differences must be included.
  10. Governance
    - The plan must include a detailed description of how the system will be governed and how it will evolve into a highly effective Integrated Delivery System. A clear corporate structure will be necessary and all providers that participate in the project will need to commit to the project for the life of the waiver. Weak governance plans that do not demonstrate a strong commitment to the success of the project will be rejected. Strong centralized project control will be encouraged especially for projects that require the greatest degree of transformation. The governance model should review the PPS´s need to pursue any state certificate of public advantage (COPA) and Accountable Care Organization (ACO) opportunities. Coalitions must define the members of the coalition and submit all supporting information about coalition governance including the business relationship, as described in Section II.b. The governance plan must address how the PPS proposes to address the management of lower performing members within the PPS network. This plan must include progressive sanctions prior to any action to remove a member from the PPS. The governance plan must also include a process by which the PPS will progressively advance from a group of affiliated providers to a high performing Integrated Delivery System. The state may provide governance template information for PPSs to utilize in the development of their governance models and plans.
  11. Data sharing and confidentiality
    - Metrics will be collected in a uniform and valid fashion across all members of a PPS. The plan must include provisions for appropriate data sharing arrangements that permit this and appropriately address all privacy protections contained in federal law including HIPAA and New York Law.
  12. Expectation of Sustainability
    - PPSs are asked to explain how the outcomes of this project will be sustained at the end of DSRIP and how gains can be continued after the conclusion of the project period.
    - This should include a financial forecast of expected savings related to the implementation.
  13. Legal Compliance
    - PPSs must comply with all relevant laws and regulations including compliance with Civil Rights Law and specifically all laws governing non-discrimination.
  14. Signed Attestations
    - The PPS will submit a description of any initiatives that the provider is participating in that are funded by the U.S. Department of Health and Human Services and any other relevant delivery system reform initiative currently in place. The PPS will, by signature, attest that the submitted DSRIP project is not a duplication of a project from these other funded projects and does not duplicate the deliverables required by the former project(s). It should be noted if this project is built on one of these other projects or represents an enhancement of such a project that may be permissible, but it must be clearly identified as such in the DSRIP project plan.
    - The provider will submit an attestation statement documenting that the information provided in this document is accurate at the time of submission and that the provider, if accepted into the DSRIP, will cooperate fully with the state in the implementation and monitoring of this project and participate in the required learning collaboratives related to this project.
    - If the PPS is receiving funds from the Public DSRIP pool it will also provide a description of the IGT source identified for the project and attest that this IGT derives from local, public funds.
- c. 1115 Waiver Managed Care Programs and Funds Flow Mechanism
  
  The New York 1115 Medicaid Waiver provides $8 billion in Federal funding over a five-year period for DSRIP and its related programs. Of this amount, the State will utilize the $2 billion in FFP for DSHPs to partially fund the safety net. The non-federal share of any Public PPS payments will come from IGTs supplied by the public hospital or its affiliated governmental entity.
  1. 1115 Waiver Managed Care Programs Overview
    
    Three Programs make up the portion of the 1115 Waiver designed to be administrated through Managed Care, as described below.
    1. Managed Long Term Care (MLTC) Workforce:
      
      The MLTC Workforce Program is designed to enhance the workforce of the New York State´s (hereinafter, "the State´s" or "NYS´s") long-term care services to ensure that the workforce is ready for the greater emphasis on non-institutional care in the State´s healthcare system once the Delivery System Reform Incentive Program (DSRIP) is complete in 2020. This initiative will target direct care workers by allowing MLTC plans to invest in workforce training for providers in their network. The goal is to support the critical long-term health care workforce infrastructure, prepare new long-term care workers and build new skills to address changes in the field.
      
      Through the workforce program investment, the State will require MLTC plans, which include Fully Integrated Dual Advantage (FIDA) plans (collectively MLTC/FIDA plans) to contract with NYS Department of Health (DOH)- designated Workforce Centers of Excellence (WCE), to:
      1. invest in initiatives to attract, recruit and retain long-term care workers in the areas they serve;
      2. develop plans to address reductions in health disparities by focusing on the placement of long-term care workers in medically underserved communities;
      3. train needed workers to care for currently uninsured populations who will seek care under the Affordable Care Act expansion; and
      4. support the expansion of home care and respite care, enabling those in need of long-term care to remain in their homes and communities and reduce New York´s Medicaid costs associated with long-term care.
      To implement this program, the NYS DOH will establish a list of WCEs across the state through a qualification process. All WCEs which meet the minimum criteria will be qualified and a comprehensive list will be made available directly to MLTC/FIDA plans to contract with. The State will not contract directly with the designated WCEs but will retain the ability to revoke the designation of any WCE that ceases meeting the minimum criteria. The state shall designate WCEs through an application process. The state shall post WCE qualifications and application for public comment for at least 14 days. An application period of at least 14 days shall begin once the public comment period closes. The state shall post the list of approved WCEs to its website. Each WCE will offer a menu of Workforce Development Initiatives (WDI) which provide training and support recruitment and retention efforts in a way that addresses the needs of plans, providers and workers in the long-term care context.
      
      WDIs may be newly developed or based on existing programs but must support the State´s ongoing efforts to transform the health care delivery system and improve population health. Further, WDIs should be consistent with and complementary to other state workforce development efforts, including those being carried out by PPSs under the DSRIP program.
      
      The State will evaluate the MLTC workforce program through quarterly reports provided by MLTC plans. Plans will be required to report on stakeholder engagement activities to set the menu of trainings for each region, numbers of trainings offered and enrollees, and allocation of funds.
    2. 1915(i) Home and Community Based Services (HCBS)
      
      The HCBS Program is designed to provide opportunities for adult Medicaid beneficiaries with mental illnesses and/or substance use disorders to receive the healthcare and social services they require in their own home or within their community. The implementation of the HCBS program will help to create an environment where Managed Care Plans, service providers, families, and government partners can work in tandem to help eligible beneficiaries in order to prevent and manage chronic health conditions and recover from serious mental illness and substance disorders. The HCBS funds have been made available over five years in order to provide care for chronic health and behavioral conditions outside of institutional settings, through a number of initiatives.
      
      The primary initiative in the HCBS Program is the Health and Recovery Plan (HARP), which was developed to promote significant improvements in the Behavioral Health System as the State moves into a recovery-based Managed Care delivery model. The HARP model of care emphasizes and supports a person´s potential for recovery by optimizing quality of life and reducing symptoms of mental illness and substance disorders through empowerment, choice, treatment, education, employment, housing, and health and well-being goals.
      
      The goals of the program, per the Behavioral Health Evaluation Plan, are to improve health, behavioral health and social functioning outcomes for HARP enrollees and to develop BH HCBS focused on recovery, social functioning, and community integration for individuals in HARPs. To evaluate these goals, the State will be examining HARP enrollees´ access to BH HCBS and other integrated services, including primary care; the extent to which HARP enrollees´ quality of care, satisfaction of care, and overall care experiences are improving; the extent that HARPs are cost effective (i.e. decreasing use of more expensive inpatient services through an increase in services offered in the homes and communities of enrollees); etc. Long term outcomes that are expected by the State as a result of implementing BH HCBS for HARP enrollees include, but are not limited to, the following:
      1. Improved access to HCBS
      2. Improved social, functioning and recovery outcomes
      3. Improved or consistent high level of satisfaction with consumer experience with care
        
        The State will evaluate these goals through oversight of HARPs and through various quantitative and qualitative analyses using the following data sources: Consumer Assessment of Healthcare Providers and Systems (CAHPS®) and HARP Perception of Care survey results, HEDIS/QARR metrics, Community Mental Health Assessment (CMHA) results, NY Medicaid Choice Enrollment Data, ONTrackNY Data System for tracking First Episode Psychosis treatment, Medicaid Managed Care HCBS Provider Network Data System, and Medicaid claims.
      4. Care Restructuring Enhancement Pilots (CREP) Program
        
        The CREPs Program is intended to assist select hospitals throughout the State in expanding upon and accelerating the programmatic goals of the two other 1115 Waiver Managed Care Programs, MLTC Workforce and HCBS, through the form of targeted pilot programs. Waiver funds have been made available over four years for the development of two pilots in CREPs that are designed to be succinct, distinct and targeted extensions of the base Managed Care Programs.
        
        The two pilots will focus on the hospital and its service area´s ability to work with projected MLTC and HCBS populations during the care transition of patients from either the hospital to their home or to lower levels of care. This information will be compared to the needs of the designated populations. Once the service gaps have been determined, hospitals will work to train the workforce in developing the skills necessary to address the needs of each pilot´s respective populations both within and outside of institutional settings
        
        The State will oversee the pilots through semi-annual progress reports submitted by the facility and semi-annual payment summaries submitted by the Managed Care Organizations (MCOs). The pilots, once complete, will be used to inform the State on how to successfully transition hospital resources to gainful employment in long-term care and community-based care settings. This will be a significant development, as demand for labor in hospitals is expected to decline following the innovations in the provision of care that DSRIP will cause throughout the State.
        
        The goal of the CREPs Program is to test the effectiveness of different strategies for retraining the hospital workforce for gainful employment in non-acute settings, specifically for MLTC and HCBS. As stated above, the training will be administrated by the facility itself to its own workforce. This program goal will be measured by assessing the pilots to evaluate the success of the varied strategies implemented by the participating facilities. The evaluation will be comprised of semi-annual reports submitted by facilities on their progress. Additionally, once the training begins, surveys will be completed by the hospital workforce on their satisfaction with various aspects of the curriculum, and the facility will be expected to revise the training based on these responses. Furthermore, facilities will measure the number of staff that successfully complete the course and receive a training certificate. Facilities must report all gathered data to the State, and the pilots´ effectiveness will be evaluated based on these results.
        
        Funds Flow Mechanism for Managed Care Programming
        
        The Special Terms and Conditions (STCs) of the Waiver stipulate that the Federal Financial Participation (FFP) funding ($2B) is available from Designated State Health Programs (DSHP) to support DSRIP. Since the non-DSRIP programs are not authorized to use the DSHP FFP, the only funding sources for these programs are the Intergovernmental Transfers (IGTs).
        
        For CREPs and MLTC Workforce, the State can distribute funds as lump sum payments to the participating Managed Care Plans utilizing IGTs via manual offline transactions, outside of the eMedNY system. This method aligns with the State´s current method of distributing PPS performance payments in DSRIP and Health Homes through IGT. The use of a lump-sum payment method for the 1115 Waiver Managed Care Programs will also allow for efficient distribution and monitoring of awarded funds in MLTC Workforce and CREPs.
        
        The HCBS Program is already being partially implemented utilizing CMS approved rate add-ons and will likely continue to do so for the balance of the program. For this Program, the State will report the federal match as federal IGT funding and utilize the non-Federal share of the safety net IGT to offset the corresponding non-Federal match occurring in the State´s general fund.
        
        For the programs that will be funded through rate add-ons, there will be no additional claims submitted; instead, the only change will be regarding reporting the source of the Federal match as IGT funded through DSRIP authority. For the programs that will be funded through a lump sum payment model, the claims will be submitted as they are in DSRIP: offline transactions will occur in the state financial system that will then create a payment voucher.
        
        To ensure that there is no duplication of payment in the rate add-on payment method, exclusion codes would be used to mark the populations involved in each program, which would limit the amount that each rate could be used to bill for. In addition, annual caps on would be placed on each rate code.
        
        To ensure that there is no duplication of payment in the lump sum payment method, awards would only be distributed based on performance on a reported deliverable. Reports, and subsequently payments, would only be completed semi-annually, and each would be based on a distinct deliverable or performance milestone different from prior years.

V. Project Valuation

The DSRIP project and application valuations will be calculated by the state (with assistance from the independent assessor) according to the methodology described below.

A maximum valuation for each DSRIP application is calculated based on the formula described in Section A below. Once the overall application value is determined, the value for the individual metrics of the DSRIP project plan is determined based on the distribution method described in Section B below. Project values are subject to monitoring by the state and CMS, as described in Section C below, and PPSs may receive less than valuation described in their DSRIP plan if they do not meet metrics and/or if DSRIP funding is reduced because of the statewide penalty (described in Section IX.d below).

As noted, PPSs are to submit a Project Plan with a minimum of 5 projects and (in most cases) a maximum of 10 projects for scoring purposes. In certain instance, a PPS may be eligible to pursue a project plan containing 11 projects. Please see below for project selection requirements per domain.

Domain 2 Projects - Applicants must select at least two projects from this domain (one of which must be from sub-list A and one of which must be from sub-list B or C) but can submit up to 4 projects from Domain 2 for scoring purposes
- For eligible PPSs pursuing 11 projects in their plan, they are allowed to select up to 5 projects from Domain 2 for scoring purposes. Domain 2 project selection must follow the same requirements listed above, but in these instances, there will also be the option for these systems to add project 2.d.i.
Domain 3- Applicants must select at least two projects from this domain (one of which must be A. Behavioral Health), but can submit up to 4 projects from Domain 3 for scoring purposes

Domain 4 - Applicants must select at least one project from this domain, but can submit up to 2 projects from Domain 4 for scoring purposes

Valuation for DSRIP Application

The maximum DSRIP project and application valuation will follow a five-step process.

Step 1 assigns each project in the Strategy Menu (Attachment J) a project index score which is a ratio out of a total of 60 possible points of each project (X/60 = project index score).
Step 2 creates a project PMPM by multiplying the project index score by the state´s valuation benchmark. The valuation benchmark is pre-set by the state and varies based upon the number of projects proposed by an applicant.
Step 3 determines the plan application score for the PPS´s application based on a total of 100 points possible for each application (X/100 = Application Score).
Step 4 calculates the maximum project value by multiplying the project PMPM, the plan application score the number of beneficiaries attributed to the project, and the duration of the DSRIP project (see example below).
Step 5 calculates the maximum application value for a PPS once the maximum project values have been determined, by adding together each of the maximum project values for a given PPS´s application.

The maximum application value represents the highest possible financial allocation a PPS can receive for their Project Plan over the duration of their participation in the DSRIP program.

PPSs may receive less than their maximum allocation if they do not meet metrics and/ or if DSRIP funding is reduced because of the statewide penalty (described in Section IX below).

Step 1: Calculating Project Index Score

The value of a single project is expressed as an index score (see below). Project index scores are based upon a grading rubric that evaluated the project´s ability to transform the health care system. The State has assigned an index score to each project based on the grading rubric and the given project´s relative value to the other projects in the state´s menu. For application planning, index scores for each project are available to providers in the DSRIP Project Toolkit. These values have been made available to applicants in advance for their application submission.

The formula for the index score for each project on the menu consists of the following elements:

Potential for achieving DSRIP goal of system transformation, including the three objectives, as described in STC 6 in section VII (Score 1 (lowest) - 30 (highest))
Potential for achieving DSRIP goal of reducing preventable events, as described in STC 1(a) in section VII (Score 1- 10)
Scope of project and capacity of project to directly affect Medicaid and uninsured population (Score 1-10)
Potential Cost Savings to the Medicaid Program (Score 1-5)
Robustness of evidence base (Score 1-5)

Adding up the scores for each element for a given project will give each project an index score of X/60. The project index score (out of the 60 possible points) will be expressed in decimal form for calculation purposes.

Step 2: Calculating Project PMPM

Each project will be assigned a valuation benchmark based on the number of projects proposed in the application as described in Table 1 below. By no later than 15 days after the public comment period for initial DSRIP applications, the state will establish a state-wide valuation benchmark based on its assessment of the cost of similar delivery reforms. This value will be expressed in a per-member per-month (PMPM) format and may not exceed $15 PMPM, as described in STC 9 in section VII.

For the purpose of the example described later in this section, an initial $3.35 PMPM valuation benchmark is used in Table 1 below. Because projects serving more beneficiaries will have synergistic properties and economies of scale that will lower a project´s per member per month cost, the final valuation benchmark will be set based on the overall scope of applications received. Table 1 (below) reflects the final valuation benchmark developed by the state in accordance with CMS´s guidelines.

Because additional projects will have synergistic properties, from leveraging shared infrastructure and resources, the valuation benchmark is discounted as follows for PPSs selecting multiple projects. Although the project PMPM levels drop with the inclusion of additional projects, the overall Performing Project System valuation will generally increase (depending on the value of the actual projects selected) as more projects are added to the overall PPS effort. If the valuation benchmark is adjusted based on the process described above, the relative discount factor for additional projects beyond will remain the same.

Table 1 - Valuation benchmark table (Statewide valuation benchmark: $3.35)

Number of projects	Valuation Benchmark Multiplication Factor	Final Valuation Benchmark
7	1	$3.35
8	0.9697	$3.25
9	0.969699	$3.25
10	0.969698	$3.25
11 (only for eligible PPS)	0.969697	$3.25

The valuation benchmark is then multiplied by the project index score to create a project PMPM for each project.

Step 3: Plan Application Score

Based on their submitted application, each project plan will receive a score based on the fidelity to the project description and likelihood of achieving improvement by using that project. This project plan application score will be used as a variable in calculating the maximum project value. Each project plan application score will be expressed as a score out of 100, which will drive the percent of the maximum project valuation for each project that will be allocated to that individual project plan. The plan application score (out of 100) will be expressed in decimal form when calculating the maximum project valuation. The state will develop a rubric for the individual project plan application score in collaboration with CMS. This rubric must include an assessment of whether each proposed project is sufficiently different from other DSRIP projects selected other existing projects being funded by other sources, so as to ensure that the PPS does not receive double-credit for performing similar activities.

PPSs eligible for and approved to deliver project 2.d.i ("the 11th project") will be awarded an application bonus to reflect the extra effort needed to address the project´s target population. The total Project Plan including the 2.d.i project application bonus points, cannot exceed 100 points. (i.e.: If a project score, before the addition of the bonus points, is a 95 - then the maximum bonus added to that project score would be 5 points for a total score of 100).

Applications will also be scored based on an applicant´s commitment to developing a capability to responsibly receive risk-based payments from managed care plans through the DSRIP project period.

Step 4: Calculating Maximum Project Value

The number of beneficiaries attributed to the project (based on the attribution method described in Section II above) and the anticipated duration (expressed in months) of the applicant´s participation in DSRIP program will also be used to calculate the maximum value for each project as follows.

Maximum Project Value = [Project PMPM] × [# of Beneficiaries] × [Plan Application Score] × [DSRIP Project Duration]

Step 5: Calculating Maximum Application Value

Once the Maximum Project Value for each of the projects in the PPS application is calculated, the maximum project values for each of the project are then added together to provide the Maximum Application Value for the DSRIP application.

Example: Putting it all together - Below is a simple example of the DSRIP valuation calculation

For illustration purposes, a PPS submits six projects in their application. Two projects are from Domain Two; Creating and integrated Delivery System, and Expand Access to Primary Care, and three projects from Domain Three; Integration of Behavioral Health in Primary care, Development of Evidence Based Medicine Adherence programs, and HIV Services Transformation; and one project from Domain Four: Evidence Based Strategies to Prevent Substance Abuse and Other Mental/Behavioral Disorders. Scoring steps are included below but all numbers are for illustration purposes only and do not reflect on the actual values that the example projects will receive.

Step 1: Calculating Project Index Scores (for illustration purposes) Project Index Scores

Project 1: Creating an Integrated Delivery System 56÷60=.93
Project 2: Create a Medical Village (Hospital) 54÷60=.9
Project 3: Integration of Behavioral Health in Primary care 39÷/60=.65
Project 4: Evidence Based Medicine Adherence 29÷60=.48
Project 5: HIV Services Transformation 28÷60=.47
Project 6: Strategies to Prevent SUD and BH Disorders 20÷60=.33

Step 2: Calculating Project PMPM (numbers below are for illustration only)

Since there are six projects in this example application, the valuation benchmark is $7.20 (for a six-project application - from the table in step 2 above). Each of the Project Index Scores (from Step 1) are then multiplied by Valuation Benchmark to compute the individual Project PMPMs.

[Project Index Score] X [Valuation Benchmark] = Project PMPM (see table below)

Table 2 - Project Index Score Example

	Project Index Score	Valuation Benchmark	Project PMPM
Project 1	0.93	7.2	$6.70
Project 2	0.9	7.2	$6.48
Project 3	0.65	7.2	$4.68
Project 4	0.48	7.2	$3.46
Project 5	0.47	7.2	$3.38
Project 6	0..33	7.2	$2.38

Step 3: Calculating Plan Application Score

A PPS submits six project PPS application and receives a plan application score of 85/100. As part of the 15-point reduction from a perfect score, the PPS received a reduction because the PPS selected two projects that share the same metric set.

Step 4 and 5: Calculating Maximum Project Value and Maximum Application Valuation

The attribution assessment completed by the provider in their application (and subsequently verified by the State´s attribution method and independent assessors) shows 100,000 beneficiaries are expected to be served by the applicant´s DSRIP. As a result, the maximum application value is calculated as $138,108,000 as illustrated below.

Table 34 - Maximum Project Valuation Example

Project	Project PMPM	# of Beneficiaries	Overall Application Score for Project	# of DSRIP Months	Maximum Project Value
Project 1	$6.70	100000	0.85	60	$34,170,000
Project 2	$6.48	100000	0.85	60	$33,048,000
Project 3	$4.68	100000	0.85	60	$23,868,000
Project 4	$3.46	100000	0.85	60	$17,646,000
Project 5	$3.38	100000	0.85	60	$17,238,000
Project 6	$2.38	100000	0.85	60	$12,138,000
		Maximum Application Valuation			$138,108,000

Metric valuation

Once the overall project valuation is set, the value for each metric is calculated through the process described below.

Step 1. Calculate Annual Percentage of DSRIP Performance Payment

To determine annual payments, it is first necessary to identify the percentage of total DSRIP funding for each year. STC 13(d) in Section VII of the STCs indicates the maximum DSRIP performance payment for each year of the demonstration as well as the combined total for all years of the DSRIP. Annual payments percentages are calculated by dividing the allowable payments per year by the total performance payment pool for all DSRIP years, as illustrated in Table 5 below.

Table 5: Annual Performance Award Percentages

	DSRIP Year 1	DSRIP Year 2	DSRIP Year 3	DSRIP Year 4	DSRIP Year 5	Total
Total DSRIP Performance Award	$981,800,000	1144300000	1668400000	1379500000	874700000	$6,048,700,000.00
	16.23%	18.92%	27.58%	22.81%	14.46%	100

Step 2. Calculate the Annual Potential Performance Payment for Each Project

As described in Section VII.6 of the STCs and in Attachment J, each year performance payments are distributed across four domains, each with corresponding projects (and within domains 2 and 3, across pay-for-performance (P4P) and pay-for-reporting (P4R) metrics) as specified in tables 6, 7, and 8 below. Note that Project Progress Milestones (Domain 1) assess overall implementation of organizational and project specific milestones. As such, Domain 1 funds will be calculated as part of the total valuation for each project selected by the PPS.

To calculate potential payments by year:

Multiply project valuation by the corresponding annual percentage, as noted in table 5 above;
Multiply the resulting amount by the corresponding domain in Tables 6,7, and 8 below (for example, in year 2, the funding for a domain 2 project would be split with 60% of the funds tied to domain 1 milestones and 40% tied to domain 2 P4R measures).

Table 6: Domain 2 Distributions

Metric/Milestone Domains	Performance Payment	Year 1	Year 2	Year 3	Year 4	Year 5
Project Progress Milestones (Domain 1)	P4R/P4P	0.8	0.6	0.4	0.2	0

System Transformation and Financial Stability Milestones (Domain 2)	P4P	0	0	0.5	0.72	0.93
	P4R	0.2	0.4	0.1	0.08	0.07

* P4P is pay for performance; P4R is pay for reporting.

Table 7: Domain 3 Distributions

Metric/Milestone Domains	Performance Payment	Year 1	Year 2	Year 3	Year 4	Year 5
Project Progress Milestones (Domain 1)	P4R/P4P	0.8	0.6	0.4	0.2	0
Clinical Improvement Milestones (Domain 3)	P4P	0	0.3	0.5	0.7	0.9
Clinical Improvement Milestones (Domain 3)	P4R	0.2	0.1	0.1	0.1	0.1

Table 8: Domain 4 Distributions

Metric/Milestone Domains	Performance Payment	Year 1	Year 2	Year 3	Year 4	Year 5
Project Progress Milestones (Domain 1)	P4R/P4P	0.8	0.6	0.4	0.2	0
Population Health Outcome Milestones (Domain 4)	P4R	0.2	0.4	0.6	0.8	1

Steps 1 through 2 above described the process to determine the potential annual performance payments for each domain and P4P and P4R grouping. Actual payments are based on PPS meeting performance metrics, or Achievement Values, as described in steps 3-5 below.

Step 3. Calculate Total Achievement Values

If a performance target or reporting milestone is met², the Performing Provider System will receive an AV of 1 for that performance target/milestone in that reporting period.³ If the Performing Provider System does not meet its milestone or performance target, the Performing Provider System will receive an AV of 0 for that reporting period. This will be done across every project in every domain.

Within each project, AVs are then grouped into either a pay-for-reporting (P4R) or pay- for- performance (P4P). The Total Achievement Values (TAV) are identified for each project within the appropriate category. For example, a domain 2 project may have possible AVs for domain 1 milestones as well as for domain 2 P4R measures and domain 2 P4P measures so that in year 3, a domain 2 project may have 6 possible domain 1 AVs, 5 possible domain 2 P4R AVs and 5 possible domain 2 P4P AVs. These possible AVs would serve as the denominator for determining the Percentage Achievement Value in Step 4.

Step 4. Calculate Percentage Achievement Value

A Percentage Achievement Value (PAV) is calculated by dividing the earned AVs by the TAV, which represents the maximum AVs available by project and by the Metric/Milestone Domain (Domain 1, Domain 2 P4P, Domain 2 P4R, etc.). The PAV will demonstrate the percentage of achieved metrics within the Metric/Milestone Domain (Domain 1, P4R and P4P) for each project for that reporting period. Note that Domain 1 AVs are assigned to each project based on organizational and project specific milestones. When a PPS completes defined organizational milestones, the PPS earns an AV for the specific organizational component (Governance, Workforce, Cultural Competency/Health Literacy, Financial Sustainability, Funds Flow) for each project the PPS is implementing. PPS can earn additional Domain 1 AVs for meeting specific project implementation and patient engagement commitments made in the DSRIP Project

Plan Application. These AVs are specific to each project and a portion of these AVs may be earned or lost on a project by project basis. For Domain 4 projects, there are no P4P measures so only P4R AVs are determined.

Example: If a Performing Provider System has a project with 5 P4R metrics/milestones, it would have a TAV of 5 for the P4R for the project. If the PPS achieves the metric/milestone for 2 of the 5 possible metrics/milestones, it would achieve a PAV of 40% (2 AVs earned out of 5 AVs available) and it would receive 40 percent of the possible award of $100,000 or $40,000. The metrics/milestone value would be assigned AV and PAVs as follows:

Table 9 - Percent Achievement Value Example
Metric/Milestone	Achievement	AV Available	AV Earned
Milestone 1	Achieved	1	1
Milestone 2	Achieved	1	1
Milestone 3	Not Achieved	1	0
Milestone 4	Not Achieved	1	0
Milestone 5	Not Achieved	1	0
	TAV	5	2
	PAV 2/5	40%

Step 5. Calculate Actual Performance Payment

Payment

The project performance payment is determined by multiplying the PAV by the associated funding for the Metric/Milestone Domain (Domain 1, Domain2/3 P4P, Domain 2/3/4 P4R). The sum of the funding earned across the Milestone/Metric Domains for a project represents the payment earned for that project. The sum of the payments earned by project reflects the total performance payment earned by the Performing Provider System for each payment period.
Additionally, the Performing Provider System will be eligible for bonus payments by reaching separate high-performance targets described in Section III and Attachment J.

Funds are evenly distributed across two payment periods per year with the exception of DY1, where 60% of the available funding will be distributed in the first payment for the approval of the DSRIP Project Plan Application with the remaining funding distributed evenly across the second and third payments.

Note: All P4P funds will be distributed in the second payment period in the year in which a measure converts to P4P. For example, when domain 3 measures convert to P4P in year 2, all domain 3 P4P funds for year 2 will be available in the second payment period of that year. P4P funds will be evenly distributed across the two payment periods in subsequent years.

c. Project Value Monitoring

Performing Provider Systems will be required to develop budgets and report on DSRIP project spending throughout the demonstration. As described in paragraph VI.c below, CMS reserves the right to review project values to ensure that the project value index, the population denominator, and the overall project valuation are calculated correctly.

___________________________________

^{2. Performing Provider System improvement targets will be established annually using the baseline data for DY 1 and then annually thereafter for DY2-5. High level performance targets will be provided by the State using results from managed care reporting data in DY1 and using results from DSRIP projects in DY2-DY5 as described in metric targets in Section III c. 2}
^{3. If the Performing Provider System result also meets a high-performance threshold, there may be additional payment through High Performance fund, which is not included in this part of the payment calculation. 3}

VI. DSRIP Project Plan Review Process

Overview of Review Responsibilities
- Each PPS that elects to participate in the DSRIP program must submit a DSRIP Plan in accordance with the DSRIP Plan guidelines outlined in section IV of this Project Funding and Mechanics protocol, Attachment J: DSRIP Strategies Menu and Metrics, and the demonstration´s Special Terms and Conditions. PPSs are expected to provide accurate information in their DSRIP plans and respond to the state and CMS´s requests for additional information and/or plan revisions in accordance with the timelines specified.
- The state is responsible for reviewing all DSRIP plans using a CMS-approved checklist and other review process requirements described below. The state´s review will be supplemented by an independent assessment of DSRIP plans and a public engagement period, which should inform the state´s decision of whether to approve a DSRIP plan.
- CMS will monitor the state´s review process and approve projects in accordance with section VI.c. below.
- All PPSs will be subject to addition review during the mid-point assessment, at which point the state may require DSRIP plan modifications and may terminate some DSRIP projects, based on the feedback from the independent assessor, the public engagement process and the state´s own assessment of project performance. CMS will also monitor this mid-point assessment review process and make determinations in accordance with V.d
State-level Review Process
1. DSRIP plan review checklist
  - On or before October 1, 2014, the state will submit the state´s approach and review criteria for reviewing DSRIP Project Plans, as well as a draft DSRIP Plan Initial Review Checklist that will be used in the state´s initial review of DSRIP Plans to CMS.
  - CMS and the state will work collaboratively to refine the criteria, approach, and DSRIP Plan Checklist to support a robust review process and compelling justification for approval of each project. The state (with support from the independent assessor) will apply the CMS approved review process to ensure that DSRIP Plans are thoroughly and consistently reviewed.
    
    At a minimum, the DSRIP Plan Checklist shall include the following criteria:
    1. The plan is in the prescribed format and contains all required elements described herein and is consistent with special terms and conditions.
    2. The plan conforms to the requirements for Domains 1, 2, 3, and 4 as described herein, as well as in Attachment J: DSRIP Strategies Menu and Metrics.
    3. The plan clearly identifies goals, milestones, metrics, and expected results.
    4. The description of the project is coherent and comprehensive and includes a logic model clearly representing the relationship between the goals, the interventions and the measures of progress and outcome.
    5. The project selection is grounded in a demonstrated need for improvement at the time that the project is submitted and is sufficiently comprehensive to meaningfully contribute to the CMS three part aim for better care for individuals, better health for the population, lower costs through improvement (i.e. Triple Aim), and while at the same time charting a path towards future sustainability.
    6. The likelihood for success of this intervention is based on, where available, accurate and robust citations to the evidence base.
    7. The plan includes an approach to rapid cycle evaluation that informs the system of progress in a timely fashion, and how that information will be consumed by the system to drive transformation and who will be accountable for results, including the organizational structure and process to oversee and manage this process. The plan must also indicate how it will tie into the state´s requirement to report to CMS on a rapid cycle basis.
    8. The plan includes a detailed description of project governance. Included in the description will detailed accounting of how decisions will be made and what corporate structure will be used throughout the life of the project. A clear description of the powers granted to the project´s corporate entity by participating providers must be described as well as what the governance plan is beyond the waiver period. The governance plan must address how the PPS will address management of lower performing providers in the PPS network.
    9. The goals are mapped to a robust and appropriate set of research hypotheses to support the evaluation. There is a coherent discussion of the PPS´s participation in a learning collaborative that is strongly associated with the project and demonstrates a commitment to collaborative learning that is designed to accelerate progress and mid-course correction to achieve the goals of the project and to make significant improvement in the outcome measures specified.
    10. The amount and distribution of funding is in accordance with Section V of this protocol "Project Valuation."
    11. The plan, project, milestones, and metrics are consistent with the overall goals of the DSRIP program.
    12. The plan where necessary includes specific goals, projects, milestones and metrics focused on directly and aggressively addressing any provider financial stability issues.
2. Independent assessment and public engagement process
  - The state must identify an independent entity ("independent assessor") to conduct an impartial review of all submitted DSRIP plans. The independent assessor will first conduct an initial screen of DSRIP plans to ensure that they meet the minimum submission requirements.
  - The independent assessor will notify the PPS in writing of any initial questions or concerns identified with the provider´s submitted DSRIP Plan and provide an opportunity for PPSs to address these concerns.
  - After determining which DSRIP plans meet the minimum submission criteria, the independent assessor will convene a panel of relevant experts and public stakeholders to assist with the scoring of projects, in a manner similar to a federal grant review process. The independent assessor will ensure that standards are followed to prevent conflict of interest in the panel scoring process.
3. Consumer Education Campaign
  - The state will conduct a statewide consumer education campaign through a competitively selected contractor with a proven track record in conducting large public education campaigns. The consumer education campaign will focus on development of educational materials and marketing efforts to help educate Medicaid and uninsured members about the benefits of the DSRIP program and the services available through local PPSs. Campaign materials will be developed in multiple languages.
4. State assessment
  - According to a timeline developed by the state and CMS that aligns with the DSRIP deliverables schedule outlined in STC 1 in section VIII, the state will make its official, initial determination on each timely submitted DSRIP Plan based on the findings of the independent assessor and the outside review panel. Any deviations from the independent assessor´s recommendations should be clearly explained to CMS.
  - The state will notify the provider system in writing that the plan has been approved and submitted to CMS.
  - During the state review process, including by the independent assessor and before the state notifies the provider system of an approval, the state will make adjustments to these reviews to accommodate any systemic gaps that CMS identifies in its review of a sample of plans as provided in VI.c. Any revisions to the reviews will be applied to all plans.
CMS Monitoring Process
- In addition to approving the review protocol, CMS will review a sample of plans reviewed by the independent assessor and by the state to determine whether the protocol was followed, will identify any systematic gaps between the protocol and the actual reviews, and will provide such findings to the state to address these gaps in reviews by the independent assessor and by the state. CMS reserves the right to do a second sampling following notification by the state that the review processes were revised and after the independent assessor and the state complete additional reviews. Assuming that CMS finds that the reviews are consistent with the review protocol, CMS will accept the state´s recommendations for approval with the following possible exceptions which will be applied at CMS´s discretion:
  1. The state´s decision about approval is not consistent with the independent assessor.
  2. The plan is an outlier in the valuation schema.
  3. There is evidence in the plan, or exogenous information made available to CMS, that calls into question for the independent assessor or the state of funding duplication.
  4. There is evidence in the plan, or exogenous information made available to CMS, that calls into question whether the project is new or significantly expanded or enhanced from a project already underway.
- CMS will complete its review according to a timeline developed by the state and CMS that aligns with the DSRIP deliverables schedule outlined in STC 1 in section VIII. CMS reserves the right to conditionally approve plans, and to allow modifications to plans to resolve issues it identifies in its review provided that the modifications are made to the plan and found acceptable by CMS according to the timeline provided by CMS.
Mid-point Assessment
- The state´s mid-point assessment review will be developed in collaboration with CMS. All DSRIP plans initially approved by the state must be re-approved by the state in accordance with the CMS approved review protocol in order to continue receiving DSRIP funding in DY 4 and 5. The state will use and independent assessor and public engagement process similar to the process used for the initial approval of projects, described in paragraph b.ii above.
- The state will submit to CMS for approval, on or before October 1, 2016, draft mid- point assessment review criteria, a description of its approach to review, and a draft DSRIP Plan Mid-point Assessment Checklist that will reflect the approved criteria and will be used in the assessment. CMS and the state will work collaboratively to refine the criteria, approach, and DSRIP Plan Checklist. The state will apply these criteria to ensure that DSRIP projects are thoroughly and consistently reviewed.
- Where possible, the state will notify providers in advance of the mid-point assessment if providers need to make changes in order to comply with the approved review criteria.
- During DY 2, the independent assessor will work with the state to conduct a transparent mid-point assessment of all DSRIP projects using CMS-approved criteria. This review will provide an opportunity to modify projects and/or metrics in consideration of learning and new evidence. The independent assessor will conduct a focused review of certain high-risk projects identified by the state, CMS or the independent entity based on information provided for all projects in the provider´s monitoring reports.
- The mid-point assessment review will, at a minimum, include an assessment of the following elements:
  1. Compliance with the approved DSRIP project plan, including the elements described in the project narrative;
  2. Compliance with the required core components for projects described in the DSRIP Strategies Menu and Metrics, including continuous quality improvement activities;
  3. Non-duplication of Federal funds;
  4. An analysis and summary of relevant data on performance on metrics and indicators to this point in time;
  5. The benefit of the project to the Medicaid and uninsured population and to the health outcomes of all patients served by the project (examples include number of readmissions, potentially preventable admissions, or adverse events that will be prevented by the project);
  6. An assessment of project governance including recommendations for how governance can be improved to ensure success. The composition of the PPS network from the start of the project until the midpoint will be reviewed. Adherence to required policies regarding management of lower performing providers in the network will be reviewed with a special focus on any action with regard to removing lower performing members prior to DYs 3, 4, and 5. (Note: Modifying coalition members requires a plan modification);
  7. The opportunity to continue to improve the project by applying any lessons learned or best practices that can increase the likelihood of the project advancing the three-part aim; and
  8. Assessment of current financial viability of all lead providers participating on the DSRIP project.
- Based on the recommendations by the independent assessor, the state or CMS may require prospective plan modifications that would be effective for DYs 3, 4 and 5, including adjustments to project metrics or valuation. Significant changes to the number of Medicaid beneficiaries attributed to a PPS will require adjustments to the project valuation.
- The state will review all modifications resulting from the mid-point assessment prior to CMS review and consideration, consistent with the process for review of plan modifications, described in section X of this document. Future DSRIP payment for a provider may be withheld until the necessary changes as identified by the mid-point assessment are submitted (and all other requirements for DSRIP payment are met).

VII. Reporting Requirements and Ongoing Monitoring

Performance management and assessment of DSRIP will occur throughout its duration and will take several forms. Each area of assessment is interrelated to ensure a continuous cycle of quality improvement and shared learning. The final project work plans will provide the basis for monitoring each project.

Ongoing provider-level evaluations will occur on a regular basis, as described below, and seek to provide timely and actionable feedback on the initiative´s progress, in terms of infrastructure changes, implementation activities and outcomes. The formative evaluation, or performance management, will track and report regularly on actions, performance on objective attainment and overall progress towards achieving a health care system based on the improving health, improving care, and reducing costs, and progress toward achieving the primary goals of DSRIP, to reduce avoidable hospitalization and seek improvements in other health and public health measures by transforming systems.
Learning collaboratives will be implemented to seek peer-to-peer (provider-to- provider) and community stakeholder input on project level development of action plans, implementation approaches and project assessment. New York will be responsible for leading the collaborative approach to ensure effective sharing of information (e.g. best practices, case studies, challenges, results). The schedule for the collaboratives meeting will be shared with CMS.
On a quarterly basis, the state will publish on its website project-by-project status updates which will show available data that reflects each strategy´s progress on metrics and indicators, as relative to pre-approved targets.
A mid-point assessment will be completed by an independent assessor. The midpoint assessment which will provide independent quantitative analysis of DSRIP planning and implementation through June 2016, as well as timely qualitative research findings which will provide context for reports on provider´s progress in planning and implementing selected DSRIP programs. The qualitative findings will contribute to understanding implementation issues which go beyond the quantitative analyses. In addition, the qualitative analysis will inform and sharpen analytic plans for the summative evaluation. The mid-point assessment will be submitted by the end of March 2017.
In addition to monitoring, an interim and final summative statewide evaluation of DSRIP will be completed by the independent evaluator to examine the effect of DSRIP activities on achieving the State goals of (1) safety net system transformation at both the system and state level; (2) accountability for reducing avoidable hospital use and improvements in other health and public health measures at both the system and state level; and (3) efforts to ensure sustainability of delivery system transformation through leveraging managed care payment reform. The data and findings of the mid-point assessment will be among the information used by the independent evaluator for the interim evaluation. Among other things, the interim evaluation will provide broad learning both within the state and across the nation. Part of this interim evaluation will examine issues overlapping with ongoing provider- level evaluations, and part of this effort will examine questions overlapping with the final evaluation.
1. Semi-annual Reporting on Project Achievement
  - Two times per year, PPSs seeking payment under the DSRIP program shall submit reports to the state demonstrating progress on each of their projects as measured by the milestones and metrics described in their approved DSRIP plan. The reports shall be submitted using the standardized reporting form approved by the state and CMS. Based on the reports, the state will calculate the incentive payments for the progress achieved in accordance with Section IX "Disbursement of DSRIP Funds". The PPS shall have available for review by the state or CMS, upon request, all supporting data and back-up documentation. These reports will be due as indicated below after the end of each reporting period:
    - Reporting period of April 1 through September 30: the reporting and request for payment is due October 31.
    - Reporting period of October 1 through March 31: the reporting and request for payment is due April 30.
    These reports will serve as the basis for authorizing incentive payments to PPSs for achievement of DSRIP milestones. The state shall have 30 days to review and approve or request additional information regarding the data reported for each milestone/metric and measure. If additional information is requested, the PPS shall respond to the request within 15 days and the state shall have an additional 15 days to review, approve, or deny the request for payment, based on the data provided. The state shall schedule the payment transaction for each PPS within 30 days following state approval of the PPS´s semi-annual report. As part of CMS´s monitoring of DSRIP payments, CMS reserves the right to review a sample of the PPS Reports and withhold or defer FFP if DSRIP milestones have not been met.
    
    Because many domain 2, 3, and 4 metrics are annual measures, these annual measures will only be available to be reported once a year for purposes of authorizing and determining incentive payments.
2. State Monitoring Reports
  - The state, or its designee, will conduct robust monitoring and assessment of all submitted reports, PPS progress, challenges and completion no less frequently than quarterly, and as appropriate in order to monitor DSRIP implementation and activities.
  - Upon this review, an analysis will be made regarding:
    - the extent of progress each PPS is making towards meeting each milestone;
    - the specific activities that appear to be driving measurable change;
    - the key implementation challenges, including governance issues, associated with specific activities designed to drive improvement; and
    - the identification of adjustments to the DSRIP program, and/or projects as observed through the analysis of submitted provider-level data and/or onsite findings as they occur.
    Comparative analysis and findings will be performed and summarized into actionable reports that provide the right level of information to various project stakeholders to help facilitate learning at the PPS level, as well as the DSRIP program level. The reports will be used to drive peer-to-peer discussion regarding opportunities for improvement and methods for course correction through the use of the Learning Collaborative. The results of these assessments will be disseminated to the independent DSRIP evaluation contractor and CMS. This information is expected to inform the DSRIP evaluation during both the mid-point and summative evaluations to understand key factors related to the performance and progression of the DSRIP program to date.
    
    The state, or its designee, will take effective action, as needed, to remedy a finding to promote fulfillment of the DSRIP goals. This may include providing feedback to the health care industry at-large, or individual project participants if significant issues are observed.
    1. Operational Report
      - An operational report at the project level will be the primary report to manage and report DSRIP performance. The operational report will have the functionality to report on project-level data related to PPSs performing the same project. This report will also include an Executive Summary which will be used by CMS, senior state officials and the public as a means of following the overall progress of the DSRIP demonstration. This report will include the following data elements:
        
        Identification of participating providers
        
        Completion factor of providers, by provider
        
        Dashboard of project-specific measure results, aggregated at project, plan, regional and state levels
        
        Summary of applied interventions
        
        Summary of pilot models
        
        Summary of reported challenges
        
        Summary of reported successes
        
        Update on governance
        
        Noted best practices
        
        Summary of approved payments (compared to the valuation in the DSRIP project plan), which should reconcile to the DSRIP funding reported on the CMS-64
        
        This report will be used to inform and direct the Learning Collaboratives. It will be used to ensure consistent analysis on key implementation activities across PPSs and act as a platform for discussion during monthly conference calls and quarterly in-person collaboration meetings. This report may be utilized by the PPS project personnel as a primary tool to aid routine collaboration among PPSs implementing the same project. This level of reporting may also show progress of the learning process itself by tracking the frequency of meetings by activity and participation in order to confirm that the learning collaborative activity is being fulfilled by the PPS.
        
        It will be the responsibility of each project participant to ensure effective diffusion of learning amongst PPSs who have selected the same project focus area. This includes discussing the types of innovations, strategies and Plan-Do-Study-Act (PDSA) cycles that have been implemented throughout the demonstration.
    2. Consumer Level Report
      - A consumer level report will have the functionality to report on high-level geographic and project-specific data elements in order to understand which providers in their area are driving to improve quality and the area of focus for that PPS. The report may include:
        
        County-level map that indicates all New York hospitals
        
        County-level map that indicates all participating hospitals and participating outpatient providers
        
        This report may also have drill-down functionality to learn summary detail about the objective, methodology, current performance, and expected results of each PPS.
3. Learning Collaborative
  - One facet of the DSRIP program is the development of the Learning Collaborative. The purpose of the Learning Collaborative is to promote and support a continuous environment of learning and sharing based on data transparency within the New York healthcare industry in an effort to bring meaningful improvement to the landscape of healthcare in New York.
  - The Learning Collaborative will be managed by the state and/or its independent assessor through both virtual and in-person collaboration that both builds relationships as well as facilitates project analysis and measurement. The Learning Collaborative will be designed to promote and/or perform the following:
    1. Sharing of DSRIP project development including data, challenges, and proposed solutions based on the PPS´s quarterly progress reports
    2. Collaborating based on shared ability and experience
    3. Identifying key project personnel
    4. Identification of best practices
    5. Provide updates on DSRIP program and outcomes
    6. Track and produce a "Frequently Asked Questions" document
    7. Encourage the principles of continuous quality improvement cycles
    There will be multiple collaboratives developed based on the number and type of projects chosen by PPSs. Each learning Collaborative will include key personnel from the PPSs and selected members of the stakeholder community including provider association representation. For each collaborative, the state will designate personnel to be responsible for guiding and facilitating the Learning Collaborative.
    
    An online, web-based tool will be utilized in order to effectively manage the collection and the dissemination of information related to the DSRIP and projects. A key component of the online tool will be a reporting feature that allows tiered-level reporting that conveys key information to the various levels of stakeholder groups interested in learning and tracking performance of the DSRIP program. This tool will act as a repository with reporting capability for various audiences including that of the general public, the Department, CMS, and the healthcare industry. The tool will deliver data in ways that can be 1) easily interpreted by various stakeholders, 2) promote self-evaluation, and 3) promote the diffusion of effective intervention models.
4. Program Evaluation
  - As described in STC 5d. in section VIII, the state will identify an independent evaluator to provide an interim and summative evaluation. The interim evaluation will consider among other things the findings of the mid-point assessment conducted by the independent assessor. The evaluations must be in accordance with the evaluation STCs 19-30 in section VII and as approved by CMS through the evaluation design phase provided in STC 20 in section VII.
  - The interim evaluation will be due one-year prior to the expiration of the demonstration and will include data from DY 1, 2, and 3. The final, summative evaluation will be completed by the end of March 2020. The interim and summative evaluation will meet all standards of leading academic institutions and academic peer review, as appropriate for both aspects of the DSRIP program evaluation, including standards for the evaluation design, conduct, interpretation, and reporting of findings.
5. Overall Data Standards
  - The state will collect data from providers often as is practical in order to ensure that project impact is being viewed in as "real time" a fashion as possible. Collecting and analyzing data in this fashion will allow for rapid, life-cycle improvement which is an essential element of the DSRIP project plan.
  - Since managed care is an important component of the state´s quality improvement strategy, the state will implement a provider/plan data portal that will allow access to appropriately permissioned patient and provider specific data in the Medicaid Data Warehouse. Role based access to this portal will allow providers and their partnering health plans access to current Medicaid claims and encounters data and eventually real time EMR and care management data provided through connectivity with local regional health organizations (RHIOs). Faster access to more real time clinical and managed care data will be particularly relevant to this project and is also the rationale for using state-measured health plans metrics or Quality Assurance Reporting Requirements (QARR) as a major data source for this project. In addition, providers and their partnering health plans will have access to the analytical capabilities of 3M and Salient the State´s suite of performance tools through the portal. This will allow DSRIP providers and the health plans to partner with the state to measure case mix adjusted avoidable hospitalization metrics at the local level using standardized definitions and eventually with more real time updates. More information on DSRIP performance and the portal will be posted on the state´s DSRIP website.
    
    The state will use the Quality Committee, established in 2013 to assist DOH on quality measurement and improvement that will be responsible for supporting the clinical performance improvement cycle of DSRIP activities. The Quality Committee includes representatives from various sectors of healthcare including hospitals, nursing homes, managed care plans, provider organizations and consumer representation. Additional members will be added specifically to reflect DSRIP including representatives from local governmental units and additional consumer representation. The current charge of the Committee is to provide NYSDOH with expertise in various sectors of health care quality, assist on proposed quality improvement goals and provide guidance on measuring and reporting quality information to the public. The Quality Committee will serve as an advisory group for DSRIP offering expertise in health care quality measures, clinical measurement and clinical data used in performance improvement initiatives.
    
    Final decision-making authority will be retained by the state and CMS, although all recommendations of the committee will be considered by the state and CMS.
    
    Specifically, the Quality Committee will provide feedback to the state regarding:
    1. Development of attribution models
    2. Selection of metrics
    3. Selection of the high-performance target goals including the behavioral health high performance avoidable hospitalization threshold for bonus payment purposes.
    Data and metrics that form the basis of incentive payments in DSRIP should have a high degree of accuracy and validity. Consistent with current requirements for MCO and PIHPs under 438.242, the state must ensure, through its contracts with the PPSs, that each PPS receiving payments under DSRIP maintains (or participates in) a health information system that collects, analyzes, integrates, and reports data and can achieve the objectives of this DSRIP. The state must require that each PPS ensure that data received from providers within the system is accurate and complete by ensuring that PPSs have appropriate data agreements in place (as described in section IV.b) and verifying the accuracy and timeliness of reported data (including such data that contributes to chart review metrics), screening the data for completeness, logic, and consistency. To the degree that the data and metrics are generated and obtained via managed care systems already subject to 438.242, no additional validation of the data is required.
    
    For data and metrics reported in systems not subject to 438.242, these agreements between the state and PPSs should also be accompanied by validation process performed by the independent assessor to ensure that the processes are generally valid and accurate. Penalties will be applied to PPSs that are not reporting data that are valid and accurate as described.

VIII. DSRIP Funding Limits

Statewide limit on DSRIP Funding

Total DSRIP Fund expenditures are limited as specified in STC 15 in section VII. In addition to this limit, DSRIP Fund expenditures cannot exceed the lesser of the aggregate valuation of DSRIP projects as adjusted to include DSRIP planning funding and funds allocated to the High-Performance Fund.

Table 10 Sources of DSRIP Funding.

Sources of Funding	Year-0	Year-1	Year-2	Year-3	Year-4	Year-5	Total
Public Hospital IGT Transfers (Supports DSRIP IGT Funding for Public Performing Provider Transformation Fund, Safety Net Performance Provider System Transformation Fund, DSRIP, State Plan and Managed Care Services)	505.1	657.2	960.8	1533.7	1418	925.2	6000
State Appropriated Funds	134.3	429.8	614.5	497.4	249.3	74.6	2000
Total Sources of Funding	639.4	1087	1575.3	2031.1	1667.4	999.8	8000
Uses of Funding
DSRIP Expenditures	602.3	1049.1	1249.3	1698.3	1410.5	908.9	6918.5
Interim Access Assurance Fund (IAAF)	500	0	0	0	0	0	500
Planning Payments	69.9	0	0	0	0	0	69.9
Performance Payments	0	981.8	1144.3	1668.4	1379.5	874.7	6048.6
Administration	32.4	67.4	105	29.9	31	34.2	300

Health Homes	37.1	37.9	20.3	0	0	0	95.3

MC Programming	0	0	305.7	332.8	256.8	90.9	986.2
Health Workforce MLTC Strategy	0	0	91.2	80.2	52.5	47.4	271.2
Home and Community Based Services/1915i Services	0	0	214.5	252.6	204.3	43.6	715
Total Uses of Funding	639.4	1087	1575.3	2031.1	1667.4	999.8	8000

Allowable expenditures are further limited by the availability of non-federal funding (through proper IGT or other funding), and provider performance on DSRIP milestones and metrics. Public PPS will be required to provide completed attestations to the state for each payment in which IGT funding will be used for the non-federal share. The PPS is expected to maintain documentation of the source of the non- federal share of the funds and be able to provide documentation upon request from the state and CMS.

In order to support the successful engagement of community-based organizations (CBO) in DSRIP, the state will direct 5% of the DY1 Administrative Costs, as identified in STC 13.d, towards a CBO planning grant.

Public Hospital and Safety Net Provider Performance Provider System Transformation Funds

All PPSs with approved DSRIP Project Plans will be eligible to apply for funding from one of two DSRIP pools. The first, Public Hospital Transformation Fund, will be open to applicants led by a major public hospital system. The public hospital systems allowed to participate in this fund include:
1. Health and Hospital Corporation of New York City
2. State University of New York Medical Centers
3. Nassau University Medical Center
4. Westchester County Medical Center
5. Erie County Medical Center
The second fund Safety Net Performance Provider System Transformation Fund would be available to all other DSRIP eligible providers.

Allocation of funds between the two pools will be determined after applications have been submitted, based on the valuation of applications submitted to each pool.
High performance fund
- A portion of the Public Hospital Transformation Fund and Safety Net Performance Provider System Transformation Fund will be set aside to reward high performing systems according to the process specified below. The high-performance fund will be made up of the following funds:
  1. For DY 2-5, up to 10% of the total DSRIP funds set aside for the high-performance fund
  2. Target Funds that are forfeited from providers that do not achieve project milestones and metrics, less any prior year appealed forfeited funds where the appeal was settled in the current demonstration year in favor of the PPS.
The total amount of funding allocated for the high-performance fund shall be distributed to qualifying providers based on meeting a specific set of Domain 2 and 3 metrics identified as a high-performance metrics by the state with input from the quality and measures committee. The metrics for the high-performance fund are specified in Attachment J.

Additional funds will be set aside within each fund for PPSs reaching stretch/bonus level targets (set by the state with input from the quality and measures committee) for significant improvement in avoidable hospitalization reduction for their attributed behavioral health population.

IX. Disbursement of DSRIP Funds

Total Available DSRIP Incentive Payments for a Project based on Project Valuation Aggregate incentive payments available over the 5-year demonstration period to a PPS will be based on the project valuation approved by the state, subject to the limits set forth in section V above.
Payment Based on Milestone Achievement for DY 1 - DY 5
- Incentive payments are calculated separately for each project. The amount of the incentive funding paid to a provider will be based on the amount of progress made within specific milestones and the valuation of those milestones.
- Sixty percent of the incentive funding for DY 1 will be awarded for approval of the DSRIP plan. Ten percent will be paid upon the delivery of an acceptable first semiannual report. Ten percent will be paid upon the delivery of an acceptable second semiannual report. The remaining 20 percent of DY 1 funding will be distributed among Domains 2, 3 and 4 in accordance with Table 4. Over the remainder of the DSRIP, P4R will decrease as P4P increases, with a focus on System Transformation (Domain 2) and Clinical Improvement (Domain 3), and funding for metrics in each DSRIP year will be in accordance with Table 4.
- For each metric, the provider will include in the required DSRIP provider report the progress made in completing each metric along with sufficient supporting documentation. Progress for a given metric will be categorized as fully achieved or not achieved. If a provider has previously reported progress in a domain and received partial funding after the first semi-annual reporting period, only the additional amount is eligible for funding in the second semi-annual reporting period.
Payments from the High-Performance Fund
- PPSs who have achieved performance improvement beyond the stated target improvement value in their approved DSRIP project plan will be eligible for additional payment from the DSRIP High-Performance fund, not to exceed 30 percent of their DSRIP project value.
- A half of the High-Performance fund will be available for tier 1 payments, and half will be available for tier 2 payments which will be distributed as follows:
  1. Higher performing participating providers whose performance closes the gap between their current performance and the High-Performance level by 20 percent shall receive Tier 1 level reward payments.
  2. Higher performing participating providers whose performance meets or exceeds the statewide performance goal for the measurement period shall receive Tier 2 level reward payments.
  High performance fund payments shall be adjusted based on Medicaid and indigent population size served by the project being implemented by the provider. The percentages above may be adjusted up or down by the State for each metric as appropriate to account for volume of demand on the High-Performance fund.
- The state, working with the quality committee, will set a High-Performance threshold for the measures described in attachment J specifically avoidable hospitalizations for the entire attributed population and separate High-Performance targets (physical and behavioral metrics) for the behavioral health population subset. High-Performance payments will be based on attaining 20 percent gap to goal or the statewide performance on the High-Performance metrics listed in Attachment J.
Accountability for state performance
- As described in STC 14 in section VII providers and the state are accountable for statewide performance. DSRIP funding for providers may be reduced based on poor performance statewide described below.
- If any of the four milestones below are not met, then DSRIP payments to providers will be reduced by the amount specified in STC 14 in section VII DSRIP payment reductions will be applied proportionately to all DSRIP PPSs based on the valuation of their DSRIP project plans. DSRIP reductions will not be applied to the DSRIP High-Performance fund payments.
- Achievement of the statewide milestones is calculated as follows:
  1. Statewide performance on universal set of delivery system improvement metrics. The core set of delivery system improvement metrics in domain 2 of Attachment J will be assigned a direction for improving or worsening and will be calculated to reflect the performance of the entire state. This milestone will be considered passed in any given year if more metrics in this domain are improving on a statewide level than are worsening (i.e. the performance level is the same or better, no error bar applied), as compared to the prior year as well as compared to initial baseline performance (DY 1).
  2. A composite measure of success of projects statewide on project-specific and population-wide quality metrics. The number of metrics met by each PPS in a given year based on the project-specific improvement standards specified in their approved DSRIP project plan will be added together to determine the composite success of all DSRIP projects. For the purposes of this addition, pay for reporting measures will only be counted once in the aggregate for each domain. This statewide milestone will be considered passed in any given year if the number of metrics met by all PPSs in the aggregate is greater than the number of metrics that were not met.
  3. Growth in statewide total Medicaid spending that is at or below the target trend rate. As further described in STC 14 in section VII statewide performance on this milestone will be considered passed if the state improves on the following two metrics on a per member per month (PMPM) basis, comparing the most recent state fiscal year to the year that immediately precedes it.
    1. a. Growth in statewide total inpatient and emergency room spending that is at or below the target trend rate (Measure applies in DSRIP Year 3, DSRIP Year 4 and DSRIP Year 5).
    2. Growth in statewide total Medicaid spending that is at or below the target trend rate (measure applies in DSRIP Year 4 and DSRIP Year 5). PMPM amounts will be adjusted to exclude growth in federal funding associated with the Affordable Care Act. The state will not be penalized if it uses these higher FMAP rates generated by the ACA to reinvest in its Medicaid program.
    For total Medicaid spending, the target trend rate is the ten-year average rate for the long-term medical component of the Consumer Price Index (as used to determine the state´s Medicaid Global Spending Cap for that year), for DSRIP Years 4 and 5 only. For inpatient and emergency room spending the target trend rate is the ten-year average rate for the long-term medical component of the Consumer Price Index (as used to determine the state´s Medicaid Global Spending Cap for that year) minus 1 percentage points for DSRIP Year 3 and 2 percentage points for DSRIP Years 4 and 5.
  4. Implementation of the managed care plan. This milestone will be measured by targets agreed upon by CMS and the state after receipt of the managed care strategy plan in STC 37 in section VII related to reimbursement of plans and providers consistent with DSRIP objectives and measures. These targets will include one associated with the degree to which plans move away from traditional fee for service payments to payment approaches rewarding value.
Intergovernmental Transfer Process
- The state will calculate the non-federal share amount to be transferred by an IGT Entity in order to draw the federal funding for the incentive payments related to the milestone achievement that is reported by the PPS and approved by the IGT Entity and the State. Within 14 days after notification by the state of the identified nonfederal share amount, the IGT Entity will make an intergovernmental transfer of funds. The state will draw the federal funding and pay both the nonfederal and federal shares of the incentive payment to the PPS. If the IGT is made within the appropriate 14-day timeframe, the incentive payment will be disbursed within 30 days. The total computable incentive payment must remain with the PPS.

X. DSRIP Project Plan Modifications

Modifying Existing Project Plans in Limited Circumstances
- No more than once a year, PPS may submit proposed modifications to an approved DSRIP project plan for state and CMS review. These modifications may not decrease the scope of the project unless they also propose to decrease the project´s valuation. Removal of any PPS member organization requires a proposed modification and removal of any such lower performing member must follow the required governance procedures including progressive sanction requirements.
- The state and CMS will follow the same review process described in section VI above, except that the independent assessor will not be expected to convene review panels.
Reinvestment of Unused DSRIP Funds in DY3, DY4 and DY5
- Unused DSRIP funding for DY 3, DY 4 and 5 (including funding allocated to projects that were terminated as part of the midpoint assessment) may be directed towards further replicating high performing DSRIP projects or other waiver priority efforts that have proven to be particularly successful and can be implemented elsewhere (in approved PPS that are not currently employing such projects or in providers that can demonstrate the need to further expand such efforts) and achieve results within the remaining DSRIP years. The state will develop its methodology for expanding successful projects and submit this to CMS for review and approval before the midpoint assessment is completed.

|top of section| |top of page|

ATTACHMENT J

DSRIP Strategies Menu and Metrics

I. Preface

Delivery System Reform Incentive Payment Fund

On April 14, 2014, the Centers for Medicare and Medicaid Services (CMS) approved New York´s request for an amendment to the New York´s Partnership Plan section 1115(a) Medicaid demonstration extension (hereinafter "demonstration") authorizing the creation of a Delivery System Reform Incentive Payment (DSRIP) Fund. The demonstration was extended in 2016, and renamed the New York Medicaid Redesign Team Demonstration. This demonstration is approved through March 31, 2021.

Section IX of the Special Terms and Conditions (STC) describes the general rules and requirements of the Delivery System Reform Incentive Payment (DSRIP) Fund.
DSRIP Strategies Menu and Metrics and Program Funding and Mechanics Protocol

The DSRIP requirements specified in the STCs are supplemented by two attachments to the STCs. The Program Funding and Mechanics Protocol (Attachment I) describes the State and CMS review process for DSRIP project plans, incentive payment methodologies, reporting requirements, and penalties for missed milestones. The DSRIP Strategies Menu and Metrics (this attachment, Attachment J) details the specific delivery system improvement projects and metrics that are eligible for DSRIP funding. The projects are listed in Part I and the metrics are listed in Part II. Additional information is provided in two additional documents as described below.

This version of the DSRIP Strategies Menu and Metrics is approved January 19, 2017. In accordance with STC 10.b of section IX, the state may submit modifications to this protocol for CMS review and approval in response to comments received during the post-award comment period and as necessary to implement needed changes to the program as approved by CMS.
Supporting operational guides

This attachment will be supplemented by two additional operational guides developed by the state and submitted to CMS, which will assist performing provider systems in developing and implementing their projects and will be used in the state´s review of the approvability and the valuation of DSRIP projects.

First, the state will develop a Project Toolkit that will describe the core components of each DSRIP project listed on the DSRIP project menu below (Part I). This supplement will also describe how DSRIP projects are distinct from each other and the state´s rationale for selecting each project (i.e. the evidence base for the project and its relation to community needs for the Medicaid and uninsured population). The core components and other elements of the project description will be used as part of the DSRIP plan checklist (described in section V of Attachment I). To assist providers in valuing projects, this supplement will also include the index score of transformation/ health care improvement potential determined by the state (according to the process described in section IV.c. of Attachment I).

Second, the state will develop a Metric Specification Guide that provides additional information on the metrics described in the metrics list below (Part II). Specifically, the state will specify the data source for each measure (specifically whether the measure is collected by the state or providers), the measure steward for each metric (if applicable), the National Quality Forum reference number (if applicable), and the high-performance level for each pay-for-performance metric. The high-performance level for each metric will be used to establish outcome targets for all pay-for- performance measures, as described in Attachment I.

Part I - Projects Menu

Each Performing Provider System will employ multiple projects both to transform health care delivery as well as to address the broad needs of the population that the performing provider system serves. These projects described in Attachment J are grouped into different strategies, such as behavioral health, within each Domain (System Transformation Projects (Domain 2), Clinical Improvement Projects (Domain 3), and Population-Wide Projects (Domain 4). For each strategy, there is a set of metrics that the performing provider system will be responsible for if they do any one of the projects within that strategy.

Each project selected by a Performing Provider System will be developed into a specific set of focused milestones and metrics that will be part of the Performing Provider System´s DSRIP project plan. Project selection will be driven by the mandatory community needs assessment, and the rationale and starting point for each project must be described in the DSRIP project plan, as described in Attachment I.

DSRIP project plans must include a minimum of five projects (at least two system transformation projects, two clinical improvement projects, and one population-wide project). As described further in Attachment I, a maximum of 11 projects will be considered for project valuation scoring purposes. Additional projects can be included in the application, but they will not affect the project valuation.

Domain 2: System Transformation Projects

All DSRIP plans must include at least two of the following projects based on their community needs assessment. At least one of those projects must be from sub-list A and one of these projects must be from sub-list B or C, as described below. Performing Provider Systems can submit up to 4 projects from Domain 2 for valuation scoring purposes. For eligible Performing Provider Systems pursuing 11 projects in their plan, they will be allowed to select up to 5 projects (the fifth project being project 2.d.i) from Domain 2 for scoring purposes (as described in attachment I).
Create Integrated Delivery Systems (required)
- 2.a.i Create Integrated Delivery Systems that are focused on Evidence-Based Medicine / Population Health Management
- 2.a.ii Increase certification of primary care practitioners with PCMH certification and/or Advanced Primary Care Models (as developed under the New York State Health Innovation Plan (SHIP))
- 2.a.iii Health Home At-Risk Intervention Program: Proactive management of higher risk patients not currently eligible for Health Homes through access to high quality primary care and support services.
- 2.a.iv Create a medical village using existing hospital infrastructure
- 2.a.v Create a medical village/alternative housing using existing nursing home
Implementation of Care Coordination and Transitional Care Programs
- 2.b.i Ambulatory Intensive Care Units (ICUs)
- 2.b.ii Development of co-located of primary care services in the emergency department (ED)
- 2.b.iii ED care triage for at-risk populations
- 2.b.iv Care transitions intervention model to reduce 30-day readmissions for chronic health conditions
- 2.b.v Care transitions intervention for skilled nursing facility (SNF) residents
- 2.b.vi Transitional supportive housing services
- 2.b.vii Implementing the INTERACT project (inpatient transfer avoidance program for SNF)
- 2.b.viii Hospital-Home Care Collaboration Solutions
- 2.b.ix Implementation of observational programs in hospitals
Connecting Settings
- 2.c.i. Development of community-based health navigation services
- 2.c.ii Expand usage of telemedicine in underserved areas to provide access to otherwise scarce services
Utilizing Patient Activation to Expand Access to Community-based Care for Special Populations
- 2.d.i Implementation of Patient Activation Activities to Engage, Educate and Integrate the uninsured and low/non-utilizing Medicaid populations into Community-based Care

Domain 3: Clinical Improvement Projects

All DSRIP plans must include at least two projects from this domain, based on their community needs assessment. At least one of those projects must be a behavioral health project from sub-list A, as described below. Performing Provider Systems can submit up to 4 projects from Domain 3 for valuation scoring purposes (as described in Attachment I).

Behavioral Health (required)
- 3.a.i Integration of primary care and behavioral health services
- 3.a.ii Behavioral health community crisis stabilization services
- 3.a.iii Implementation of evidence-based medication adherence program (MAP) in community-based sites for behavioral health medication compliance
- 3.a.iv Development of Withdrawal Management (e.g., ambulatory detoxification, ancillary withdrawal services) capabilities and appropriate enhanced abstinence services within community-based addiction treatment programs
- 3.a.v Behavioral Interventions Paradigm (BIP) in Nursing Homes
Cardiovascular Health

Note: Performing provider systems selecting cardiovascular health projects will be expected to utilize strategies contained in the Million Hearts campaign as appropriate.
- 3.b.i Evidence-based strategies for disease management in high risk/affected populations (adult only)
- 3.b.ii Implementation of evidence-based strategies in the community to address chronic disease -- primary and secondary prevention projects (adult only)
Diabetes Care
- 3.c.i Evidence-based strategies for disease management in high risk/affected populations (adults only)
- 3.c.ii Implementation of evidence-based strategies in the community to address chronic disease - primary and secondary prevention projects (adults only)
Asthma
- 3.d.i Development of evidence-based medication adherence programs (MAP) in community settings - asthma medication
- 3.d.ii Expansion of asthma home-based self-management program
- 3d.iii Implementation of evidence-based medicine guidelines for asthma management
HIV/AIDS
- 3.e.i Comprehensive Strategy to decrease HIV/AIDS transmission to reduce avoidable hospitalizations - development of a Center of Excellence for management of HIV/AIDS
Perinatal Care
- 3.f.i Increase support programs for maternal & child health (including high risk pregnancies) (Example: Nurse-Family Partnership)
Palliative Care
- 3.g.i Integration of palliative care into the PCMH Model
- 3.g.ii Integration of palliative care into nursing homes
Renal Care
- 3.h.i Specialized Medical Home from Chronic Renal Failure

Domain 4: Population-wide Projects

The following represent priorities in the State´s Prevention Agenda with health care delivery sector projects to influence population-wide health (available here.) The alignment of these projects with the New York State Prevention Agenda (including focus areas, etc.) is described further in the Project Description Supplement.

All DSRIP plans must include at least one project from this domain, based on their community needs assessment. Performing Provider Systems can submit up to 2 projects from Domain 4 for valuation scoring purposes (as described in Attachment I).

Promote Mental Health and Prevent Substance Abuse (MHSA)
- 4.a.i Promote mental, emotional and behavioral (MEB) well-being in communities
- 4.a.ii Prevent Substance Abuse and other Mental Emotional Behavioral Disorders
- 4.a.iii Strengthen Mental Health and Substance Abuse Infrastructure across Systems
Prevent Chronic Diseases
- 4.b.i Promote tobacco use cessation, especially among low SES populations and those with poor mental health
- 4.b.ii Increase Access to High Quality Chronic Disease Preventive Care and Management in Both Clinical and Community Settings (Note: This project targets chronic diseases that are not included in domain 3., such as cancer)
Prevent HIV and STDs
- 4.c.i Decrease HIV morbidity
- 4.c.ii Increase early access to, and retention in, HIV care
- 4.c.iii Decrease STD morbidity
- 4.c.iv Decrease HIV and STD disparities
Promote Healthy Women, Infants and Children
- 4.d.i Reduce premature births

II. Metrics

The domains of metrics here are intended to provide specificity to the overall intent to promote system transformation, using measures of system transformation as well as including avoidable events as a marker for positive transformation. Items associated with pay for reporting or pay for performance are described in requirements for all domains as well.

An overview of the metric domains from the funding and mechanics protocol is below:

Overall project progress metrics (Domain 1)
System transformation metrics (Domain 2)
Clinical improvement metrics (Domain 3)
Population-wide project implementation metrics (Domain 4)

All DSRIP plans must include all core metrics in Domain 1, all metrics in Domain 2, and all core metrics in Domain 4. DSRIP plans must also include the behavioral health metrics in Domain 3a and strategy-specific metrics based on the Domain 3 and 4 projects selected, as further described in the Project Toolkit. The state or CMS will add project-specific Domain 1 metrics to DSRIP project plans as necessary to address concerns with "at risk" projects, based on input from the independent assessor. Behavioral health metrics are included because those diagnoses are highly correlated with avoidable events.

A subset of these metrics related to avoidable hospitalizations, behavioral health and cardiovascular disease will also be part of the high-performance fund, described in attachment I and as noted below: These latter markers align with the nationwide Million Hearts Initiative on cardiac outcomes, in order to tackle the leading cause of mortality in New York State.

Metric	Domain reference
Potentially Preventable Emergency Room Visits (All Population)	2.a
Potentially Preventable Readmissions (All Population)	2.a
Potentially Preventable Emergency Room Visits (BH Population)	3.a
Potentially Preventable Readmissions (BH Population)	3.a
Follow–up after Hospitalization for Mental Illness (NQF 0576)	3.a
Antidepressant Medication Management	3.a
Diabetes Monitoring for People with Diabetes and Schizophrenia (NQF 1934)	3.a
Cardiovascular Monitoring for People with Cardiovascular Disease and Schizophrenia (NQF 1933)	3.a
Controlling High Blood Pressure (NQF 0018)	3.b
Medical Assistance with Smoking and Tobacco Use Cessation (NQF 0027) (component on discussing smoking and tobacco use cessation strategies)	3.b

Where possible, the state will make drillable data available for PPSs to be able to better understand the impact of disparities on the PPSs and improvements seen in specific populations through these projects. Because of small population size and lack of standards for comparison, the state will not be able to provide meaningful statewide metrics for each population segment.

Domain 1. Overall Project Progress Metrics

Domain 1 metrics assess overall implementation of all DSRIP projects (regardless of whether the project was developed from a project selected from Domain 2, 3, or 4 listed above).

Core Domain 1 Metrics (for all providers):

Semi-annual reports (pay for reporting), which will include:
1. Project narrative on status and challenges
2. Information on project spending/budget and any other financial information requested by the state, including financial sustainability of system and projects.
3. Documentation on the number of beneficiaries served through the projects
4. Update on project governance
5. Update on workforce strategy implementation
6. Percent of providers that are reporting relevant DSRIP project data
7. Description of steps taken by the system to prepare for non-FFS reimbursement systems (including an update on any on-going negotiations with Medicaid managed care plans)
8. Engagement in learning collaboratives
Approval of DSRIP Plan (DY 1 only)
Workforce milestones (P4P/ P4R, as specified in the Metrics Specification Guide)
- Percent Complete of System´s preapproved Workforce Plan Number of health care workers retrained/redeployed vs. # eligible based on system service changes
- Net change in number of new MDs hired - PCP; specialty
- Net change in number of new mid-levels providers hired (RPA, NP, NM)
- Net change in number of other mid-level providers hired
System Integration milestones (P4P/ P4R, as specified in the Metrics Specification Guide)
- Percent complete of preapproved system integration plan in the PPS project plan
- For HH population, % in O/E; % in Active Care Management; % with Care Plan

Additional project-specific Domain 1 metrics:

Additional project-specific metrics, established by the state or CMS for a particular project, especially "at risk" projects. (Pay for performance, i.e. achievement of corrective action as specified by the state or CMS for "at risk" projects) The state´s independent assessor will develop a rubric for assessing semi-annual reports, workforce milestones, and system integration milestones to identify at risk projects.

Domain 2. System Transformation Metrics

All Domain 2 metrics are pay-for-reporting in DY 1 and 2. As described below, some metrics become pay-for-performance in DY 3-5. All of these metrics will be assessed on a statewide level as part of the statewide Domain 2 performance test described in STC 14.g.i in section VII, with the exception of the Medicaid spending metric and the provider reimbursement metric and (which are included as part of other statewide accountability tests described in STC 14.g.iii and 14.g.iv in section VII respectively).

Domain 2 – System Transformation Metrics
			DSRIP Year 2	DSRIP Years 3 – 5
State– wide Measure	Measure Name	Measure Steward	Pay for Reporting/Pay for Performance	Pay for Reporting/Pay for Performance
A. Create Integrated Delivery System
Potentially Avoidable Services
X	Potentially Preventable Emergency Room Visits	3M	Reporting	Performance
X	Potentially Preventable Readmissions	3M	Reporting	Performance
X	PQI 90 – Composite of all measures	AHRQ	Reporting	Performance
X	PDI 90 – Composite of all measures	AHRQ	Reporting	Performance
Provider Reimbursement
	Percent of total Medicaid provider reimbursement received through sub–capitation or other forms of non–FFS reimbursement		Reporting	Reporting
System Integration
X	Percent of Eligible Providers meeting Meaningful Use criteria, who have participating agreements with qualified entities (RHIOS) and are able to participate in bidirectional exchange		Reporting	Reporting
Primary Care
X	Percent of PCP meeting PCMH (NCQA)/ Advance Primary Care (SHIP)		Reporting	Reporting
X	CG CAHPS Measures including usual source of care Patient Loyalty (Is doctor/clinic named the place you usually go for care? How long have you gone to this doctor/clinic for care?)	AHRQ	Reporting	Performance
Access to Care
X	HEDIS Access/Availability of Care (Adult Access to Preventive or Ambulatory Care, Children´s Access to Primary Care )	NCQA	Reporting	Performance
X	CG CAHPS Measures: – Getting Timely Appointments, Care and Information	AHRQ	Reporting	Performance
Medicaid Spending for Projects Defined Population on a PMPM Basis
	Medicaid spending on ER and Inpatient Services		Reporting	Reporting
	Medicaid spending on PC and community-based behavioral health care		Reporting	Reporting

B. Implementation of care coordination and transitional care programs
Performing Provider Systems will be required to meet all of the above metrics with the addition of the following:
Care Transitions
	H–CAHPS – Care Transition Metrics	AHRQ	Reporting	Performance
X	CG CAHPS Care Coordination composite	AHRQ	Reporting	Performance
C. Connecting Settings
Performing Provider Systems will be required to meet all of the above metrics for A and B.
D. Utilizing Patient Activation to Expand Access to Community Based Care for Special Populations
	Interval change of the mean in Patient Activation Measure® (PAM®) –scores among eligible members by weighted project cohorts compared to baseline year.	Insignia Health	Reporting	Performance
	Non–use of Primary and Preventive Care Services	NYS	Reporting	Performance
	Emergency department use by uninsured persons as measured by percent of emergency room visits among individuals that are uninsured in the measurement year compared to same in baseline year.	NYS	Reporting	Performance
	CG CAHPS done by PPS documenting the uninsured population experience with the health care system	AHRQ	Reporting	Performance

Domain 3. Clinical Improvement Metrics

All Domain 3 metrics are pay-for-reporting in DY 1. As described below, some metrics continue as pay-for-reporting in DY 2-3 but become pay-for-performance in DY 4-5. In general, provider systems will include all metrics associated with the project selected, unless otherwise specified below. Metrics will be reported annually. In the event that a measure is removed from a project (such as if the measure is retired by the steward), the State and CMS reserve the right to introduce a replacement measure for the project. PPSs conducting the project will be notified of the removal or replacement.

Domain 3 - Clinical Improvement Metrics
					DSRIP Years 2 - 3	DSRIP Years 4 - 5
Measure Name	Measure Steward	NQF#	Source	Measure Type	Pay for Reporting/Pay for Performance	Pay for Reporting/Pay for Performance
A. Behavioral Health (Required) - All behavioral health projects will use the same metrics except for SNF programs implementing the BIP in Nursing Homes project. These providers will include the additional behavioral health measures below in A-2.
Potentially Preventable Emergency Room Visits (for persons with BH diagnosis)	3M		Claims	Outcome	Performance	Performance
Antidepressant Medication Management	NCQA	105	Claims	Process	Performance	Performance
Diabetes Monitoring for People with Diabetes and Schizophrenia	NCQA	1934	Claims	Process	Performance	Performance
Diabetes Screening for People with Schizophrenia or Bipolar Disease Using Antipsychotic Medication	NCQA	1932	Claims	Process	Performance	Performance
Cardiovascular Monitoring for People with Cardiovascular Disease and Schizophrenia.	NCQA	1933	Claims	Process	Performance	Performance
Follow-up care for Children Prescribed ADHD Medications	NCQA	103	Claims	Process	Reporting	Performance
Follow-up after hospitalization for Mental Illness	NCQA	576	Claims	Process	Performance	Performance
Screening for Clinical Depression and follow-up	CMA	418	Medical Record	Process	Reporting	Performance
Adherence to Antipsychotic Medications for People with Schizophrenia	NCQA	1879	Claims	Process	Performance	Performance
Initiation and Engagement of Alcohol and Other Drug Dependence Treatment	NCQA	4	Claims	Process	Performance	Performance
A - 2. Additional behavioral health measures for provider systems implementing the Behavioral Interventions Paradigm (BIP) in Nursing Homes project
Antipsychotic Use in Persons with Dementia for Long Stay Residents	3M		MDS 3.0	Outcome	Performance	Performance
Percent of Long Stay Residents who have Depressive Symptoms	CMS		MDS 3.0	Process	Performance	Performance
B. Cardiovascular Disease
PQI # 7 (HTN)	AHRQ		Claims	Outcome	Performance	Performance
PQI # 8 (Heart Failure)	AHRQ		Claims	Outcome	Performance	Performance
Statin Therapy for Patients with Cardiovascular Disease	NCQA		Medical Record	Process	Reporting	Performance
Controlling High Blood Pressure	NCQA	18	Medical Record	Outcome	Reporting	Performance
Aspirin Discussion and Use	CAHPS		Survey	Process	Reporting	Performance
Medical Assistance with Smoking and Tobacco Use Cessation	NCQA	27	Survey	Process	Reporting	Performance
Flu Shots for Adults Ages 50 - 64	NCQA	39	Survey	Process	Reporting	Performance
Health Literacy Items (includes understanding of instructions to manage chronic condition, ability to carry out the instructions and instruction about when to return to the doctor if condition gets worse	CAHPS		Survey	Process	Reporting	Performance
C. Diabetes Mellitus
PQI # 1 (DM Short term complications)	AHRQ	274	Claims	Outcome	Performance	Performance
Comprehensive Diabetes screening (HbA1c, dilated eye exam, nephropathy)	NCQA		Medical Record	Process	Reporting	Performance
Comprehensive Diabetes Care: Hemoglobin A1c (HbA1c) Poor Control (>9.0%)	NCQA	59	Medical Record	Outcome	Reporting	Performance
Medical Assistance with Smoking and Tobacco Use Cessation	NCQA	27	Survey	Process	Reporting	Performance
Flu Shots for Adults Ages 50 - 64	NCQA	39	Survey	Process	Reporting	Performance
Health Literacy Items (includes understanding of instructions to manage chronic condition, ability to carry out the instructions and instruction about when to return to the	CAHPS		Survey	Process	Reporting	Performance
D. Asthma
PQI # 15 Adult Asthma	AHRQ	283	Claims	Outcome	Performance	Performance
PDI # 14 Pediatric Asthma	AHRQ	638	Claims	Outcome	Performance	Performance
Asthma Medication Ratio	NCQA	1800	Claims	Process	Performance	Performance
Medication Management for People with Asthma	NCQA	1799	Claims	Process	Performance	Performance
E. HIV/AIDS
HIV/AIDS Comprehensive Care: Engaged in Care	NYS		Claims	Process	Performance	Performance
HIV/AIDS Comprehensive Care: Viral Load Monitoring	NYS		Claims	Process	Performance	Performance
HIV/AIDS Comprehensive Care: Syphilis Screening	NYS		Claims	Process	Performance	Performance
Cervical Cancer Screening	NCQA	32	Claims	Process	Reporting	Performance
Chlamydia Screening	NCQA	33	Claims	Process	Performance	Performance
Medical Assistance with Smoking and Tobacco Use Cessation	NCQA/	27	Survey	Process	Reporting	Performance
Viral Load Suppression	HRSA	2082	Medical Record	Outcome	Reporting	Performance
F. Perinatal Care
PQI # 9 Low Birth Weight	AHRQ	278	Claims	Outcome	Performance	Performance
Prenatal and Postpartum Care—Timeliness and Postpartum Visits	NCQA	1517	Medical Record	Process	Reporting	Performance
Frequency of Ongoing Prenatal Care	NCQA	1391	Medical Record	Process	Reporting	Performance
Well Care Visits in the first 15 months	NCQA	1392	Claims	Process	Reporting	Performance
Childhood Immunization Status	NCQA	38	Medical Record	Process	Reporting	Performance
Lead Screening in Children	NCQA		Medical Record	Process	Reporting	Performance
PC-01 Early Elective Deliveries	Joint Commission	469	Medical Record	Process	Reporting	Reporting
G. Palliative Care - All projects will use the same metric set.
Percentage of patients indicating need who were offered or provided an intervention for pain symptoms experienced during the past week	NYS		IPOS	Process	Reporting	Performance
Percentage of patients indicating need who were offered or provided an intervention for physical symptoms (other than pain) experienced during the past week	NYS		IPOS	Process	Reporting	Performance
Percentage of patients indicating need who were offered or provided an intervention for not feeling at peace during the past week	NYS		IPOS	Process	Reporting	Performance
Percentage of patients indicating need who were offered or provided an intervention for depressive feelings experienced during the past week	NYS		IPOS	Process	Reporting	Performance
Percentage of patients who were offered or provided an intervention when there was no advance directive in place	NYS		IPOS	Process	Reporting	Performance
H. Renal Care
Comprehensive Diabetes screening (HbA1c, dilated eye exam, nephropathy)	NCQA		Medical Record	Process	Reporting	Performance
Comprehensive Diabetes Care: Hemoglobin A1c (HbA1c) Poor Control (>9.0%)	NCQA	59	Medical Record	Outcome	Reporting	Performance
Annual Monitoring for Patients on Persistent Medications - ACE/ARB	NCQA		Claims	Process	Reporting	Performance
Controlling High Blood Pressure	NCQA	18	Medical Record	Outcome	Reporting	Performance
Flu vaccine 18-64	NCQA	39			Reporting	Performance
Medical Assistance with Smoking and Tobacco Use Cessation	NCQA	27			Reporting	Performance

Domain 4. Population-Wide Metrics

This domain includes pay-for-reporting for relevant measures from the New York State Prevention Agenda related to the Domain 4 projects selected. All Domain 4 metrics will be measured by a geographical area denominator of all New York State residents that New York State has already developed for the Prevention Agenda. Some metrics are not collected on an annual basis but will be reported on their usual collection cycle. For example, the Behavioral Risk Factor Surveillance System (BRFSS) is done biannually.

The metrics that are part of the New York State Prevention Agenda are available here and will be further described in the metric specification guide.

		Source	Geographic Granularity
Improve Health Status and Reduce Health Disparities (required for all projects)
1.	Percentage of premature death (before age 65 years)	NYS NYSDOH Vital Statistics	State, County
2.	Ratio of Black non-Hispanics to White non-Hispanics
3.	Ratio of Hispanics to White non-Hispanics
4.	Age-adjusted preventable hospitalizations rate per 10,000 - Aged 18+ years	SPARCS	Statewide Region County
5.	Ratio of Black non-Hispanics to White non-Hispanics
6.	Ratio of Hispanics to White non-Hispanics
7.	Percentage of adults with health insurance - Aged 18-64 years	US Census
8.	Age-adjusted percentage of adults who have a regular health care provider - Aged 18+ years	eBRFSS	Statewide NYC/ROS County
Promote Mental Health and Prevention Substance Abuse
8.	Age-adjusted percentage of adults with poor mental health for 14 or more days in the last month	eBRFSS	Statewide NYC/ROS County
9.	Age-adjusted percentage of adult binge drinking during the past month	eBRFSS	Statewide NYC/ROS County
10.	Age-adjusted suicide death rate per 100,000	NYS NYSDOH Vital Statistics	State, county
Prevent Chronic Diseases
11.	Percentage of adults who are obese	eBRFSS	Statewide NYC/ROS County
12.	Percentage of children and adolescents who are obese	NYS excluding NYC: - Student Weight Status Category Reporting; NYC - Fitness gram	Statewide NYC/ROS County
13.	Percentage of cigarette smoking among adults	eBRFSS	Statewide NYC/ROS County
14.	Percentage of adults who receive a colorectal cancer screening based on the most recent guidelines - Aged 50-75 years	eBRFSS	Statewide
15.	Asthma emergency department visit rate per 10,000	SPARCS	Statewide Region County
16.	Asthma emergency department visit rate per 10,000 - Aged 0-4 years	SPARCS	Statewide Region County
17.	Age-adjusted heart attack hospitalization rate per 10,000	SPARCS	Statewide Region County
18.	Rate of hospitalizations for short-term complications of diabetes per 10,000 - Aged 6-17 years	SPARCS	Statewide Region County
19.	Rate of hospitalizations for short-term complications of diabetes per 10,000 - Aged 18+ years	SPARCS	Statewide Region County
Prevent HIV/STDs
20.	Newly diagnosed HIV case rate per 100,000	NYS HIV Surveillance System
21.	Difference in rates (Black and White) of new HIV diagnoses
22.	Difference in rates (Hispanic and White) of new HIV diagnoses
23.	Gonorrhea case rate per 100,000 women - Aged 15-44 years	NYS STD Surveillance System
24.	Gonorrhea case rate per 100,000 men - Aged 15-44 years	NYS STD Surveillance System
25.	Chlamydia case rate per 100,000 women - Aged 15-44 years	NYS STD Surveillance System
26.	Primary and secondary syphilis case rate per 100,000 males	NYS STD Surveillance System
27.	Primary and secondary syphilis case rate per 100,000 females	NYS STD Surveillance System
Promote Healthy Women, Infants, and Children
28.	Percentage of preterm births	NYS NYSDOH Vital Statistics	State, County
29.	Ratio of Black non-Hispanics to White non-Hispanics
30.	Ratio of Hispanics to White non-Hispanics
31.	Ratio of Medicaid births to non-Medicaid births
45.	Percentage of infants exclusively breastfed in the hospital	NYS NYSDOH Vital Statistics	State, County
46.	Ratio of Black non-Hispanics to White non-Hispanics
47.	Ratio of Hispanics to White non-Hispanics
48.	Ratio of Medicaid births to non-Medicaid births
49.	Maternal mortality rate per 100,000 births	NYS NYSDOH Vital Statistics	State, County
54.	Percentage of children with any kind of health insurance - Aged under 19 years	U.S. Census	State, County
57.	Adolescent pregnancy rate per 1,000 females - Aged 15-17 years	NYS NYSDOH Vital Statistics	State, County
58.	Ratio of Black non-Hispanics to White non-Hispanics
59.	Ratio of Hispanics to White non-Hispanics
60.	Percentage of unintended pregnancy among live births	NYSDOH Vital Statistics	State
61.	Ratio of Black non-Hispanics to White non-Hispanics
62.	Ratio of Hispanics to White non-Hispanics
63.	Ratio of Medicaid births to non-Medicaid births
64.	Percentage of women with health coverage - Aged 18-64 years	U.S. Census	State, County

|top of section| |top of page|

ATTACHMENT K

DSRIP Operational Protocol

I. Delivery System Reform Incentive Payment (DSRIP) Program Background

On April 14, 2014 Governor Andrew M. Cuomo announced that New York had finalized Special Terms and Conditions (STCs) with the federal government for a groundbreaking waiver that would allow the New York State Department of Health (DOH) to reinvest $8 billion in federal savings generated by Medicaid Redesign Team (MRT) reforms. The STCs will serve as the governing agreement between DOH and the Centers for Medicare and & Medicaid Services (CMS) under the 1115 Waiver, also formally the Partnership Plan. The STCs outline the implementation of MRT waiver amendment programs, authorized funding sources and uses, and other requirements.

The waiver amendment dollars will address critical issues throughout the State and allow for comprehensive reform through a Delivery System Reform Incentive Payment (DSRIP) program. The DSRIP program will promote community-level collaborations and focus on system reform, specifically a goal to achieve a 25% reduction in avoidable hospital use over five years. Safety net providers will be required to collaborate to implement innovative projects focusing on system transformation, clinical improvement and population health improvement. All DSRIP funds will be awarded based on performance linked to achievement of project milestones.

The $8 billion federal financial participation is allocated in the following ways:

$500 million for the Interim Access Assurance Fund (IAAF) - temporary, time limited funding to ensure current trusted and viable Medicaid safety net providers can fully participate in the DSRIP transformation without disruption to the delivery of key health services
$6.42 billion for (DSRIP) - including DSRIP Design Grants, the DSRIP Performance Fund including, DSRIP Provider Incentive Payments, and DSRIP Administrative costs
$1.08 billion for other Medicaid Redesign purposes - this funding will support Health Home development, and investments in long-term care, workforce and enhanced behavioral health services

In addition, the STCs commit DOH to comprehensive payment reform and to continue New York´s efforts to effectively manage its Medicaid program within the confines of the Medicaid Global Spending Cap.

II. Executive Summary

This document, Attachment K: DSRIP Operational Protocol provides an overview of monitoring and implementation activities and requirements of stakeholders engaging in the administrative, support, technical and design activities aligned with DSRIP goals and objectives. Through monitoring, the State expects to: (1) recognize and resolve operational barriers immediately (2) quickly identify the State´s needs for technical assistance; and (3) assure program integrity and accountability.

The sections of this document have been designed to capture all of the core elements required in the implementation and ongoing operation of the DSRIP program. Brief descriptions of the content of each section is provided below.

Section III, "DSRIP Timeline" highlights the key dates and corresponding activities or events associated with each period for program evaluation. The DSRIP Timeline supports both public and internal timelines associated with key deliverables.

Section IV, "DSRIP Project Team: Key Staff and Responsibilities" highlights the various State agencies and contracted entities engaged in the DSRIP program. The section also identifies key staff for the agencies or entities that are primarily engaged in the implementation and operational activities supporting the DSRIP program.

Section V, "Provider Requirements" highlights some of the main requirements for the Performing Provider Systems (PPS) and the tools available to the PPS for the purposes of data sharing with the State.

Section VI, "Performance Payments" highlights the two mechanisms through which PPS are eligible to receive payments from the DSRIP Performance Fund including the DSRIP High Performance Fund.

Section VII, "PPS Support" identifies the resources in place to provide PPS with ongoing support of planning and implementation activities throughout the term of the waiver.

Section VIII, "Statewide Measurements and Accountability" highlights the measures and processes that are in place to evaluate the performance of the State against required milestones as well as the State´s efforts around reporting and transparency.

Section IX, "Independent Evaluator" describes the roles and responsibilities of the State in procuring an independent evaluator including the roles, process and responsibilities of constructing the evaluation design.

Section X, "Implementation Activities" identifies the resources provided to the PPS to support their planning efforts to successfully submit DSRIP Project Plan Applications.

DSRIP Timeline

The DSRIP timeline is defined by DSRIP Years (DY) beginning with the planning and pre- implementation year, known as DY0 and covering the first year of implementation in DY1 through the end of the program in DY5. April 2014 marked the beginning of DY0 with key activities such as stakeholder education and engagement, planning activities, procurement of DSRIP contractors and development of key DSRIP applications, awards, policies and procedures occurring through March 2015. An extensive DSRIP website was launched on April 14, 2014 and is available at www.health.ny.gov/dsrip and includes a current high-level DY timeline outlining key activities.

Timelines specific to the PPS Quarterly Reports, the semi-annual performance payments, and the Independent Evaluator (IE) are included in the respective sections of this document.
DSRIP Project Team: Key Staff and Responsibilities

The DSRIP project team is comprised of staff from across multiple State agencies and from contracted vendors. The following sections highlight the various State agencies and contracted vendors engaged in the implementation and operation of the DSRIP program with a description of the role of each entity. Where applicable, key staff positions have been identified for entities that play primary roles in the implementation and ongoing operations of the DSRIP program.
1. New York State Department of Health, Office of Health Insurance Programs (OHIP)
  
  The Office of Health Insurance Programs (OHIP) is the primary State entity responsible for the DSRIP program. Led by Jason Helgerson, Medicaid Director and Deputy Commissioner for DOH, with support from Greg Allen, Director for the Division of Program Development and Management and Peggy Chan, the DSRIP Program Director. OHIP is the primary liaison for the State with CMS. OHIP is responsible for all communications with CMS with regards to the DSRIP program including the negotiations for waiver approval, the approval of the STCs and attachments, and any policy or funding questions related to the program. OHIP serves as the primary entity responsible for policy decisions related to the implementation and operation of the DSRIP program.
  
  OHIP has been responsible for the procurement of the services of vendors to support the DSRIP program including the DSRIP Independent Assessor and the DSRIP Support Team, as well as supporting the procurement of the DSRIP Independent Evaluator. OHIP staff provide contract management and oversight for the selected Independent Assessor and Support Team vendors.
  
  OHIP also serves as the primary point of contact for all DSRIP related matters with other State agencies. This includes other entities within DOH, such as the Office of Primary Care and Health Systems Management (OPCHSM), the Office of Quality and Patient Safety (OQPS), and the Office of Public Health (OPH), as well as agencies outside of DOH such as the Office of Mental Health (OMH), the Office of Alcohol and Substance Abuse Services (OASAS), the Office for People with Developmental Disabilities (OPWDD), and the Office of the Medicaid Inspector General (OMIG).
  
  Additionally, OHIP plays a central role with the PPS and the providers participating in DSRIP. OHIP formulates programmatic and operational policy to guide the PPSs in implementation. Regular communication with PPSs and other DSRIP stakeholders are coordinated through OHIP. OHIP monitors the progress, performance and activities of the PPSs through direct engagement as well as through its contractors as described below. Technical assistance or other programs are provided by OHIP as needed to assist in successful implementation of DSRIP goals. Other efforts include engagement with the PPS and providers on specific topics that arise throughout the life of the DSRIP program as well as supporting PPS through Enhanced Support and Oversight (ESO) activities to assist PPS in meeting program goals and milestones.
2. Other NYSDOH Divisions
  1. Office of Primary Care and Health Systems Management (OPCHSM)
    
    The Office of Primary Care and Health Systems Management (OPCHSM) regulates and oversees the health care system of New York State to ensure access to high quality, affordable, and equitable health care services for all who receive care from New York´s health care providers and facilities. Core functions include health care policy and standards development; data analysis and health services research; health care facility planning, financing, and licensure; health care workforce development and oversight of graduate medical education; health care provider licensure, certification, surveillance, investigation, and enforcement; and the implementation of federal and State health care system reform. The regulated health care community includes hospitals, diagnostic and treatment centers, nursing homes, adult care facilities, intermediate care facilities, home and community-based care providers, emergency medical service providers, primary care providers, pharmacists, funeral directors, and physicians. Key areas of coordination with the DSRIP program are in NYS Regulatory Waivers for DSRIP projects. PPS are able to submit requests for specific waivers in State regulations that support the implementation of DSRIP projects and goals. OPCHSM reviews the requests and coordinates with other DOH divisions and State agencies for an agreed upon response. Upon approval, licenses and surveillance criteria are revised accordingly.
  2. Office of Quality and Patient Safety (OQPS)
    
    The Office of Quality and Patient Safety (OQPS) is focused on improving the health, quality of care and patient safety for New York State residents, consistent with the State´s Strategic Plan and the MRT recommendations. OQPS is responsible for key coordination within the DSRIP program including; ensuring consistency in data management, testing data validity and, ensuring consistency between DSRIP Metrics and federal and State clinical measurements. OQPS is the key developer for the calculation of the performance metrics for the DSRIP program. OQPS has formulated the metrics specification guide for the DSRIP program and runs the performance data for each PPS.
    
    OQPS is also responsible for the procurement and managing the contract of the Independent Evaluator.
  3. Office of Public Health (OPH)
    
    The Office of Public Health (OPH) will bring evidence-based public health interventions to the DSRIP process, and ultimately to the DSRIP projects, to improve health outcomes and specifically decrease emergency room utilization, reduce hospital readmissions and reduce hospitalizations. OPH´s role is especially important with respect to secondary prevention and ambulatory sensitive conditions, since community components have been shown to positively impact these health outcomes. In addition, the OPH has capacity to offer support in some facility based health outcome areas including healthcare acquired infection, flu immunization and maternal morbidity and mortality.
3. Other State Agencies
  1. Office of Mental Health (OMH)
    
    The Office of Mental Health (OMH) oversees and regulates mental health services in the State. It promotes the mental health and well-being of all New Yorkers with a mission to facilitate recovery for young to older adults receiving treatment for serious mental illness, to support children and families in their social and emotional development and early identification and treatment of serious emotional disturbances, and to improve the capacity of communities across New York to achieve these goals. OMH will support the DSRIP program by providing technical assistance and guidance related to best practices for the behavioral health projects during project development and implementation as well as supporting the review of any regulatory waiver requests submitted by PPS related to behavioral health services or providers.
  2. Office of Alcoholism and Substance Abuse Services (OASAS)
    
    The Office of Alcoholism and Substance Abuse Services (OASAS) oversees one of the nation largest addiction service systems with nearly 1,600 prevention, treatment and recovery programs. Its mission is to improve the lives of New Yorkers by leading a comprehensive, premier system of addiction services for prevention, treatment, and recovery. OASAS will support the DSRIP program by providing technical assistance and guidance related to best practices for the alcohol and substance abuse related projects during project development and implementation as well as supporting the review of any regulatory waiver requests submitted by PPS related to alcohol and substance abuse related services or providers.
  3. Office for People with Developmental Disabilities (OPWDD)
    
    The Office for People with Developmental Disabilities (OPWDD) coordinates services for more than 126,000 New Yorkers with intellectual and developmental disabilities. OPWDD will support the DSRIP program by providing technical assistance and guidance as well as supporting the review of any regulatory waiver requests submitted by PPS related to developmental disability services or providers.
  4. Office of the Medicaid Inspector General (OMIG)
    
    OMIG´s overall mission is to enhance the integrity of the New York State Medicaid program by preventing and detecting fraudulent, abusive, and wasteful practices within the Medicaid program and recovering improperly expended Medicaid funds while promoting high quality patient care. OHIP has been actively working with the OMIG regarding assessing risks and respective roles for the DSRIP program. Among the key areas identified are:
    1. PPS Provider networks that drive the Project Plan valuation
    2. Medicaid compliance programs for DSRIP funds
    During the initial PPS network development efforts OMIG required the PPS to attest that the providers included in their network had affirmatively agreed to be in the network and that there was evidence of such agreement. OMIG conducted audits among a selected sample of the PPS providers to solicit the evidence for including the providers. In addition, OMIG electronically reviewed the PPS networks to identify ineligible providers. OMIG provided OHIP with their findings to allow OHIP to notify the affected PPS. Reductions to PPS networks due to insufficient documentation of agreements between the PPS and proposed network providers. A small number of ineligible providers were also excluded as a result of this effort.
    
    OMIG will continue to conduct periodic reviews of the PPS networks to ensure the integrity of the networks and to ensure that ineligible providers are identified and excluded from the DSRIP program.
    
    Another area under OMIG jurisdiction has been the Medicaid Compliance Program. OMIG and OHIP recognize that the DSRIP incentive payment system for performance is different from the traditional Medicaid program. OMIG and OHIP have coordinated communications regarding expectations of the PPS lead entities in their Medicaid compliance programs as it pertains to the use of DSRIP funds which are incentive payments. OMIG has provided guidance documents to PPS leads regarding compliance program structure, assessment of risk and other key elements to protect against fraud and abuse.
    
    OMIG will be reviewing the PPS for their Medicaid Compliance Programs as required under NYS Social Services Law 363.d. OMIG will also be receiving quarterly funds flow reports from OHIP and the Independent Assessor that will show the amount of DSRIP performance payments received by each participating provider. This data will be used in conjunction with other Medicaid payment data such as fee-for-service and managed care payments to determine the individual providers that are required to establish a Medicaid Compliance Program.
4. Vendors and Contractors
  1. DSRIP Independent Assessor (IA)
    
    New York released a Funding Availability Solicitation (FAS) for the purpose of procuring the services of an entity to serve as the DSRIP IA on May 20, 2014. Through the FAS procurement process, New York selected Public Consulting Group (PCG) to serve as the DSRIP (IA). Notification of the award was made on July 31, 2014 and PCG began work on August 4, 2014 and will continue through the end of the DSRIP demonstration.
    
    The FAS and resulting contract contains not only the activities of the IA but additional activities to be performed by PCG in support of the DSRIP program. The PCG team has been separated into two primary teams; the IA team and the Program Support team. The two teams have been created to fulfill all requirements of the contract while maintaining the independence of the IA team with a single Engagement Manager responsible for the oversight of all DSRIP deliverables.
    
    The IA team is responsible for conducting the independent reviews of PPS quarterly reports for progress towards meeting program milestones and goals and for determining the semi-annual performance payments the PPS are eligible to receive. The IA team is also responsible for conducting the Mid- Point Assessment.
    
    The Program Support team includes all functions and responsibilities defined in DOH´s contract with PCG outside of the IA functions. The Program Support team functions include the following:
    1. Account Support Team (AST): the AST is responsible for working directly with the PPS and to support PPS implementation and quality improvement efforts.
    2. Performance Management team: the Performance Management team works directly with OHIP and OQPS on the calculation of the annual claims and non-claims based performance measures. This team also supports OQPS in the annual reviews of the Measure Specification Guide.
    3. Medicaid Analytics Performance Portal (MAPP) team: the MAPP team is responsible for working with the State´s IT vendors in the development of the web-based platform to support PPS completion of the PPS Quarterly Reports.
    4. Certificate of Public Advantage (COPA)/Accountable Care Organization (ACO) team: the COPA/ACO team is responsible for supporting DOH in the review of all COPA and ACO applications submitted by the PPS or entities participating in DSRIP.
    5. Learning Symposium team: the Learning Symposium team is responsible for conducting the annual and regional Learning Symposia, including the identification of sites and the development of the agenda.
    6. Health Homes team: the Health Homes team is responsible for supporting DOH in ensuring the integration of Health Homes into DSRIP projects.
    7. DSRIP Director Support team: the DSRIP Director Support team is responsible for assisting the DSRIP Director on various DSRIP program efforts.
    The two functional areas are separated by a firewall to ensure the independence of the IA team in reviewing the PPS progress and achievement. The DSRIP Independent Assessor Firewall Policies and Procedures defines the policies that govern the interaction of the PCG Program Support team with the IA team. Compliance with the Firewall Policies and Procedures is monitored through monthly meetings between DOH and the PCG Engagement Manager. A summary of the firewall policies has been included as Appendix A to this Operational Protocol once finalized by PCG and DOH.
  2. DSRIP Support Team (DST)
    
    New York released a FAS for the purpose of procuring the services of a vendor to serve as the DST on May 21, 2014. Through the FAS procurement process, New York selected KPMG to serve as the DST. KPMG began work on August 1, 2014 and will continue through July 31, 2017.
    
    Through the DSRIP Project Plan Application process, the DST assisted the PPS in completing the Project Plan Application as well as to understand the expectations of the IA in evaluating the applications. Following the completion of the application process, the DST continued to support the PPS through the development of Implementation Plans and provided other account management and assistance. The support team functions to the PPS was transitioned to PCG AST, who provides more focus on project performance improvement and assistance, as of March 1, 2016.
    
    The DST continues to support the DSRIP program through the provision of subject matter expertise on certain elements of DSRIP program implementation: VBP, data analytics, PPS IT and performance strategies, rapid cycle evaluation and project management initiatives to DSRIP Policy and Program Directors
  3. Medicaid and Data Portal Contractors
    
    NYSTEC, CMA, IBM and Salient have longstanding experience working in support of the NYS Medicaid program. These contractors will apply experience in this environment to develop and maintain DSRIP IT infrastructure to support PPS reporting requirements, Medicaid Data and Analytics, public resources and DSRIP applications.
5. DSRIP Project Approval and Oversight Panel (PAOP)
  
  Based on the standards set forth in the STCs, the IA was tasked with convening the DSRIP Project Approval and Oversight Panel (PAOP). The PAOP was carefully selected to ensure there were no existing conflict of interests with any participating PPS. The PAOP is comprised of 27 members, which include healthcare professionals, consumers, Commissioners from State Agencies and members designated by the NYS Assembly and Senate.
  
  The role of the PAOP was to initially review the determinations of the IA scoring results of each PPS project application and to make recommendations to the State Commissioner for the Department of Health for any modifications. Subsequent to the application process, the PAOP role is to provide ongoing oversight and review of PPS progress. The PAOP will conduct annual reviews of PPS progress based on the results of the PPS Quarterly Reports. The PAOP will convene in public meetings in an effort to identify successes as well as barriers to project implementation.
  
  At the Mid-Point Assessment, the PAOP will be responsible for reviewing the recommendations for DSRIP Project Plan modifications made by the IA and for making recommendations to the State Commissioner for the Department of Health.
  
  Additional details on PAOP´s training, activity and charge are available here.

Provider Requirements

Eligible Providers for DSRIP Participation

The STCs set forth requirements for participation in DSRIP and specifically for PPS lead organizations. STC VII, Section 1.c states that "DSRIP funding is available to Performing Provider Systems that consist of safety net providers whose project plans are approved and funded through the process described in these STCs and who meet particular milestones described in their approved DSRIP Project Plans." The PPS lead organizations can engage and contract with other safety net and non-safety net providers to implement and execute the approved DSRIP Project Plans, however per STC requirements, funding to non-safety net providers is limited to no more than five percent of the total DSRIP Project Plan funding.
Potential PPS Letter of Intent Process

The State conducted a non-binding PPS Letter of Intent process to help identify providers across the state that could serve as PPS lead organizations. The Letter of Intent submission required the potential PPS lead organization to identify the proposed county or counties the PPS planned to serve through DSRIP and a list of the potential providers the PPS lead organization expected to include in their PPS network. DOH reviewed all letters of intent and identified those potential PPS lead organizations that could become a PPS as an ‘emerging PPS´. Providers that did not receive ‘emerging PPS´ status were informed that they would need to consider joining with another PPS to participate in DSRIP.
Safety Net Determination and Appeals Process

As noted in STC VII, Section 2, there are specific criteria outlined for determining the safety net status of providers participating in DSRIP. Hospitals are expected to meet one of three criteria to qualify as a safety net provider while non-hospital providers that are not participating as part of a state-designated health home, are required to meet a single criteria to qualify as a safety net provider. DOH used Medicaid claims and encounter data as the primary sources for determining the safety net status of hospital and non-hospital providers.

Following the initial safety net determination process, providers that did not receive safety net designation were given an opportunity to appeal the safety net determination. The appeals process allowed for providers to submit a narrative explaining why the provider believes they meet the safety net definition along with revised data to support the appeal in how the provider meets the safety criteria. DOH reviewed the safety net appeals and made determinations based on the narrative and data provided through the appeals.

Providers could also request a Vital Access Provider (VAP) exception to the safety net definition. The VAP exception was considered on a case-by-case basis if it was deemed in the best interest of Medicaid members. DOH allowed a VAP exception to the safety net definition based on one of three criteria. All VAP exceptions required CMS approval and were subject to a 30-day public comment period prior to final approval.

Details on the safety net determinations and the VAP exceptions can be found on the DSRIP website here.
New Corporation (NewCo) VAP Exception Process

During the DSRIP Demonstration Period, PPS Lead entities may decide to pursue different corporate structures to facilitate DSRIP implementation. For this purpose, the DOH will permit current PPS Lead entities ONLY to submit new corporation VAP exception applications for safety net designation. PPS may submit to DOH for consideration their proposal and rationale for the new legal corporation. Applications for NewCo VAP exceptions will be reviewed by DOH and then posted for 30-day public comment before submitting to CMS for review and approval.

Information regarding the NewCo VAP exception process can be found on the DSRIP website here.
DSRIP Design Grants

The State, as defined in STC VII, Section 10.a, was given authority to provide DSRIP Design Grants to providers. The DSRIP Design Grant funds were made available to providers to assist in the preparation for and the development of the DSRIP Project Plans. Providers were required to submit an application, including a proposed budget for the use of Design Grant funds, for review by DOH. All applications were reviewed, and initial awards were made by DOH to providers with the requirement that awardees submit a DSRIP Project Plan application. Following the approval of DSRIP Project Plan applications, DOH made supplemental Design Grant awards to the approved PPS to assist in the development of implementation plans.

In order to ensure that the Design Grant funds were used in accordance with the requirements set forth in the STCs and in the terms of the award letters, the IA conducted an audit of the Design Grant awards. The audits were conducted on all Design Grant awards, including the initial and supplemental awards, with a final report on the findings to be delivered to DOH.

Information regarding the DSRIP Design Grants can be found on the DSRIP website here.
DSRIP Project Plan Application and Award Process

The State, along with various stakeholders and vendors, collaborated on efforts to support the development, submission and evaluation of DSRIP PPS Project Plan Applications. Since beginning work on August 4, 2014, the IA worked extensively on the development of the DSRIP Project Plan Application, the Project Metrics and Milestones, Scoring Guide, and PPS Lead Financial Stability Test. The IA worked closely with DOH staff to ensure the DSRIP Project Plan and all accompanying documents have been consistent with the STCs, Attachments I and J, and the DSRIP Project Toolkit.

The following processes implemented by DOH and vendors supported the Project Plan Application and Award Process:
1. Release of the Project Plans Application for public comment
2. Review and revised Project Plan Application based on public comment
3. Creation of an application and application review tool as well as a process for a transparent and impartial review of all proposed Project Plans
4. Assembling a team of IA´s to review and score the Project Plan Applications
5. Development of DSRIP Project Plan prototypes, "how to" guides and other tools to help providers as they prepare their Project Plan applications
6. Creating and Monitoring Centralized Storage and Retrieval of Deliverables
7. Making project approval recommendations to the State using CMS-approved criteria
8. On the ground support to PPS from shortly after DSRIP Design Grant awards until final submission of their Project Plan applications
9. Public release of Project Plan Application along with IA´s evaluation scores
10. Assembling the DSRIP PAOP, an independent review panel chosen by DOH based on standards set forth in the DSRIP STCs
11. Conducting public meeting for the review of the IA´s scoring recommendations
Completed DSRIP Project Plan applications were due from the PPS on December 22, 2014. DSRIP Project Plan applications were received from 25 PPS.

The IA conducted an initial completeness review by December 24, 2014 and began the comprehensive evaluation of the Project Plans on December 26, 2014. The IA completed scoring of the 25 Project Plans on January 15, 2015. The DSRIP program requirements outlined by CMS require DOH to provide the opportunity for public comment on the Project Plans. Accordingly, the applications were posted to the DSRIP website and the public comment period for the Project Plans began on January 15, 2015 and ended on February 15, 2015.

Following the public comment period the State convened the PAOP for a multi-day working session where the IA reviewed each PPS application and attested to the suggested application scores. The PAOP provided a secondary review of PPS Project Plans and made recommendations to the NYS Commissioner of Health (the Commissioner) prior to final approval of Project Plans.

The PAOP´s charge included reviewing Project Plans scored by the IA to advise the Commissioner whether to accept, reject or modify the IA score recommendations.

Following the open PAOP work sessions all Project Plan scores were adjusted to reflect the PAOP´s recommendations. In March 2015, OHIP staff and the IA presented the modified Project Plan scores to the Commissioner for consideration and review. The Commissioner accepted the proposed Project Plan scores and submitted the results to CMS for acceptance. Additionally, in accordance with the STCs, DOH sent letters to each PPS describing DOH´s recommendations and transmittal to CMS.

The completed Project Plans can be found on the web here.

A summary of public comments received for the Project Plans can be found on the web here.
Provider Data Infrastructure and Implementation Process

In order to support the DSRIP program, DOH has worked to develop a comprehensive infrastructure to facilitate the sharing of data from the State to the PPS and from the PPS back to the State. Much of this capacity is housed within the Medicaid Analytics Performance Portal (MAPP) which incorporates:
1. A retail front-end to the Medicaid Data Warehouse for PPS
2. A Health Home community
3. Robust dashboard and data drilldown capabilities provided by Salient
4. Online tools available in portal technology to support DSRIP, including:
  1. DSRIP Project Plan Application
  2. Network management
  3. DSRIP Implementation Plan
  4. DSRIP Quarterly Reports
5. Health Home Business and Care Management Functionality
6. Data management and analytics to drive performance
  
  Access to MAPP is available through the State´s Health Commerce System (HCS) and PPS have the ability to request additional and vetted users with access to MAPP.
  1. Data Exchange Application & Agreement (DEAA)
    
    For the purpose of accessing Protected Health Information (PHI) provided by DOH, the PPS must follow a defined process that is intended to ensure that the data is handled in a secure fashion. The first step requires the PPS to complete a Data Exchange Application & Agreement (DEAA), the necessary DEAA Addendum and the Systems Security Plan. Once the DEAA, the DEAA Addendum and the Systems Security Plan are completed the PPS may receive PHI data from DOH.
  2. Opt-Out Process
    
    An additional step in the process for ensuring the appropriate handling of PHI is the requirement of an Opt-Out process, modeled after the Medicare ACO process, for Medicaid members. This Opt-Out process will allow Medicaid members to not have their PHI Medicaid data held by the State shared with the PPS Lead and partners. Once the Opt-Out process has been completed, PPS will be able to share the PHI received from DOH with their network partners for any beneficiaries that have not ‘opted out´.
  3. 2 Factor Authentication
    
    DOH has also implemented a 2-factor authentication process within MAPP to provide an additional level of protection for PHI. The 2 factor authentication process first requires all MAPP users to have a NYS Department of Motor Vehicle (DMV) issued identification and to subsequently register a phone number to facilitate subsequent logins to the system. For users logging into MAPP, they will receive a numeric security code through their registered phone number that will be entered to access MAPP. DOH will also be working to develop a mechanism for sharing non-PHI data that would not require the 2 factor authentication.
Monitoring and Compliance

PPS are subject to rigorous and regular reporting requirements, State and federal monitoring requirements, and site visits. A system of self-reporting and internal controls monitor routine assessment of PPS Project Plan achievement and will be maintained within MAPP. These activities occur on a Quarterly Cycle beginning in DY1 and continue throughout DY5. PPS are required to upload data to support milestone completion into the automated MAPP tool for demonstration and validation by the IA. These Self-assessments and milestone requirements are reviewed by the IA and/or are subject to site visits, sample collection and audits.

As described in the following sections, the IA will determine PPS progress toward project milestones and DSRIP goals. All measurements included within the Quarterly Reports determine PPS semi-annual award payments based off of a P4P or P4R methodology.

Quarterly Reporting and Achievement Values

On a quarterly basis, the PPS are tasked with submitting a quarterly report to the DSRIP IA for review and approval. The quarterly reports are used to document the progress PPS have made towards meeting prescribed organizational and project milestones that have been identified as core measures for successfully implementing DSRIP projects. These organizational and project milestones are used in determining the Domain 1 funding a PPS is eligible to earn based on the review by the IA. The IA reviews the reports to determine that the PPS is making sufficient progress towards meeting prescribed milestones and once a milestone has been met, to determine that the supporting documentation submitted supports that the milestone has been met.

Following the completion of the review by the IA, the PPS are given an opportunity to address any issues identified by the IA during a remediation period with the IA reviewing any updated information provided by the PPS before making a final determination on the quarterly report. Once the IA has completed its final review and approval of the quarterly reports, the PPS are awarded Achievement Values (AVs) to reflect their performance for that period as defined in Attachment I of the STCs. On a semi-annual basis, the AVs are used to determine the amount of DSRIP performance payments the PPS have earned.

The quarterly reporting schedule will follow a timeline as outlined below based on the guidance in Attachment I of the STCs. The quarterly report for the April 1 - June 30 period will be combined with the quarterly report for the July 1 - September 30 period to cover the first semi-annual reporting and payment period for each DSRIP year while the quarterly report for the October 1- December 31 period will be combined with the quarterly report for the January 1 - March 31 period to cover the second semi-annual reporting and payment period for each DSRIP year. The dates for the IA Review Period, the PPS Remediation Period, and the Final Approval may vary based on the actual calendar days available. The actual dates may shift due to circumstances such as a leap year.

Quarterly Report Period Covered	Quarterly Report Due from PPS	Independent Assessor Review Period	PPS Remediation Period Complete	Final Approval Date
April 1 - June 30	July 31	August 30	September 14	September 29
July 1 - September 30	October 31	November 30	December 15	December 30
October 1 - December 31	January 31	March 2	March 17	April 1
January 1 - March 31	April 30	May 30	June 14	June 29

Quarterly Report Appeals

As part of the PPS Quarterly Report process, the PPS are afforded the opportunity to appeal to the IA for a reconsideration of any determinations made by the IA on the awarding of Achievement Values. The PPS appeals take place following the final approval date for the PPS Quarterly Report and requires the PPS to submit in writing to the IA a request for reconsideration. The appeals process does not allow for the submission of any new documentation that was not previously provided during the initial Quarterly Report submission by the PPS or through the PPS Remediation process. The PPS appeals are strictly based on the request of the PPS for the IA to review previously provided documentation based on further explanation of the documentation or clarification on the documentation from the PPS. If an appeal is submitted during a payment driving quarter (Quarter 2 or Quarter 4), the payment to the PPS, may be delayed, to accommodate the PPS appeal.

PPS are notified by the IA of the results of the appeal following a review of the documentation. The IA´s determination following appeal is considered final as the State will not consider appeals from the PPS.
Mid-Point Assessment

In accordance with the STCs, the IA will conduct a Mid-Point Assessment to evaluate progress the PPS have made towards implementing selected projects in their approved DSRIP Project Plans. During DY2, the IA will develop a Mid-Point Assessment tool which will include criteria that will be used to assess the progress made by the PPS. The criteria will be developed in accordance with the Mid-Point Assessment requirements set forth in the STCs and will be submitted to CMS for review prior to the initiation of the Mid-Point Assessment. The Mid-Point Assessment will, at a minimum, include an assessment of the following elements:
1. Compliance with the approved DSRIP Project Plan, including the approved Implementation Plans;
2. Compliance with the required core components for projects described in the DSRIP Strategies Menu and Metrics, including continuous quality improvement activities;
3. Non-duplication of federal funds;
4. An analysis of the relevant data on performance on metrics and indicators to that point in time
5. The benefit to the Medicaid and uninsured (project 2.d.i only) population and to the health outcomes of all patients served by the projects
6. An assessment of project governance including recommendations for how governance can be improved to ensure success;
7. The opportunity to continue to improve the project by applying any lessons learned or best practices; and
8. Assessment of the current financial viability of the PPS lead entities participating in DSRIP
The IA will leverage the data collected through the quarterly reports in addition to the additional data collected through the Mid-Point Assessment tool to make recommendations to the State for the continuation of DSRIP funding for PPS and their selected projects. The IA recommendations will be released for an initial public comment period during which the PPS and the public will have the opportunity to comment and propose additional or alternative recommendations. The IA will consider the comments and any additional or alternative recommendations in developing final recommendations that will be presented to the PAOP. IA recommendations and a summary of the public comments will be made available through the DSRIP website. The recommendations from the Mid-Point Assessment will be subject to review by the PAOP in a manner consistent with that of the initial DSRIP Project Plan Applications, with the PAOP recommendations submitted to the Commissioner. The recommendations of the Commissioner, based on the original recommendations from the IA and those of the PAOP, will be submitted to CMS for review and approval. Any modifications to PPS Project Plans resulting from the Mid- Point Assessment will require a remediation plan outlining the timelines and tasks for implementation to be submitted by the PPS for IA approval before the start of DY3.
Progress towards Value Based Payment (VBP) Goals

To ensure the long-term sustainability of the improvements made possible by the DSRIP investments in the waiver, the STCs require the State to submit a multiyear Roadmap for comprehensive Medicaid payment reform, ultimately outlining the State´s vision for achieving 80-90% of all payments made between managed care organizations (MCOs) and providers to be in Value Based payments (VBP) by the end of the waiver period.
1. The Roadmap
  
  The creation of the Roadmap was completed in an iterative process resulting in a number of drafts, which were refined over time and adjusted to reflect thoughts, feedback, and concerns from stakeholders. This was achieved by the creation of the VBP Workgroup, and a public comment period. The Roadmap was approved by CMS in July of 2015.
2. Preparation for VBP
  
  Upon CMS approval of the Roadmap in July 2015, the work of operationalizing the vision for payment reform commenced. To achieve success, all components of the New York State Medicaid program must understand the fundamental shift that DSRIP and VBP represent. Recognizing the far-reaching impact of the State´s ambitious goal, in year 1 of the VBP Roadmap, the State developed and initiated what has become one of the single largest stakeholder engagement processes ever undertaken by the State. With assistance and expertise from the VBP Workgroup, the State implemented a robust engagement process that resulted in over 500 Stakeholders across the State participating in this critical work. The State and the VBP Workgroup, created subcommittees and advisory groups of stakeholders who were charged with moving the VBP Roadmap towards implementation. This work will result in updates to the VBP Roadmap and will be submitted on an annual basis to CMS.
3. Implementation and Next Steps
  
  Now that the vision and details of the VBP Roadmap have been developed in joint collaboration with the VBP workgroup, the State will turn its attention to implementing VBP and achieving the ambitious goal of 80% VBP. To support this work, the State will launch a number of pilot programs across the State to pilot the innovative arrangements detailed in the Roadmap, as well as provide early feedback and lessons learned on a smaller scale as the rest of the State prepares for implementation. To assist in that preparation, the State will develop a comprehensive training and roll it out across the all regions of the State, called VBP Boot Camp, again, to assist in the implementation of VBP. In addition, the State will continue to call on the VBP workgroup and other advisory groups to continue to provide an important voice on the path to reform.
  
  Additional details on Value Based Payment are available here.

Performance Payments

DSRIP Performance Fund Payments

Performance payments from the DSRIP Performance Fund are made in accordance with the STCs and Attachment I. The DSRIP Performance Funds will be distributed across the five DSRIP years consistent with the distribution of the funds identified in the Sources and Uses table in STC VII, Section 14.d. The Performance Fund payments will be further apportioned across the four domains and across Pay for Reporting (P4R) and Pay for Performance (P4P) metrics as described in Section V.b. of Attachment I.

In DY1, PPS will be eligible to receive three performance payments with the first payment based on the approved DSRIP Project Plans and the second and third payments based on approved semi-annual reports as described in Quarterly Reports and Achievement Values in Section V of this protocol. In all subsequent DSRIP years (DY2 - DY5), the PPS will be eligible to receive two performance payments based on approved semi-annual reports. The semi-annual reports are the combined results of the two quarterly reports submitted for the applicable semi-annual period, with the first semi-annual reporting and payment period incorporating the April - June and July - September quarterly reporting periods and the second semi-annual reporting and payment period incorporating the October - December and January - March quarterly reporting periods. Performance payments will be made within 30 days of the final approval of the semi-annual reports by the IA. Payments for the semi-annual period of April - September will be made in January and payments for the October - March semi-annual period will be made in July.

When PPS fail to achieve the defined milestones and measures, the unearned performance payments will be transferred to the DSRIP High Performance Fund. Any unearned performance payments will be carried forward for a period of one-year in accordance with STC VII, Section 14.c. Following the one-year carry forward period, any performance payments that remain unearned will be returned to CMS.

The following table identifies the reporting and payment schedule for all five years of the waiver.

DSRIP Year	Deliverable	PPS Submission Date	IA Review Complete	PPS Remediation Complete	Final Approval Date	Payment Date
DY 1	DSRIP Project Plan	12/22/2014			3/2/2015	4/23/2015
DY 1	DSRIP Domain 1 Implementation Plan	6/1/2015	6/30/2015	N/A	9/29/2015	N/A
DY 1	PPS 1^st Quarterly Report (4/1/15 – 6/30/15) / Domain 1 (revised) & Project Implementation Plan	8/7/2015	9/8/2015	9/22/2015	9/30/2015	1/29/2016
DY 1	PPS 2^nd Quarterly Report (7/1/15 – 9/30/15)	10/31/2015	11/30/2015	12/15/2015	12/30/2015	1/29/2016
DY 1	PPS 3^rd Quarterly Report (10/1/15 – 12/31/15)	1/31/2016	3/1/2016	3/16/2016	3/31/2016	7/29/2016
DY 1	PPS 4^th Quarterly Report (1/1/16 – 3/31/16)	4/30/2016	5/30/2016	6/14/2016	6/29/2016	7/29/2016
DY 2	PPS 1^st Quarterly Report (4/1/16 – 6/30/16)	7/31/2016	8/30/2016	9/14/2016	9/29/2016	1/29/2017
DY 2	PPS 2^nd Quarterly Report (7/1/16 – 9/30/16)	10/31/2016	11/30/2016	12/15/2016	12/30/2016	1/29/2017
DY 2	PPS 3^rd Quarterly Report (10/1/16 – 12/31/16)	1/31/2017	3/2/2017	3/17/2017	4/1/2017	7/29/2017
DY 2	PPS 4^th Quarterly Report (1/1/17 – 3/31/17)	4/30/2017	5/30/2017	6/14/2017	6/29/2017	7/29/2017
DY 3	PPS 1^st Quarterly Report (4/1/17 – 6/30/17)	7/31/2017	8/30/2017	9/14/2017	9/29/2017	1/29/2018
DY 3	PPS 2^nd Quarterly Report (7/1/17 – 9/30/17)	10/31/2017	11/30/2017	12/15/2017	12/30/2017	1/29/2018
DY 3	PPS 3^rd Quarterly Report (10/1/17 – 12/31/17)	1/31/2018	3/2/2018	3/17/2018	4/1/2018	7/29/2018
DY 3	PPS 4^th Quarterly Report (1/1/18 – 3/31/18)	4/30/2018	5/30/2018	6/14/2018	6/29/2018
DY 4	PPS 1^st Quarterly Report (4/1/18 – 6/30/18)	7/30/2018	8/30/2018	9/14/2018	9/29/2018	1/29/2019
DY 4	PPS 2^nd Quarterly Report (7/1/18 – 9/30/18)	10/31/2018	11/30/2018	12/15/2018	12/30/2018	1/29/2019
DY 4	PPS 3^rd Quarterly Report (10/1/18 – 12/31/18)	1/31/2019	3/2/2019	3/17/2019	4/1/2019	7/29/2019
DY 4	PPS 4^th Quarterly Report (1/1/19 – 3/31/19)	4/30/2019	5/30/2019	6/14/2019	6/29/2019	7/29/2019
DY 5	PPS 1^st Quarterly Report (4/1/19 – 6/30/19)	7/31/2019	8/30/2019	9/14/2019	9/29/2019	1/29/2020
DY 5	PPS 2^nd Quarterly Report (7/1/19 – 9/30/19)	10/31/2019	11/30/2019	12/15/2019	12/30/2019	1/29/2020
DY 5	PPS 3^rd Quarterly Report (10/1/19 – 12/31/19)	1/31/2020	3/1/2020	3/16/2020	3/31/2020	7/29/2020
DY 5	PPS 4^th Quarterly Report (1/1/20 – 3/31/20)	4/30/2020	5/30/2020	6/14/2020	6/29/2020	7/29/2020

DSRIP High Performance Fund

The DSRIP High Performance Fund has been established to reward PPS that have exceeded their set performance benchmarks for a given DSRIP year beginning in DY2 as Domain 3 measures convert from P4R to P4P. Additional measures will become High Performance eligible in DY3 as Domain 2 measures convert from P4R to P4P. The funding for the High-Performance fund will be based on a 3% withhold from the DSRIP Performance Fund as well as any unearned performance payments resulting from PPS missed metrics. Unearned performance payments from the two semi-annual payment periods in one DSRIP Year will be included in the High Performance Fund through the end of the following DSRIP Year, i.e. unearned performance payments from the first DY2 payment period and from the second DY2 payment period will be included in the High Performance Fund for DY3. The High Performance Fund will be spread across DSRIP years 2 through 5 following the same distribution of the DSRIP Performance Fund described in STC VII, Section 14.d.

The DSRIP High Performance Fund will be distributed across two tiers:
1. Tier 1 will reward PPS whose performance closes the gap between their current performance and the statewide performance goal by 20% or more in a given DSRIP year
2. Tier 2 will reward PPS whose performance meets or exceeds the statewide performance goal in a given DSRIP year.
PPS can earn DSRIP High Performance Fund payments by achieving one of the two tiers for any of the ten measures identified as High Performance measures in Attachment J of the STCs.

Payments from the DSRIP High Performance Fund will be made annually, in a time frame consistent with that of the second semi-annual performance payments. PPS that achieve high performance levels will be eligible to receive DSRIP High Performance Fund payments up to 30% of their DSRIP project value.
Managed Care Contracting Program Payments

There are several programs authorized by the MRT Waiver Amendment that exist outside of DSRIP but align with DSRIP programmatic goals. These programs in the STCs are described as for non-DSRIP Medicaid Redesign purposes. DOH will implement these programs through the State´s Medicaid Managed Care Contract, to meet the goals of these programs along with DSRIP and the transition to Value-Based Payment.

PPS Support
1. DSRIP Account Support Structure
  
  The purpose of the Account Support structure is to monitor each PPS in their Implementation efforts by providing tools, analysis, information sessions, and day-to- day support to assist with DSRIP implementation. The AST clarifies DSRIP-related documents and requirements, answers DSRIP-related questions, and evaluates each PPS approach and reports risks if necessary. The AST provided assistance to PPS with respect to guidelines for completion of Implementation Plans.
  
  The AST has a monthly check in with each PPS, an in-person check in four times each year, as well as frequent notifications to the PPS of upcoming releases, such as guides, webinars, trainings, and other associated communications. The AST also has bi-weekly check-ins with relevant vendors and stakeholders. The AST maintains activity logs to track movements around the PPS and submits bi-weekly PPS Implementation progress status reports and Weekly Enhanced Oversight progress reports to DOH, specifically OHIP.
2. Additional PPS Support Activities
  
  DOH, in conjunction with its vendors, provides support to the PPS through a wide array of support to the PPS. These support activities include in person meetings with the PPS and web-based trainings with dedicated question and answer opportunities.
  
  All-PPS Meetings
  
  DOH convenes all PPS throughout the State once every two months for PPS to showcase their best practices and provide DSRIP program updates and tools. The All- PPS meetings feature updates from DOH and the IA as well as PPS led presentations or panels on specific topics.
  
  Webinars and Operator Assisted Calls
  
  In addition to the All-PPS Meetings, DOH uses webinars and operator assisted calls to convey program updates and provide guidance to the PPS on specific topics.
  
  Materials from the webinars and operator assisted calls, including presentations and Question & Answer (Q&A) documents, are posted to the DSRIP website following the webinar or operator assisted call.
  
  Materials that are developed specifically for the PPS are also posted to a Digital Library. The Digital Library is a secure SharePoint site that limits access to the PPS users with MAPP access.
  
  Webinar information can be found on the DSRIP website here.
  
  Enhanced Support and Oversight (ESO)
  
  DOH has also implemented an Enhanced Support and Oversight (ESO) program to provide additional resources and support for PPS that DOH or the AST has identified as requiring further support for specific efforts under the DSRIP program. ESO automatically includes PPSs where their PPS award is far greater than their previous organizational and financial history has managed. ESO activities include additional meetings with DOH and AST staff to ensure that the PPS is moving towards the successful implementation of the DSRIP Project Plan.
3. NY DSRIP PPS Learning Symposium
  
  The purpose of the NY DSRIP PPS Learning Symposium is to promote and support an environment of learning and information sharing based on data transparency within the New York healthcare industry in an effort to bring meaningful improvement to the landscape of healthcare in New York. An annual statewide Learning Symposium is held once each year to bring PPS together for a multi-day opportunity to focus on DSRIP and seek peer-to-peer (provider-to- provider) and community stakeholder input on project level development of action plans, implementation approaches and project assessment. Invitees to the annual statewide Learning Symposium will include PPS providers, community-based organizations, consumer advocates, national health care reform experts, county agency representatives, health plan representatives, HIT/HIE representatives and State and public health officials. The AST is responsible for collaborating with relevant stakeholders and designing the content of the meeting.
  
  The theme of the first annual statewide Learning Symposium was "New York DSRIP Year 1: Launching PPS Transformation Efforts." Objectives of the Learning Symposiums are to:
  1. Enhance collaborative within each PPS with broad group of partners
  2. Develop partnerships across PPS and ways to share emerging best practices and evidence-based approaches
  3. Learn about strategies to improve care transitions and transform delivery systems
  4. Further dialogue between PPS and the State about how to achieve DSRIP goals
  5. Initiate relationship-building discussions within and across PPS including providers, consumer advocates, and community-based partners
  6. Spur discussion about promising efforts across the State and nation to transform current care delivery practices
  7. Share implementation strategies and success factors to achieve DSRIP goals
    
    In addition to the annual statewide Learning Symposiums, the State will host two regional Learning Symposiums each year. The regional Learning Symposiums will be structured to focus on issues that are relevant to subsets of the PPS and may be driven by regional (Upstate vs. Downstate) or specific project implementation differences.
4. MRT Innovation eXchange (MIX) and the DSRIP LinkedIn Group
  
  From October 2014 to February 2016, the purpose of the MIX was to collect ideas on a digital platform on how to improve and accelerate the progress of the projects targeting Medicaid redesign under DSRIP. The MIX was designed to help increase speed of the communication, idea sharing, education, collaboration and relationships formed between DSRIP stakeholders to contribute to the success of the DSRIP program.
  
  The MIX was a collaborative tool, supplemented by topic-specific discussion groups to further engage participants and encourage collaborative dialogue.
  
  In February 2016, the MIX was transitioned to a group on LinkedIn called the New York State Delivery System Reform Incentive Payment (DSRIP) Program Group. The group is currently used by DOH staff, MRT and DSRIP stakeholders, PPS, Community Based Organizations, and the public. The group is still being used in many of the same ways as the MIX was, and will continue to be a space to foster collaboration and communication among PPS and others.
  
  The DSRIP group on LinkedIn, can be found here.
5. Medicaid Accelerated eXchange (MAX)
  
  The MAX series program is designed as a multidisciplinary collaboration that empowers clinicians to lead change, apply leading practices, and learn from leading subject matter professionals and their peers. It is focused at a local level (i.e. specific providers within a PPS) to generate grass-roots level change and act as an enabler to impact overall DSRIP measures, as well as local improvement measures.
  
  The program brings together 8-10 multidisciplinary providers and a patient representative to form Action Teams that will participate in a 12-month intensive learning experience. The learning experience is delivered across three full-day Workshops that empower participants to implement meaningful change during intermediary Action Periods. The program builds skills and capacity for process improvement at a local level that can be scaled and shared across the broader PPS.
  
  The MAX Series Program covers specific clinical focus areas that were chosen with direct input from PPS based on existing opportunities to improve health outcomes and progress toward DSRIP goals. The initial four topics being offered are: 1) Managing Care for Super Utilizers, 2) Integration of Behavioral Health and Primary Care, 3) Primary Care Access Optimization, and 4) High-Risk Populations: Prevention and Patient Engagement.
  
  Participation in the MAX Series is voluntary; the PPS were all provided information on the MAX Series before the deadline for making a definitive decision regarding participation.
6. Regulatory Issues
  
  In an effort to further facilitate the successful implementation of DSRIP projects, the State has provided the PPS with opportunities to seek relief from certain regulatory requirements in the form of regulatory waivers, Certificate of Public Advantage (COPA), and Accountable Care Organization (ACO) applications.
  1. Regulatory Waivers
    
    Requests for regulatory waivers related to DSRIP projects are submitted to and coordinated by OPCHSM. DOH, OMH, OASAS, and OPWDD have issued guidance to PPS interested in seeking regulatory waivers in connection with the DSRIP Program and the Capital Restructuring Financing Program, pursuant to NYS Public Health Law (PHL) § 2807(20)(e) and (21)(e).
    
    A team comprised of the four aforementioned agencies was established to review regulatory waivers. All appropriate agencies are directly engaged in the waiver review process. Waivers are granted to allow applicants to avoid duplication of requirements and to allow the efficient implementation of the proposed projects. Waiver requests are approvable only for State regulations where patient safety would not be implicated, and are not available with respect to federal statutes and regulations or State statute. DOH will maintain a spreadsheet to track and describe each waiver and the tracker is shared among the four agencies, which will ensure that clinical program and regulatory surveillance are fully coordinated for areas that have received approved waivers.
    
    Requests for regulatory waivers were initially submitted in conjunction with the DSRIP Project Plan application. These Project Plans are posted on the DOH DSRIP website and were made available to CMS. The State´s responses to waiver requests are posted online and available online here.
  2. Certificate of Public Advantage (COPA) / Accountable Care Organization (ACO) Applications
    
    With respect to COPA, Public Health Law (PHL) Article 29-F sets forth the State's policy of encouraging appropriate collaborative arrangements among health care providers who might otherwise be competitors. The statute requires DOH to establish a regulatory structure allowing it to engage in appropriate State supervision as necessary to promote State action immunity under State and federal antitrust laws.
    
    In accordance with the statute, DOH issued regulations which establish a process for providers to apply for a COPA for their collaborative arrangements such as mergers and clinical integration agreements. The application process under the regulations is available to PPS interested in applying for a COPA in connection with a DSRIP Project Plan application.
    
    Public Health Law (PHL) Article 29-E requires DOH to establish a program governing the approval of ACOs. In conjunction with the statute, DOH issued regulations establishing a process for entities that wish to become ACOs to apply for certificates of authority. The application process under the regulations is available to PPS interested in applying for a certificate of authority in connection with a DSRIP Project Plan application.
    
    Stakeholders were consulted in the development of the COPA and ACO regulations, which were published in the State Register, and the public had the opportunity to comment on them.
    
    Summaries of COPA applications received to date are available here and information regarding ACO certificates of authority is available here.
    
    All COPA and ACO applications will be reviewed by DOH, in consultation with PCG, which provides technical assistance for these purposes. In addition, DOH will consult with the Office of the Attorney General and, if appropriate, other partner agencies. Further, as set forth under the statute, DOH will not issue a COPA before consulting with and receiving a recommendation from the Public Health and Health Planning Council. The review process is generally expected to take approximately four months but may vary from case to case. COPA and ACO certificates of authority may not be appropriate or granted in all cases; if granted, they will be subject to ongoing supervision and monitoring. Both the COPA and ACO regulations permit DOH to terminate a certificate, after an opportunity for a hearing, in appropriate circumstances.
    
    Staff are engaged in an ongoing effort to ensure there are sufficient resources within DOH to process COPA and ACO applications. Oversight of the IA contract will ensure that the contractor is meeting its deliverables with respect to the evaluation of such applications. It has been determined that existing Medicaid internal control systems will not need to be modified to carry out DSRIP activities.

Statewide Measurements and Accountability

Statewide DSRIP Achievement and Accountability

Starting in Year 3 of the waiver, achievement of the statewide four milestones is detailed in STC VII, section 14, providers and the State are accountable for statewide performance. DSRIP funding for providers may be reduced based on missed statewide performance milestones (described below). If any of the four milestones below are not met, then DSRIP payments to providers will be reduced by the amount specified in STC VII, section 14, with reductions of 5% of the DSRIP Year 3 funding up to 20% of the DSRIP Year 5 funding. DSRIP payment reductions will be applied proportionately to all DSRIP PPS based on the valuation of their DSRIP Project Plans. DSRIP reductions will not be applied to the DSRIP high performance fund payments. The High Performance Fund is not subject to penalties.

The four statewide milestones include:
1. Statewide performance on universal set of delivery system improvement metrics.
  
  Starting in DY3, the State will be evaluated on the performance of all PPS on a core set of delivery system improvement metrics in domain 2. Each of the metrics will be assigned a direction for improving or worsening and performance will be calculated to reflect the performance of the State. This milestone will be considered passed in a given year if more metrics in this domain are improving on a statewide level than are worsening.
2. A composite measure of success of projects statewide on project-specific and population wide quality metrics.
  
  The State will be evaluated based on the number of metrics that are met by the PPS in a given year for project specific improvement standards specified in the approved DSRIP Project Plans. The metrics will be added together to determine the composite success of all DSRIP projects. The statewide milestone will be considered passed in a given year if the number of metrics met by all PPS in the aggregate is greater than the number of metrics that were not met.
3. Growth in statewide total Medicaid spending that is at or below the target trend rate.
  
  The State will be measured, on a per member per month (PMPM) basis, on the growth in statewide total Medicaid spending relative to a target trend rate for two metrics:
  1. Growth in statewide total inpatient and emergency room spending that is at or below the target trend rate (applicable for DY3, DY4, and DY5). The target trend rate is the ten-year average rate for the long- term medical component of the CPI minus 1 percentage point for DY3 and 2 percentage points for DY4 and DY5.
  2. Growth in statewide total Medicaid spending that is at or below the target trend rate (applicable in DY4 and DY5). The target trend rate is the ten year average rate for the long-term medical component of the Consumer Price Index (CPI).
    
    The PMPM amounts will be adjusted to exclude growth in federal funding associated with the Affordable Care Act (ACA). The State will not be penalized if it uses the Federal Medical Assistance Percentage (FMAP) rates generated by the ACA to reinvest in its Medicaid program.
    
    This milestone will be considered passed in a given year if both metrics are at or below the target trend rate for the metric.
4. Implementation of the managed care plan.
  
  This milestone will be measured by targets agreed upon by CMS and the State after receipt of the managed care strategy plan. The targets will include one associated with the degree to which plans move away from traditional fee for service payments to payment approaches rewarding value.
The State must pass all four milestones in order to avoid DSRIP penalties. If the State fails on any of the four milestones, the DSRIP funding may be reduced in accordance with the reductions identified in STC VII, section 14.h. The DSRIP penalties will only be applied to the DSRIP Performance Fund payments and will not cause a reduction to the DSRIP High Performance Fund.
Statewide Controls and Measures

The State and CMS agree that, in conjunction with any Partnership Plan demonstration renewal beyond December 31, 2014, the State will undertake additional activities and steps to strengthen internal controls, compliance with federal and State Medicaid requirements and financial reporting to ensure proper claiming and to self-identify and initiate timely corrective action on problems and issues. To support the development of these additional special terms and conditions, the State has completed and submitted to CMS a report outlining its assessment of current strengths and weaknesses of State´s system of internal and financial management controls including any steps the State proposes to strengthen compliance, documentation, transparency, and the expected path for resolution of any outstanding deferrals or disallowances initiated by CMS. This process described will allow for the new DSRIP functional controls consistent with the approved process be folded into the existing framework.

Through monitoring the State expects to: (1) recognize and resolve operational barriers immediately (2) quickly identify the State´s needs for technical assistance; and (3) assure program integrity and accountability. The New York DSRIP is comprised of three funding phases as described further in the STCs, including: 1) Interim Access Assurance Fund; 2) DSRIP Design Grant; and 3) DSRIP Fund.

Independent Evaluator (IE)

As required by the STCs, the role of the IE will be to conduct a multi-method, robust, statewide evaluation to document the impact of DSRIP on health care service delivery, health improvements, and cost to the New York´s Medicaid program, as well as to determine program components that posed particular successes or challenges for implementation and outcomes. The broad goals of the New York DSRIP evaluation are to 1) assess program effectiveness on a statewide level with respect to the MRT Triple Aim of improved care, better health, and reduced cost, 2) conduct PPS-level comparisons to obtain information on the effectiveness of specific projects and strategies selected and the factors associated with program success, and obtain feedback from stakeholders including DSRIP planners, administrators, providers and patients, regarding the planning and implementation of the DSRIP program and on the health care service experience under DSRIP reforms.

Procurement of the IE will follow the standard New York State Request for Proposal (RFP) process under which bidders will submit proposals that include an evaluation design and a budget appropriate to execute the evaluation, and produce the deliverables on the timeline specified in the RFP. A review team will be assembled by OQPS with OHIP representation, which will review all proposals received in response to the RFP and score them on adherence to the specified evaluation goals, scientific merit, feasibility, and cost. The procurement and contracting process will be overseen by the OHIP Administrative Services Unit. Additional information can be found here.

Once an IE is selected, OQPS will provide the necessary support, including access to data, to enable the evaluation to be effectively conducted, as well as oversee the evaluation activities and review contract deliverables. On a schedule to be determined, regular meetings between OQPS and the IE will take place, as part of the oversight process to discuss progress and to resolve any problems that may arise.

The IE will be responsible for the following reports on the specified due dates:

Interim Evaluation Report. Per agreement between the State and CMS, this report will contain evaluation results from quantitative and qualitative data available for reporting by due date.

Draft Due to NYSDOH for Review	February 15, 2019
Draft Interim Evaluation Report due to CMS	March 30, 2019
Final Interim Evaluation Report due to NYSDOH for review	May 15, 2019
Final Interim Evaluation Report due to CMS	June 30, 2019

Summative Evaluation Report. Per agreement between the State and CMS, this report will cover the entire five-year demonstration, and contain the major results and conclusions with respect to DSRIP´s operation and effectives. This will be the final report from the DSRIP evaluation. Content of the report is described in STC VII section 24.

Preliminary Summative Evaluation Report due to NYSDOH for review	May 15, 2020
Preliminary Summative Evaluation Report due to CMS	June 30, 2020
Draft of Final Summative Evaluation Report due to NYSDOH for Review	November 15, 2020
Draft of Final Summative Evaluation Report due to CMS	December 28, 2020
Final Summative Evaluation Report due to NYSDOH for review	February 15, 2021
Final Summative Evaluation Report due to CMS	March 28, 2021

Annual Statewide Reports. For the first four years of the demonstration, annual summaries of major DSRIP evaluation results will be shared with State policymakers, PPS planners, administrators and providers in order to highlight areas of success and those in need of improvement, and to guide any needed program modifications and enhancements.

Each demonstration year´s annual report due on March 31 of the following year. No annual statewide report is due for DY 5, as it will be replaced by the Summative Evaluation Report.
Annual PPS Reports. The Contractor will, on an annual basis for each of the five demonstration years, distribute results from interviews and surveys administered on the PPS level back to those PPSs, with the expectation that receipt of information that is specific to their own projects will assist their ongoing quality improvement efforts.

Each demonstration year´s PPS report is due on March 31 of the following year.

DSRIP DY0 Implementation Activities
1. Interim Access Assurance Fund (IAAF)
  
  The STCs allow for temporary, time limited, funding, up to $500 million in FFP, to be available from an IAAF to protect against degradation of current access to key health care services in the near term. The IAAF is available to provide supplemental payments that exceed upper payment limits, DSH limitations, or state plan payments, to ensure that current trusted and viable Medicaid safety net providers, according to criteria established by the state consistent with these STCs, can fully participate in the DSRIP, transformation without unproductive disruption. The IAAF is authorized as a separate funding structure from the DSRIP program to support the ultimate achievement of DSRIP goals. The State made all decisions on the distribution of IAAF payments with the funding directed to those providers that serve significant numbers of Medicaid members and, based on a State assessment, had a financial hardship in the form of financial losses or low margins.
  
  The State initiated the IAAF process by posting the qualifications that providers had to meet to receive IAAF payments on the DSRIP website. The qualification criteria was open for public comment, following which the State initiated an open application period for all providers seeking IAAF funding. The State made final determinations on the IAAF applications using the application documents and relevant public comments.
  
  The STCs required the State to ensure the non-duplication of funds for any provider meeting the qualifications for IAAF funding that also received funds through the State´s vital access program, or any other supplemental payment program for which the federal government provides matching funds, or Medicaid disproportionate share hospital payments. Providers were eligible to receive IAAF funds as well as program design grants funds as part of the DSRIP program.
  
  Information on the IAAF can be found on the DSRIP website here.
2. DOH Stakeholder Engagement
  
  The State will continue to engage the public and all relevant stakeholders (including CMS, community stakeholders, Medicaid members, physician groups, hospitals, and health plans) throughout the DSRIP Planning and Implementation process. Establishing transparency throughout the DSRIP process will be achieved through the following activities: publishing application materials and prototypes, releasing responses provided to public comment periods, increasing website engagement, conducting surveys, posting resources and offering webinars.
  1. Engagement Activities Detail
    
    DOH and its selected DSRIP vendors will continue to support public access to resources and outreach activities that will guide and assist PPS throughout the planning and implementation process. Examples of such resources are:
    1. Webinars
    2. Public Forum Meeting
    3. Web Resources
    4. MAPP and Network Tools
    5. Digital Library
    6. NY DSRIP PPS Learning Symposium
    Details on these resources have been provided in Section V. Provider Requirements and Section VII. PPS Support.

|top of section| |top of page|

ATTACHMENT L

DSHP Claiming Protocol

DSHP List 1 - The state may claim FFP in support of DSRIP for DSHP expenditures made after March 31, 2014, upon the date of approval of the protocols for DSHP List 1.
DSHP List 2 - The state may claim FFP in support of DSRIP for List 2 DSHP expenditures made after December 31, 2014. The state may not claim FFP until after the date on which CMS has approved a DSHP Claiming Protocol for DSHP List 2.

I. State Documentation of Expenditures for DSHP List 1 and 2 Programs

Documentation provided by the State to CMS for quarterly DSHP expenditures will include the following:

the agency;
the program;
provider;
payment amount;
voucher/contract information or Automated Claiming System (ACS) line number (where applicable); and
provider costs.

II. Off-Sets: In accordance with Section VII STC 15(c)(iii) DSHP expenditures submitted to CMS will not include payment for:

grant funding to test new models of care;
construction costs (bricks and mortar);
room and board expenditures;
animal shelters and vaccines;
school based programs for children;
unspecified projects;
debt relief and restructuring;
costs to close facilities;
HIT/HIE expenditures;
services provided to undocumented individuals;
sheltered workshops;
research expenditures;
rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development;
prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave;
revolving capital fund;
expenditures made to meet a maintenance of effort requirement for any federal grant program;
administrative costs;
cost of services for which payment was made by Medicaid or CHIP (including from managed care plans);
cost of services for which payment was made by Medicare or Medicare Advantage; and
funds from other federal grants.
To assure DSHP expenditures do not include coverage of services to undocumented individuals, the State will reduce each service provider´s reported program costs by ten percent unless a more detailed accounting of actual costs for these individuals is provided that is acceptable to CMS.

III. Documentation of State Expenditures for Designated State Health Programs

In claiming DSHP expenditures, New York State will provide CMS with a summary excel sheet by agency, program and provider in an orderly format so that CMS may review and test underlying supporting documentation as detailed in this Section.

For all eligible DSHPs claimed New York State will make available for CMS the following information:
1. Direct control payment sheets for all providers
2. Identifying contract number, provider name & code (agency code?), budget period
3. Program
4. Voucher number
5. Voucher amount
6. Total amount paid to date
7. State financial system voucher entry
Documentation of expenditures for each DSHP must be clearly outlined in the state´s supporting work papers and be made available to CMS in accordance with this claiming protocol.
The State will use its voucher and accounting system to identify the amount it expended to purchase services from each service provider under each program during the claiming period.

Statewide Financial System (SFS):

SFS is the State´s accounting system, the ´book-of-record´ for New York State. Each agency enters vouchers into SFS. The Office of the State Comptroller (OSC) reviews, approves and then processes the payments. The OSC process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a certificate of approval process that is built into SFS.

DOH receives claims/vouchers and pays for such receipts through the SFS payment system. Such payment ties back to the specific budget appropriation for such program.

The New York State Office for the Aging (NYSOFA) enters vouchers into SFS through a secure bulk-load file protocol. The bulk-load files create the commensurate entries into the SFS system to generate payments to providers/counties. Once approved and processed, the transaction information is then put into a pre-programmed extract file that is bulk-loaded into NYSOFA´s "Grants" system. Information from the file is loaded into the Grants system populating all the relevant payment information into assigned data tables.

OCFS does not process Committee on Special Education (CSE) payments through SFS.

OCFS Systems

Automated Claiming System (ACS):

The ACS is an application developed to aid the local districts in submitting their reimbursement claims for settlement. Expenditures for all major programs served by the local departments of social services, with the exception of Medicaid Payments, are entered into various schedules on the ACS on a monthly basis. Each of these schedules are designed to capture necessary data for both the statewide claims to the federal government and settlement of the proper shares to the local district of both federal and state shares.

Statewide Standards of Payment (SSOP):

SSOP is the rate setting system of the Office of Children and Family Services (OCFS) Bureau of Budget Management (BBM). OPWDD Private Residential Schools (P Schools) and Voluntary Agencies (VAs) that provide special education services to OCFS youth use SSOP to submit program and fiscal data to BBM on an annual basis. The data is used to calculate a reimbursement rate for each program, called a Maximum State Aid Rate (MSAR). The agency uses the MSAR to bill Local Departments of Social Services (LDSS) for the services rendered to each youth.

NYSOFA Systems
"Grants" System:

The Grants system is a NYSOFA budgeting, financial and grants management system that tracks $200M in state and federal grant program funding from the development of the NYS Enacted Budget through the grant reconciliation/close-out process. The system tracks the authorization of over 25 funding streams that are operated by 59 Area Agencies on Aging and approximately 80 not-for-profit providers and all 62 NYS counties. The system provides Internal Controls through a variety of programmed "checks and balances" and there are user ID and date/time stamps saved for every transaction in the system. Budgetary checks are hard coded into the system to ensure sufficient funds are available to make payments. The system also includes error checks to ensure compliance with many federal and State program compliance requirements. The system has greatly reduced required as the system is linked to county planning documents, and provider budget and claiming information.

The Grants System interfaces with the following systems:
- AIP System (a system where AAAs submit annual service implementation plans electronically and NYSOFA uses to produce grant agreements)
- AAA Table (Aging Services Grantee/Provider Directory)
- Consolidated Area Agency Reporting System (CAARS)
- National Aging Program Information System (NAPIS) - a reporting system that contains information on individuals served and units of services (maintained at AAA level and aggregated data is uploaded to NYSOFA on a quarterly basis); and
- SFS (Statewide Financial System) and OSC Master Vendor File (VendRep)
The AIP system is a budgeting system that allows grantees to file their budgets with the State electronically. These budgets must to conform to the funding level authorized for the Grantee and are reviewed by NYSOFA staff through a series of automated and manual checks. The checks are linked to the programmatic and fiscal requirements outlined in the AIP Guidelines issued annually. Grantees are allowed to revise their allocations throughout the current fiscal year and NYSOFA staff are required to review, run error checks and approve the revised AIP. If a grantee fails to complete an AIP, they can be subjected to a withholding of their State Aid. County based AAAs contract a large portion of their grants to local provider organizations.

IV. DSHP List 1 Program Details

A. The Department of Health (DOH)

State Agency: Department of Health

Program: AIDS Drug Assistance Program

Program Codes: SFS 29880

Funding Sources: HCRA Resources Fund

$41,050,000 - State Appropriation

Chapter 53 of the Laws of 2014; page 564, lines 7-9

Process to identify the value of applicable offsets: The ten percent undocumented immigrant offset is applied unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS.
Process to identify program net deficit payments--N/A since there are no net deficit payments associated with this program.

Brief Description:

The New York State Department of Health AIDS Institute has established four programs for HIV Uninsured Care (ADAP, ADAP Plus, the HIV Home Care Program, and the ADAP Plus Insurance Continuation Program). The AIDS Drug Assistance Program (ADAP) provides access to a select list of medications for the care and treatment of HIV and AIDS and associated conditions, ADAP Plus covers outpatient primary care services, Home Care covers services provided to individuals meeting certain medical criteria and the ADAP Plus Insurance Continuation Program (APIC) provides premium payment assistance for cost effective insurance coverage for eligible individuals.

The mission of the programs is to provide access to medical services and HIV medications for all New York State residents with HIV/AIDS. The programs employ a dual approach to carry out their mission. First, the programs empower the individual to seek and access care by providing an "Enrollment Card", which allows the individual to choose a provider and receive care/drugs without cost. Second, the programs supply a stable and timely funding stream to health care providers, enabling them to use the revenues to develop program capacity to meet needs of the uninsured HIV population.

Expenditures the state is claiming for Federal Financial Participation are all pharmacy claims eligible for reimbursement with state funds. AIDS Drug Assistance Program (ADAP) claims are the only expenses that will be used for the purposes of this protocol. ADAP began in 1987 as part of a national program to provide free HIV/AIDS drugs to low-income individuals not covered by Medicaid or adequate third-party insurance.

Eligible Population:

The programs serve HIV-infected New York State residents who are uninsured or under-insured and meet established criteria. The programs can serve as a transition to Medicaid by providing interim assistance to persons eligible for but not yet enrolled in Medicaid or assist in meeting spend down requirements. Individuals with third-party insurance who cannot meet the deductibles or co-payments, or whose policies have waiting periods, may enroll and programs will coordinate benefits with those of their insurance company.

Funding:

The Ryan White HIV/AIDS Treatment Extension Act of 2009 Parts A and B
State Health Care Reform Act Funds
Recoveries

Eligible Providers:

Pharmacies are eligible to participate in the programs if they are enrolled in the states´ Elderly Pharmaceutical Insurance Coverage (EPIC) Program and Medicaid. Pharmacies are enrolled in ADAP through an agreement process that includes verification of their ongoing participation with the state´s larger pharmacy benefit programs (EPIC and Medicaid) and their certification to submit pharmacy claims electronically using National Council of Prescription Drug Plans (NCPDP) D.0 pharmacy claim processing standards.

Each participating pharmacy must have an EPIC ID number or National Association Board of Pharmacies (NABP) number and a National Provider Identification (NPI) number that is verified weekly before reimbursement is made.

HRI/Claiming Process:

Health Research, Inc. (HRI) is an independent 501(c) (3) not-for-profit corporation that is affiliated with the New York State Department of Health. HRI functions as the fiduciary agent for the programs, no HCRA funds are used to reimburse HRI directly. All charges against HCRA funds are for pharmacy services provided to eligible individuals at eligible providers.

In addition to personnel and other operational functions, HRI verifies program accounting functions, reviews systems for operational accountability, confirms provider eligibility and generates pharmacy reimbursement checks for eligible providers.

HRI assists DOH in evaluating, soliciting, and administering external financial support for DOH projects. HRI´s funding sources include Federal and State agencies, not-for-profit foundations, and commercial firms. State Health Care Reform Act (HCRA) funds are requested by memo transfer based on the annual state expenditure plan and are used by HRI to pay eligible ADAP pharmacy claims.

Documentation of State ADAP Program Expenditures:

All pharmacy claim processing functions are HIPAA compliant. ADAP uses New York State Medicaid fee for service pharmacy reimbursement methodologies for all covered medications.

State expenditures for the ADAP program consist of paid pharmacy claims with State HCRA funds submitted by enrolled pharmacies on behalf of eligible individuals for drugs included in the program formulary. An Individual´s eligibility for the ADAP program is identified by a unique 11-digit identification number and eligibility card. Pharmacies participating in the ADAP program are identified by a program provider enrollment number. Drugs eligible for reimbursement are identified by an 11-digit National Drug Code (NDC).

Documentation related to all expenditures is maintained at the HRI offices, for confidentiality purposes, participant data is maintained at the ADAP offices. HRI audit staff perform transaction audits to ensure that all eligibility and expenditure documentation relating to selected participants and payments are appropriately maintained. HRI is also audited by its sponsors on a periodic basis and undergo an annual A-133 independent audit.

Payments and grant and funding source expenses are tracked though HRI´s financial information management system with individual claim and participant level data managed through ADAP´s information management systems.

Advances are not provided for this program. The program reimburses eligible providers for covered costs for eligible participants. The program is modeled after Medicaid, uses Medicaid fee for service reimbursement methodologies and rates to pay eligible providers for eligible services.

Step 1 ADAP operates a pharmacy benefit management system (PMB) that captures claims submitted by participating pharmacies for eligible individuals for covered drugs in real time using NCPDP claim processing standards.

Step 2 Claims are captured at the time an individual presents at a pharmacy for a prescription fill or refill. The following fields are required for claim submission -

Bank Identification Number - six-digit number that health plans use to process electronic pharmacy claims
NCPDP version being used to transmit the data
Transaction Code - Eligibility only E1, Claim Submission B1, Reversal B2
Capture if it is TROOPUCP or ADAP for transaction code (above)
Pharmacy EPIC ID
Participant ADAPID
Date the prescription was filled
The prescription number
Check to see if it is a refill
If it is a refill check to see which refill number it is
Metric Quantity of the medication
Number of days the supply
National Drug Code of the medication
Prescriber ID
Date Prescribed
Cost of the medication
Dispensing Fee
Medication approved or not
Denial Reasons/Rejection Codes
Date on POS the claim was processed
Time on POS the claim was processed
Transaction Type
If the claim is Dispense as Written
Therapeutic Cross Reference
Usual and Customary - Costs
Gross Amount Due
Basis for reimbursement Provides Pharmacy with info on how reimbursement was calculated

Step 3 ADAP aggregates all pharmacy claims weekly and creates a "batch summary report" for submission to HRI for payment.

Step 4 Information sent electronically to HRI to process claims: (Sample Below)

Corporations - Attention of at the Corporation, Corporation name, Sum for checks, Corporation ID and details for Individual pharmacies with that Corp ID in the individual file.
Individual Pharmacies - Store Name, Address, Amount, Corp ID if necessary, NABP and the sum of amount for claims

A sample batch summary report:

NEW YORK STATE DEPARTMENT OF HEALTH

INTEROFFICE MEMORANDUM

TO: Cheryl Mattox - Controller HRI
FROM: Julie Vara - Assistant Director
DATE: 11/5/2014
RE: Pharmacy Batch #1258

Enclosed with this summary batch transmittal are sub-reports that detail pharmacy expenditures by funding entity for the above numbered batch.

They are:

NYC billable
NYC non-billable (Maintenance of Effort)
Rest of State
Pharmacy Medicaid Spenddown

Nutritional supplement and vitamin expenditures for

NY City
NY City Non-Billable
Rest of State
Nutritional Medicaid Spenddown.
This batch was processed during the period of October 29, 2014 through November 5, 2014.

Note the grant# years to be charged for these service expenditures are listed below and are based on participant residency, dates of service and adequacy of grant funding.

Location		Dollar Amount	# Claims	Grant
New York City Billable -
Post	4/1/2014	$ -		19-0026-08 (Cat 7410)	Recoveries - 340B
		$ -		19-0027-08 (Cat 7410)	Recoveries - Supplemental
		$ 48,729.69		14-2343-23 (Cat 7410)	Federal - NYC
		$ 38,566.94		15-3360-24(Cat 7410)	Federal State
		$ 5,726.10		14-2430-03 (Cat 7410)	HCRA - FFP
		$ 4,796,630.00		19-0026-08 (Cat 7410)	Recoveries - 340B
		$ 2,778.84		19-0027-08 (Cat 7410)	Recoveries - Supplemental
Pre	4/1/2014	$ -		19-0027-08 (Cat 7410)	Recoveries - Supplemental
		$ -		15-3360-23 (Cat 7410)	Federal State
Pre	3/1/2014	$ -		14-2343- 22(Cat 7410)	Federal - NYC
		$ -		19-0027-08 (Cat 7410)	Recoveries - Supplemental
			$ 9,523	# of claims -
NYC Non-Billable -		$ -		15-3360-24 (Cat 7410)	Federal State
Post	4/1/2014	$ 666.98		19-0026-08 (Cat 7410)	Recoveries - 340B
		$ -		14-2430-03 (Cat 7410)	HCRA
Pre	4/1/2014	$ -		19-0027-08 (Cat 7410)	Recoveries - Supplemental
		$ -		14-2430-03 (Cat 7410)	HCRA
			4	# of claims -
Rest of State
Post	4/1/2014	$ -		15-3360-24 (Cat 7410)	Federal - State
		$ 1,127,556.63		19-0026-08 (Cat 7410)	Recoveries - 340B
		$ -		19-0027-08 (Cat 7410)	Recoveries - Supplemental
		$ -		14-2430-03 (Cat 7410)	HCRA
		$ -		15-3356-24 (Cat 7410)	Federal - State
Pre	4/1/2014	$ -		19-0027-08 (Cat 7410)	Recoveries - Supplemental
		$ -		19-0026-08 (Cat 7410)	Recoveries - 340B
		$ 6,020,655.18	3,556	# of claims -
Medicaid Spenddown
Post	4/1/2014	$ 324,063.55		14-2430-03 (Cat 7420)	HCRA - no FFP
Pre	4/1/2014	$ -		14-2430-03 (Cat 7420)	HCRA
# of claims			365	# of claims -
	Batch Total	$ 6,344,718.73

Step 5. ADAP fiscal staff review the batch memo and available funding to determine the payment allocation. All funds allocated to the programs are categorized in separate grant accounts. Payment and grant allocation decisions are based on the business rules associated with the grant, the funding period and available funds within the accounts.

Step 6. HCRA funds are used to reimburse pharmacies for eligible pharmacy claims when HCRA funds are available.

Step 7. ADAP uses the batch summary report to generate a pharmacy payment request that is submitted to HRI. HRI reviews the batch summary report and electronic pharmacy payment request for accuracy and agreement and generates the checks for individual pharmacy payments and charges the accounts specified in the memo. Checks are generated by HRI and sent to ADAP for distribution.

Step 8. All funds allocated for this purpose are accounted for through a unique and segregated grant.

Step 9. Funds are booked against account codes within the grant and a quarterly report is generated that details eligible expenditures for matching purposes. No claims paid with Federal dollars are included in the quarterly report.

Step 10. Claims allocated to federal funding sources are identified through separate and distinct accounts/grants and are not comingled with state expenditures.

Step 11. Health Research Inc. sends periodic memos to the DOH Deputy Commissioner for Administration requesting draws of ADAP funding within available appropriated/cash amounts for each fiscal year. Such requests are processed as payments to HRI through the SFS using discrete codes identifying them as ADAP expenditures. These amounts are not the amounts included as the DSHP claims. As outlined elsewhere in this protocol narrative, HRI uses these dollars for a variety of ADAP activities, some of which are not allowable costs for DSHP claims (e.g. Medicaid spenddown). The records of certified claimable costs are retained by HRI. HRI sends the DOH Bureau of Medicaid Financial Management a DOH 4103 claim form on a quarterly basis certifying the allowable DSHP costs/claims.

Step 12. The state submits a claim to CMS for FFP based on the total computable expenditure incurred by the State in making the eligible payment to DSHP provider. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Step 13. The State attests expenditures used are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2.

Complete pharmacy payment guidelines and participation criteria are in the ADAP Pharmacy manual online.

Non-Matchable Expenditure List:

Pertinent staff from the AIDS Institute and the Fiscal Management Group reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the AIDS Drug Assistance Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. All expenditures on these contracts are related to grants and there is no Federal Financial Participation (FFP) related to the HIV Uninsured Care Programs (HCUP). The 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual costs can be made for the claims. No Federal Financial Participation is being claimed for the following in accordance with Special Terms and Conditions paragraph 15 sections A-F of Waiver 11-W- 00114/2.

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants are segregated by separate grant accounts and no Federal Financial Participation is being sought for those claims.

State Agency: Department of Health

Program: Health Workforce Retraining Initiative

Program Codes: SFS 29879

Funding Sources: State Special Revenue - HCRA Resources Fund

$26,817,000 - State Appropriation

Chapter 53 of the Laws of 2014; page 564, lines 10-30

Process to identify program net deficit payments--N/A since there are no net deficit payments associated with this program group.

Brief Description:

The Health Workforce Retraining Initiative supports the training and retraining of health industry workers with the skills necessary in the health care market today. Since its inception, the program has awarded nearly $379 million to 552 grantees and trained or retrained over 150,000 health care workers.

The New York State Departments of Health and Labor are jointly soliciting applications from organizations proposing to train and retrain (hereafter referred to as "train") health industry workers to obtain new positions; meet the new job requirements of existing positions; or otherwise meet the requirements of the changing health care market.

Funds will be used to support efforts to address:

Changes in the skills required for public health and healthcare workers to maintain current employment including meeting new job or certification/licensing requirements.
Additional skills needed for a new job due to changes in the marketplace, including new employment for at-risk/laid off workers.
Occupational Shortages.
Changes in skills required to support new models of integrated care management and interdisciplinary team-based care and the linkage between population health and health care services.
Development of home and community-based long-term care. The need for long-term care workers who can assist patients to remain in their homes and communities, rather than be treated in more intensive settings.
Diversity in the health care and public health work force.
Additional skills needed to comply with the National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care´s Principal Standard "To provide effective, equitable, understandable, and respectful quality of care and services that are responsive to diverse cultural health beliefs and practices, preferred languages, health literacy and other communication needs".
Additional skills needed to enhance providers´ awareness of interventions, resources and tools for integrating health literacy into practice.

Eligible Population:

Funding preference will be given to:

Provide training in occupations with documented shortages.
Target workers who have experienced, are likely to experience job loss, or are recipients of public assistance programs due to changes in the public health or health care system.
Provide written labor union concurrence from the relevant bargaining agent(s).
Provide needed expansion of educational capacity.
Provide training to meet increased job or certification/licensing requirements in current position.
Promote the diversity of the healthcare workforce, e.g., by providing a pipeline program for students to increase racial and ethnic diversity in public health and biomedical sciences.
Promote the provision of culturally competent patient centered care, i.e., care responsive to diverse cultural health beliefs and practices, levels of health literacy, and preferred languages. May include foreign language, translation and interpretation skills, and strategies that enhance the public health workforce´s ability to meet the National Standards for Culturally and Linguistically Appropriate Services (CLAS) in Health and Health Care.
Provide training which promotes the development of new models of integrated care management, such as medical homes, health homes, or interdisciplinary team-based care, for example care coordinators, community health care workers, chronic disease managers, and linkages between population health and health care.
Provide training opportunities to increase awareness and understanding of health disparities.
Promote improved quality and outcomes of care through training in the effective reporting, analysis and use of data collected by health information technology (HIT) applications.
Provide training in the effective use of telemedicine to improve access to critical services.
Prepare workforce for participation in managed long-term care.
Leverage other resources to make projects more cost effective.

Statewide Financial System (SFS)/Claiming Process:

The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Department of Health enters vouchers into this system for payment. The Office of State Comptroller (OSC) reviews, approves and then processes such payments. The OSC process includes budget checks against appropriations and segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

Step 1. Claims for reimbursement are submitted by the contractors/grantees within 30 days after the end of each quarter via a NYS standard voucher with an expenditure report, progress report and supporting documentation to the DOH.

Step 2. A DOH contract manager reviews the information and enters the claim into the SFS system. Upon approval, vouchers are paid within 30 days of receipt with the quarterly value of the advancement (if applicable - see below) netted out. Payment is disbursed through the SFS system. For all providers, the DOH keeps the State standard voucher, expenditure reports, narrative expenditure report, and the submitted claim.

Step 3. The following Chart of Accounts is used to code reimbursement claims for this program: Department-3450366; Program-27866; Fund-20807; Budget Reference- 2014-15 (dependent on claim period); Chart field 1-11850 and Account-60301.

Step 4. The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

Step 3. DSHP expenditures for this program will constitute the voucher payments and advances described and paid through this process.

Step 4. The state submits a claim to CMS for FFP based on the total computable expenditure incurred by the State in making the eligible payment to DSHP provider. The expenditure claims must be claimed in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Step 5. The State attests expenditures used are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2.

Funding for programs is determined as part of the annual budget process whereby Executive and Legislative staff negotiate funding levels for programs based on desired goals/objectives and available resources/revenue. The Health Workforce Retraining Initiative program is funded by a State special revenue other fund (HCRA Resources) aid to localities appropriation. The Department of Health (DOH) awards these grants to eligible organizations. Such organizations include hospitals, nursing homes, home care providers and other providers, educational institutions, and labor management organizations. DOH enters into competitively bid contracts with the noted organizations.

The Health Workforce Retraining Program normally issues 2-year grant awards and contracts, with annual budgets that are typically, but not always, equal in size, via the RFA process. Awards are determined using an award process that is described in the program RFA and approved through the Office of the State Comptroller via the Grant Award Package. Basically each applicant with a combined technical and financial score above 70 receives an amount held to regional initial award ceilings increased proportionally if residual funds remain after the distribution of initial regional award amounts. Budgets are negotiated with awardees. The total contract budget amount cannot exceed the award amount. Grantees must comply with EO-38 regarding administrative costs and executive compensation.

Quarterly Expenditure based budget forms and progress reports for reporting purposes. DOH also collects Outcome data on how many participants begin and complete specific types of training programs. Contractors are required to maintain supporting documentation on file with the types of documentation specified in their contracts. DOH contracts with the Department of Labor to review supporting documentation via a contractor assistance program review process.

At the discretion of New York State, Not-for-Profit (NFP) organizations receiving grants under this program are eligible for contract advances of up to 25% of the value of the first-year of the 2-year contract. If an advance request from a NFP is granted, upon execution of the contract, it is paid through the SFS by the processing of an advance claim voucher submitted by the NFP. As noted above, these 2-year contracts have individual annual budgets, so the level of the advance in relation to the total contract value may vary. The advance is fully recouped in equal quarterly increments over the first year claims of the contract (i.e. it is collected over the first four quarterly claims @ 25% a claim.) The quarterly amount of the advance due is netted out of the full claim value for each quarter and the balance of the payment is processed through the SFS.

There is also an MOU between DOH and the NYS Department of Labor (DOL) where DOL reviews supporting documentation of the noted grantees/contractors through Contract Assistance Program (CAP) reviews. Such contract reviews ensure that proper documentation exists regarding the program requirements, applicable laws and the types and amounts of reimbursement claimed. In addition to the CAP reviews, a Yellow Book or A133 audit is performed on an annual basis to ensure proper documentation of the claims submitted.

Non-Match Able Expenditure List for the Waiver:

Pertinent staff from the Office of Primary Care and Health Systems Management and the Fiscal Management Group reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Health Workforce Retraining Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. All expenditures on these contracts are related to grants to organization to provide training services to current health care industry workers only. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. It is noted that such training services are not provided to undocumented individuals. Individuals receiving such training are health care industry workers who are either US citizens or individuals who are in this country legally. Financial Participation (FFP) is not being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No. New York State costs for administering this grant program are not claimed.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: Tobacco Use Prevention and Control

Program Codes: SFS 29549

Funding Sources: General Fund

$33,144,000 - State Appropriation

Chapter 53 of the Laws of 2014; page 555, lines 21-24

Process to identify program net deficit payments -- N/A since there are no net deficit payments associated with this program group.

Brief Description:

The New York State Department of Health envisions a tobacco-free society for all New Yorkers. The Bureau of Tobacco Control administers the state´s Tobacco Control Program (TCP) to reduce illness, disability and death related to tobacco use and secondhand smoke exposure, and to alleviate the social and economic burdens caused by tobacco use. TCP uses an evidence- based, policy-driven and cost-effective approach to decrease tobacco initiation by youth, motivate adult smokers to quit and eliminate exposure to secondhand smoke.

Since TCP´s beginning in 2000, the program has effectively implemented a strong clean indoor air law, maintained the highest state tobacco taxes in the nation to keep the price of tobacco high, strongly enforced laws that restrict minors´ access to tobacco and increased access to effective cessation services.

Because of TCP´s efforts and actions, youth and adult smoking rates in NYS are at historically low levels that are declining at rates that outpace national rates of decline.

Research demonstrates that exposure to tobacco product displays helps normalize smoking behavior, distorts adolescents´ perceptions of the availability of tobacco, and increases susceptibility to smoking. New York State Department of Health contractors use community education and mobilization strategies to educate the public about the impact of tobacco product displays and potential solutions. Since 2010, the New York State Adult Tobacco Survey has measured adults´ support for point of sale (POS) policies such as (a) restricting the number, location, and type of tobacco retailers; and (b) keeping tobacco products out of sight in retail locations open to youth.

In 2012, 67.9% of New York adults favored a policy that would prohibit tobacco sales near schools. This relatively high level of support may relate to the strong belief that seeing tobacco products displayed and advertised in retail stores affects youth smoking. Between 2010 and 2012, there was a statistically significant upward trend in the percent of adults who favor policies that limit the number of licensed tobacco retailers in New York. Between 2010 and 2012, there was a statistically significant upward trend in the percent of adults who favor policies that prohibit pharmacy sales, prohibit tobacco displays, and prohibit sales near schools in both the United States and New York.

Eligible Population:

The eligible populations associated with the Tobacco Use Prevention and Control includes the following groups:

Youth and young adults, especially among low socioeconomic status (SES) populations.
Adults with low incomes, low educational attainment or serious mental illness.

Statewide Financial System (SFS)/Claiming Process:

When the State transitioned into SFS, DOH established unique program codes within SFS for all its programs in order to track disbursements. The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

This program consists of RFA competitive procurements (6), RFP bidding procurements (5), one memorandum of understanding (MOU) and one NPS supply purchase related to the activities conducted under a related RFP. The program is administered through contracts with local health departments, not for profits, the State University of New York (SUNY), the Board of Cooperative Educational Services (BOCES), the Cornell Cooperative Extension and one State Office of General Services preferred contract for the NPS purchase. The six procurements through the competitive application process account for a total of 129 contracts and the five procurements through the competitive bidding process account for 5 contracts. The MOU is with the Department of Health´s School of Public Health.

Funding for the program is through a State budget appropriation that is derived from the State budget process and enacted via State legislation.

Step 1 The Department of Health uses the SFS to process contracts and vouchers for the review, approval and payment by the State Comptroller.

Step 2 Contractors are required to submit monthly claims for reimbursement which are certified by the contractor. Contractors are required to supply supporting documentation for actual expenditures in accordance with unit policies.

Step 3 After review and approval by the assigned contract/program manager, the claim is "keyed" into the SFS system by the Fiscal Officer and then reviewed and approved for payment within the Division´s fiscal unit as follows with an example of Journal Transaction in State Financial System.

Amount	Quantity	GL Unit	Dept.	Program	Fund	Account	Bud Ref
$16,288	1.0000	DOH01	3450268	27288	1000	60301	2014-15

Step 4 The claim is reviewed and if approved by the State Comptroller´s Office in SFS; paid via ACH or check within 30 days of receipt of claim for payment. Any claims that per the State Comptroller´s Office require follow-up to pay are assigned to program staff to resolve the issue(s) preventing such payment.

Step 5 The fiscal unit retains copies of claims for payment, budget statements/reports of expenditures, and all correspondence and supporting documentation required for approval of the claim. In addition, claims are entered electronically into a calculation workbook on SharePoint.

Step 6 The state submits a claim to CMS for FFP based on the total computable expenditure incurred by the State in making the eligible payment to DSHP provider. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Step 7 The State attests expenditures used are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2.

SFS Chart of Accounts Coding for the Anti-Tobacco programs for FY 14-15:

Advancing Tobacco Free Communities		Independent Evaluation of the New York State Tobacco Control Program
Department	Department	3450268
Program	Program	29474
Fund	Fund	10000
Account	Account	60301
Bud Reference	Bud Reference	2014-15
Project ID	Project ID	N/A
Activity	Activity	N/A
Chartfield 1	Chartfield 1	11850

Health Systems for a Tobacco Free NY	NY Smoker´s Quitline
Department	Department	3450268
Program	Program	29475
Fund	Fund	10000
Account	Account	60301
Bud Reference	Bud Reference	2014-15
Project ID	Project ID	N/A
Activity	Activity	N/A
Chartfield 1	Chartfield 1	11850

Community Partnerships	Cessation Centers
Department	Department	3450268
Program	Program	29472
Fund	Fund	10000
Account	Account	60301
Bud Reference	Bud Reference	2014-15
Project ID	Project ID	N/A
Activity	Activity	N/A
Chartfield 1	Chartfield 1	11850

Youth Action Program	Tobacco Control Training Center
Department	Department	3450268
Program	Program	29476
Fund	Fund	10000
Account	Account	60301
Bud Reference	Bud Reference	2014-15
Project ID	Project ID	N/A
Activity	Activity	N/A
Chartfield 1	Chartfield 1	11850

Center for Excellence in Tobacco Policy
Department	3450268
Program	29471
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

There are also selective Department of Health audits that are done on this program. Specifically, the review of the organization is completed programmatically on-site every 18 to 24 months by the assigned contract/regional manager and fiscally in-house through a voucher trace process that covers a sample period selected by the program. The contractor is then required to provide full supporting documentation for the respective claim period to include invoices and proof of payment for all NPS expenditures as well as copies of payroll records and related fringe and payroll tax records.

No advances are given for this program.

Per OMB circulars, contractors that meet the required threshold are required, on an annual basis, to engage an independent auditor to perform a Yellow Book or A133 audit to ensure proper documentation of the claims submitted.

Non-Matchable Expenditure List:

Pertinent staff from the Division of Chronic Disease Prevention and the Fiscal Management Group reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Tobacco Use Prevention and Control Program. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. The 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual costs can be made for the claims. Federal Financial Participation (FFP) is not being claimed for the following items:

Grant funding to test new models of care —No
Construction costs (bricks and mortar) —No
Room and board expenditures —No
Animal shelters and vaccines —No
School based programs for children —No
Unspecified projects —No
Debt relief and restructuring —No
Costs to close facilities —No
HIT/HIE expenditures —No
Services provided to undocumented individuals —No
Sheltered workshops —No
Research expenditures —No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development —No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave —No
Revolving capital fund —No
Expenditures made to meet a maintenance of effort requirement for any federal grant program —No
Administrative costs —No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) —No
Cost of services for which payment was made by Medicare or Medicare Advantage —No
Funds from other federal grants —No

State Agency: Department of Health

Program Group: Early Intervention Program

Program Codes: SFS 26825 (base funding); SFS 26601 (additional funding)

Funding Sources: General Fund - State Appropriations

$163,500,000 - Chapter 53 Laws of 2014; page 552, lines 3-18 $3,900,000

- Chapter 53 Laws of 2014; page 552, lines 19-20

The Early Intervention Program is financed through a combination of commercial third-party insurance, Medicaid, as well as state and county funds totaling approximately $650 million annually. The State of New York reimburses municipalities for 49% of cost, after third-party insurance and Medicaid is deducted, through a state vouchering process via OSC. In FFY 2014, the non-Medicaid appropriation for FFY 2013-14 was $163.6 million in State and $170.1 million in local funding.

Process to identify the value of applicable offsets: The 10% undocumented immigrant offset is applied unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS.
Process to identify program net deficit payments--N/A since there are no net deficit payments associated with this program group.

Brief Description:

Part C of the Individuals with Disabilities Education Act established the Early Intervention Program (EIP) to provide a comprehensive system of early intervention services for infants and toddlers with disabilities and their families. The New York State Department of Health (NYSDOH) is designated as the lead agency responsible for general administration, supervision and oversight of New York State´s EIP. Its mission is to identify and evaluate those children whose healthy development is compromised and provide for appropriate interventions to improve child and family development.

The EIP is administered locally by 57 counties and New York City. Children are referred to receive a timely evaluation. If a child is found eligible, an Individualized Family Service Plan (IFSP) is developed and services are provided. The EIP provides for appropriate interventions to improve child health and family development through a wide range of therapeutic and supportive home and community-based services such as speech, physical therapy, and psychological services.

All individuals and agencies providing Early Intervention services, evaluations or service coordination are approved by the Department of Health and enter into agreements with NYSDOH to deliver EIP services. Individualized family service plans are developed for eligible children and families. Local public agencies which administer the EIP arrange for providers to deliver services in children´s IFSPs.

The Department receives a federal grant from the U.S. Department of Education, Office of Special Education Programs, of approximately $24 million annually to administer the EIP. Of these funds, approximately $5.1 million is allocated to support local administrative activities via contracts with the local health departments. The balance of the federal grant supports personal service with related fringe benefits and indirect costs; training and quality improvement (monitoring) contracts; data management; public awareness; advisory council expenses; interagency agreements, and the state´s fiscal agent.

In addition to the aforementioned grant, the program is also financed through a combination of commercial third-party and State and local district/county funds (all non-Medicaid) totaling about $333.7 million (State and local). Such monies are for developmental services and evaluations. The EIP is administered locally by 57 counties and New York City. Through contracts with the local districts, the State reimburses districts for services rendered by providers.

The State of New York reimburses municipalities for 49% of cost, after third-party insurance and Medicaid are deducted, through a State vouchering process via the Office of the State Comptroller and sent to SFS. EI Providers enter into agreement with the State and submits claims via the State Fiscal Agent (SFA). The SFA submits the EI provider´s claims to commercial insurance and Medicaid. Any unreimbursed cost are then paid to the provider out of the county escrow account. The SFA, Public Consulting Group (PCG), sends in escrow invoice to the municipalities (local districts) for the amount due, municipalities then submit the amount owed to the escrow account at Key Bank and payments are made directly to providers.

Early Intervention Initiatives

Reimburse Municipalities for Provider Claims

These funds are used to reimburse municipalities 49% of eligible provider claims for providing Early Intervention services.

General Fund - Aid to Localities

Early Intervention Program State (SFS coding)

4/1/14-3/31/15

Fundline	1	2	3	4	5	6
Department	3450258	3450258	3450258	3450258	3450258	3450258
Department	3450261	3450261	3450261	3450261	3450261	3450261
Program	26733	26734	29321	29322	29224	29225
Fund	10000	10000	10000	10000	10000	10000
Account	60301	60301	60301	60301	60301	60301
Budget Ref	2012-13	2012-13	2012-13	2012-13	2012-13	2012-13
	2013-14	2013-14	2013-14	2013-14	2013-14	2013-14
	2014-15	2014-15	2014-15	2014-15	2014-15	2014-15
Project ID	N/A	N/A	N/A	N/A	N/A	N/A
Activity	N/A	N/A	N/A	N/A	N/A	N/A
Chartfield 1	11850	11850	11850	11850	11850	11850

Eligible Population:

A child needs to meet NYS Early Intervention criteria of eligibility which includes a confirmed developmental disability and/or a diagnosed physical or mental condition that has a high probability of resulting in developmental delay. For 2013 in New York State, 4.3% of children under the age of three were enrolled in the Early Intervention Program. Children in EIP are covered from birth through age three. In 2013, services were provided to approximately 68,000 children and their families of which 40,000 were Medicaid (Medicaid Enrolled not 1915(C)) and 32,000 non-Medicaid children.

Statewide Financial System (SFS)/Claiming Process:

Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

The EI Program is administered through agreements with clinicians and clinical agencies, commonly referred to as "providers". These providers enter the information about the service delivery in the New York Early Intervention System (NYEIS) Data System, which is used to track and verify payment for services as well as to collect case and programmatic information. However, NYEIS is not an accounting and payment system; those functions are performed by the EI Program State Fiscal Agent (SFA). The providers are responsible for submitting billing to commercial insurance and Medicaid via the SFA.

For any claim that is not paid or partially paid by commercial insurance and Medicaid, the municipalities must pay the provider in the first instance. Finally, the State reimburses the municipalities 49% of those claims (not paid by commercial insurance or Medicaid). To accomplish this payment to municipalities, the provider must submit a claim to the SFA. The SFA pays the unreimbursed amount using municipal funds held in a State escrow account. The SFA then produces reports which detail the amounts due to each municipality for the State´s 49% reimbursement. Payment to the municipalities is made through the SFS. These State payments are used for DSHP claiming and are accounted for in SFS.

Funding for the program is through a State budget appropriation that is derived from the State budget process and enacted via State legislation.

Step 1 The Department of Health uses the SFS to process vouchers for the review, approval and payment by the State Comptroller.

Step 2 For the majority of program payments to municipalities, they are calculated based on data already collected by the SFA. For claims related to services before the SFA was implemented, municipalities are required to submit monthly claims for reimbursement. Municipalities are required to supply supporting documentation for actual expenditures in accordance with unit policies.

Step 3 After review and approval by the assigned contract/program manager, the claim is downloaded into a bulk-load transaction file for encrypted electronic transfer to the SFS system by the Fiscal Officer and then reviewed and approved for payment within the Division´s fiscal unit as follows with an example of Journal Transaction in State Financial System.

Amount	Quantity	GL Unit	Dept.	Program	Fund	Account	Bud Ref
$1,406,877	1.0000	DOH01	3450261	26733	10000	60301	2014-15

Step 6 The state submits a claim to CMS for FFP based on the total computable expenditures incurred by the State in making the eligible payment to DSHP provider. Payments to municipalities for administration of the program typically amount to less than 1% of program expenditures; any such expenditures will be specifically analyzed and excluded from the claim to CMS. The costs for program administration can be clearly identified as they are authorized through separate contracts that can be isolated and excluded. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Non-Match Able Expenditure List for the Waiver:

Pertinent staff from the Early Intervention Program and the Fiscal Management Group reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Early Intervention Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. All expenditures are for EI administrative contracts and reimbursement to municipalities are for EI cost. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. For item J, the 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual

costs can be made for the claims. For this program, it is anticipated that the percentage will be significantly lower due to the services provided being for young children, most of which were born in the U.S.. Federal Financial Participation (FFP) is not being claimed for the following items:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No. New York State costs for administering the program are not claimed.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

B. Office of Children and Family Services (OCFS)

State Agency: Office of Children and Family Services

Program: Services to Special Education Children<

Program Codes: SFS 13919

Funding Sources: ACS Schedule K, Section 2, Lines 9A (Blind and Handicapped) and 9B (All Other)

$163,500,000 - Chapter 53 Laws of 2014; page 552, lines 3-18 $3,900,000

- Chapter 53 Laws of 2014; page 552, lines 19-20

Process to identify program net deficit payments— N/A since there are no net deficit payments associated with this program group.

Brief Description:

Pursuant to Chapter 563 of the Laws of 1980, OCFS is responsible for the reimbursement of Local Departments of Social Services (LDSS) maintenance costs of CSE placements in residential schools and state-operated schools for the deaf or blind with approved programs in accordance with Chapter 853 of the Laws of 1976. The fiscal responsibility for the maintenance payment now rests with the LDSS, while the placement decision rests with the Committee on Special Education of the local school district. The LDSS continues to have no standing in relation to the individual educational plan for the child, including placement decisions; however as mentioned above, the local school district is now fiscally responsible for 38.424 percent of the maintenance payment associated with placement decisions.
As per the terms of the Center for Medicare and Medicaid Services Special Terms and Conditions Waiver #11-#-00234/2 Federal -State Health Reform Partnership (FSHRP) Medicaid Section 1115 Demonstration, the State is allowed to claim expenditures related to a number of designated State health programs, that are not currently matched with federal financial participation to fund a group of initiatives to reform and restructure the State health care delivery system.
In 2006, the New York State Division of the Budget identified OCFS´ program for services to Special Education children as a State health program that previously had no matching federal financial participation that would be covered under this waiver initiative and eligible for federal participation dollars. These costs concern maintenance expenditures specifically for children who are blind and handicapped.
These costs are reported and compiled for OCFS via the Office of Temporary and Disability Assistance´s Automated Claiming System (ACS) on the Schedule K, Section 2,Lines 9A (State Operated Schools for the Blind and Handicapped) and 9B(State and Local funded Private Residential Schools for the Blind and Handicapped). The value of these Local Departments of Social Services (LDSS), New York City Agencies and St. Regis claims net of Cancellation (Stopped Payments) and refunds (monies repaid to the district for current to pervious care and maintenance expenditures) is reported to DOH on a quarterly basis for processing to the federal Department of Health and Human Services.
CSE programs provide much more than room and board. These programs provide services for children whose needs cannot be met at home. These programs are Therapeutic Residential

Treatment programs, where students receive necessary services both in school and in the residential program that enable the student to better function in everyday life.

In order to be eligible for CSE programs, the students must be classified as being a student with a disability as defined by SED Regulations Part 200.1 Section zz. According to this section, qualifying disabilities are autism, deafness, deaf-blindness, emotional disturbance, hearing impairment, learning disability, intellectual disability, multiple disabilities, orthopedic impairment, other health impairment, speech or language impairment, traumatic brain injury, or visual impairment. See that section for the specific definition of each disability.
Students in CSE programs are eligible to receive related services. Related services means developmental, corrective, and other supportive services as are required to assist a student with a disability and include speech-language pathology, audiology services, interpreting services, psychiatric and psychological services, physical therapy, occupational therapy, counseling services, including rehabilitation counseling services, orientation and mobility services, medical services as defined in this section, parent counseling and training, social work, assistive technology services, appropriate access to recreation, including therapeutic recreation, other appropriate developmental or corrective support services, and other appropriate support services and includes the early identification and assessment of disabling conditions in students according to SED Regulations Part 200.1 Section qq.
CSE programs are staffed at a much higher intensity than regular programs.
Some students in CSE programs are so severely disabled that they sometimes require 1:1 or 2:1 aides to be with them during the school day and evening hours or, in extreme cases, 24 hours per day.

Eligible Population:

CSE-The eligible population is comprised of children who possess a specific physical, mental, emotional condition, or disability of such severity or kind which in the opinion of the Department constitutes a significant obstacle to the child´s adoption and is defined as a handicapped child. This handicapped child may receive services via placement in a special act school, an 853 school, state-supported residential school, or a state-operated school for the deaf or blind by an entity other than the child´s school district of residence or via placement by a local school district´s Committee on Special Education in an approved private residential school. Placement decision rests with the Committee on Special Education of the local school district, up to age 21. CSE placed youth remain in the custody of their parents.

Blind and Handicapped - This handicapped child may receive services via placement in a state- operated school for the deaf or blind by an entity other than the child´s school district of residence or via placement by a local school district´s Committee on Special Education in an approved private residential school. Placement decision rests with the Committee on Special Education of the local school district, up to age 21. CSE placed youth remain in the custody of their parents.

NOTE: The OCFS population receiving these services is disabled youth deemed eligible for CSE programs. This population is comprised of children who possess a specific physical, mental, emotional condition, or disability of such severity or kind which the Department determines the child is unable to be maintained in his or her home or participate in receiving a free and appropriate education and, as such, is defined as a handicapped child. This handicapped child may receive services via placement in a Special Act school, an Article 853 school, a state- supported residential school, or a state-operated school for the deaf or blind (Rome/Batavia).

OCFS Claiming Systems:

There are two systems that control the reporting and claiming of these expenditures. The first system, the Automated Claiming System (ACS), documents claims by the Local Departments of Social Services (LDSS) for reimbursement of monies paid to the Voluntary Agencies (VAs) for CSE maintenance services rendered by the school district or VAs. The other system, the Statewide Standards of Payment (SSOP), captures utilization and program expense data submitted by the OPWDD private residential schools and OCFS-licensed residential CSE programs (However, the School for the Blind and School for the Deaf are administered by the State Education Department and report costs in the Consolidated Fiscal Report). The SSOP data for private residential schools licensed by OPWDD is used to develop the proxy disallowance of administrative and "room and board" costs.

Automated Claiming System (ACS)

The Office of Temporary and Disability Assistance´s Automated Claiming System (ACS) tracks the value of CSE claims for the Local Departments of Social Services (LDSS), and New York City Voluntary Agencies; net of Cancellation (Stopped Payments) and refunds (monies repaid to the LDSS for current to pervious care and maintenance expenditures). These claims are reported to DOH on a quarterly basis for processing to the federal Department of Health and Human Services. The specific data is located on the Schedule K, Section 2, Lines 9A (Blind and Handicapped) and 9B (All Other) for the Committee on Special Education.

Statewide Standards of Payment (SSOP)

SSOP is the rate setting system of the Office of Children and Family Services (OCFS) Bureau of Budget Management (BBM). OPWDD Private Residential Schools (P Schools) and Voluntary Agencies (VAs) that provide special education services to OCFS youth submit program and fiscal data to BBM on an annual basis. The data is used to calculate a reimbursement rate for each program, called a Maximum State Aid Rate (MSAR). The agency uses the MSAR to bill LDSS for the services rendered to each youth. The MSAR for P Schools includes specific medical costs incurred by the agency providing medical services to their youth. Medical services may include costs for physicians, psychologists, nurses and nurse practitioners, medical technicians, and psychiatrists; as well as clinical social services staff.

In contrast to the P schools, the VA data for OCFS-licensed residential CSE programs does not include program specific medical costs, as the VAs report Medical costs in an aggregate cost center. However, DOH receives submissions of program specific medical costs through the

EMedNY system. Rates for CSE youth placed at these agencies are calculated by adding the program specific DOH Medicaid per diem rate to the agency´s MSAR rate for each program to develop a CSE maintenance rate.

OCFS Claiming Process:

Step 1 Bills for services provided are submitted by the Foster Care Voluntary Agencies statewide for reimbursement by the LDSS.

Step 2 LDSS reviews the information and enters the claim into the ACS system. Entering the claim into the ACS allows the LDSS to request reimbursement for the State portion of the claim, which the LDSS pays in the first instance. These costs are entered by the LDSS on Schedule K, Section 2, Lines 9A (Blind and Handicapped) and 9B (All Other) for the Committee on Special Education.

Step 3 The ACS system tracks the claims submitted by the LDSS for state reimbursement. The Office of Temporary and Disability Assistance (OTDA) processes the claim for reimbursement.

Step 4 OCFS reports the net claims annually to DOH, reduced by the 30% room and board proxy, as the basis for the OCFS portion of the DSHP claim amount.

Non-Match Able Expenditure List:

Staff reviewed the attached list of non match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Services to Special Education Children Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T, as the portion OCFS estimates is attributable to room and board has been removed as a proxy; and administrative costs have been removed as part of that room and board proxy methodology. For item J, undocumented children are not eligible to receive CSE services, therefore the ten percent reduction is not applicable. OCFS costs for administering the program are not included in the DSHP claims. Financial Participation (FFP) is not being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No, NYS applies a proxy methodology to exclude room and board from claiming identified on the next page of the protocol below.
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No, this is included with room and board which is excluded from claiming through the room and board proxy methodology which is described below.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

Room and Board Proxy Methodology:

The CSE eligible population is composed of children who possess a specific physical, mental, emotional condition, or disability of such severity or kind which in the opinion of the Department constitutes a significant obstacle to the child´s ability to be maintained in his or her home or participate in receiving a free and appropriate education, and is defined as a handicapped child. This handicapped child may receive services via placement in a Special Act school, an Article 853 school, a state-supported residential school, or a state-operated school for the deaf or blind (Rome/Batavia).

Maintenance and education services are provided by an entity other than the child´s school district of residence via placement by a local school district´s Committee on Special Education in an approved private residential school. The placement decision rests with the Committee on

Special Education of the local school district, up to age 21. It is important to remember that CSE- placed youth remain in the custody of their parents.

It is not possible to determine an appropriate proportion of medical costs using the MSAR data, because program specific medical costs are not reported. To develop an appropriate proxy, the P School data must be used.

The P School data from rate year 2012-13 (7/1/12 - 6/30/13) showed $61,031,803 in total costs for those programs. Of that amount, $42,726, 863 were determined allowable expenses for DSHP. The non-allowable expenses for DSHP were $18,304,940, or 30% (see allowable/disallowable Expense/Cost Titles listed below).

Analyzing data reported by the P Schools shows that 70% of the costs submitted by the agencies are allowable under DSHP. The remaining 30% is applied as a non-allowable "room and board" proxy. This 30% proxy represents the high-end of the program costs. For many other program types, the percentage of room and board costs would be lower, so 30% is a strong proxy for removal of room and board costs. Please note that administrative costs have been removed as part of the room and board proxy methodology.

Undocumented children are not eligible to receive CSE services, therefore the ten percent reduction is not applicable.

Allowable Expense/Cost Titles include:

Child Care
Transportation
Social Services
Utilities
Supplies and Equipment
Equipment/Vehicles
Medical

Non-Allowable Expense/Cost Titles include:

Food and Child Support
Miscellaneous (Activities, Clothing, etc.)
Maintenance
Office
Fixed Property
Administration

C. New York State Office for the Aging (NYSOFA)

State Agency: New York State Office for the Aging

Program: Expanded In-Home Services to the Elderly Program (EISEP)

Program Code: SFS 10381

Funding Sources: State General Fund

$48.1 million - State Appropriations

Chapter 53 of the Laws of 2014; Page No. 4, Line No. 7; and Page No. 8 line Nos. 27.

Process to identify the value of applicable offsets: To account for undocumented individuals, a ten percent reduction will be applied to all claims unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS. There are no additional offsets applicable to this program.

Brief Description:

EISEP is a community-based long-term care program that provides case management, non-medical in-home, non-institutional respite, and ancillary services needed by NY functionally impaired residents aged 60 and over. It is intended to fund an array of non-medical supportive services to older persons in their own homes and apartments who are not eligible for similar services under Medicaid. Services include housekeeping, personal care, respite, case management and ancillary services (such as emergency response systems). To be eligible for EISEP, older adults must need assistance in everyday activities of daily living and can be maintained safely at home in the community. Based on their incomes, consumers may cost share according to a sliding scale. This program is targeted to those with low income. This program helps to comply with provisions of the Olmstead decision, delays nursing home placements and delays the individual´s spend down to Medicaid eligibility.

NYS Department of Health provides personal care services for Medicaid-eligible persons. In order to be eligible for such services the Medicaid recipient must have a completed physician´s order. Services are coordinated through the Local Department of Social Services, who makes arrangements for a nursing assessment. The orders written by the physician along with the nursing assessment are used to determine the level of services needed. A reassessment of need is

completed every six months. Services provided under the program include, housekeeping, meal preparation, bathing, toileting, and grooming.

Eligible Population:

Older New Yorkers (60 years of age or older) are assessed for ADLs and IADLs to determine if services are needed. Although income is not a factor in determining program eligibility, their income is assessed and if their income exceeds 150% of the federal poverty level, the older New Yorkers are required to cost share based on a sliding scale where the older New Yorkers with the highest financial means share a larger percentage of the cost of the services. Older New Yorkers with incomes exceeding 250% of the federal poverty level may participate but they must cost share at 100% of the services costs, except for Case Management where the cost sharing rules don´t apply.

Statewide Financial System (SFS)/Claiming Process:

NYSOFA uses the SFS to process vouchers for the review, approval and payment by the State Comptroller. When the State transitioned into SFS, NYSOFA established unique program codes within SFS for all of its programs in order to track disbursements. The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

Under the EISEP Program, NYSOFA issues one-year grant award notices to our network of grantees. On an annual basis, NYSOFA allocates NYS General Fund monies appropriated in the enacted State Budget according to a population-based formula that is contained in NYS statute. The allocations are made to our 59 Area Agencies on Aging (AAAs). 52 AAAs are units of County Government, one AAA is a unit of NYC Government, two AAAs are units of Tribal Governments and four AAAs are not-for-profit agencies each serving a single NYS County. Based on these allocation amounts, the AAAs submit an annual plan and budget detailing the services that they will provide either directly or by contract with local service providers (mostly NFPs). Before developing the annual plan, each AAA conducts a public hearing. Based on the public hearings and the AAA´s knowledge of older New Yorkers and the service needs in their County, the plan is developed and submitted to NYSOFA for approval. NYSOFA then reviews and approves the plan before issuing the grant award notice. The AAAs and local NFP service providers then begin providing services to the older New Yorkers in their Counties.

The AAAs can spend up to $48.1 million for direct care services. NYS reimburses the AAAs for the net amount of their costs incurred, i.e., minus a required local match (25% or greater) and income generated by the program (generally participant contributions and cost share).

As the AAAs incur direct costs and reimburse their contractors for contractual costs, they submit reimbursement claims to NYSOFA at the end of each quarter via a paper copy of the NYS SFS Claim for Payment with expenditure detail attached. The claims include detailed schedules that link costs incurred to the line item budgets that were approved in the plan. There is a summary schedule that includes the costs incurred and budgetary comparison for Personal Services, Fringe Benefits, Equipment, Travel, Maintenance and Operations, Contracts and Other Expenses. Then there is a supporting Budget schedule that provides necessary details on line item costs included in the summary schedule.

Step 1 The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Office for the Aging enters vouchers into this system for payment. Prior to any entry of local program expenditures into the SFS, NYSOFA processes all claims through our "Grants" system, a home-grown grants management system.

Step 2 Upon receipt of each reimbursement claim from the local grantee, NYSOFA support staff that are responsible for mail distribution enter basic identifying information for each claim into NYSOFA´s "Grants" system, a home-grown grants management system.

Step 3 The claims are then reviewed by NYSOFA Professional Staff with a focus on cost allowability, allocability and reasonableness as well as other grant program requirements.

Step 4 Once ready for processing, the claims are coded to Department 1010224, Program 10381, Fund 10000 and account 60301 as well as the budget reference which is currently 2014-15 into the Grants System.

Step 5 The claims are then reviewed by a higher-level supervisor who approves the claim. Upon approval, more detailed information is entered into the Grants system which contains many automated error checks to ensure that nothing critical got past the staff review.

Step 6 On a daily basis the data in the Grants System that is approved for payment is downloaded into a bulk-load transaction file for encrypted electronic transfer to the SFS system where they are again audited by the Office of the State Comptroller (OSC).

Step 7 The Office of State Comptroller (OSC) reviews, approves and then processes such payments. The OSC process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

Step 8 Through this process vouchers are paid by the OSC through the SFS system within 30 days of receipt. If requested by grantees, NYSOFA will advance up to 25% of the annual grant award*. These payment are also disbursed through the SFS system. For all providers, NYSOFA keeps the Claim for Payment with expenditure detail on file for six years after the annual grant is closed out as the claim of record.

Step 9 The state submits a claim to CMS for FFP based on the total computable expenditure incurred by the State in making the eligible payment to DSHP provider. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Step 10 The State attests expenditures used are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2.

* At the discretion of NYSOFA, organizations receiving grants under this program are eligible to receive advances of up to 25% of the value of the annual grant award. If an advance request is granted, upon the issuance of the annual grant award document, the grantee submits a claim for NYSOFA to process and send through the SFS for payment. Each year, the advance is fully recouped by reducing the amount of reimbursement claims.

Non-Match Able Expenditure List:

NYSOFA has reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Health Workforce Retraining Program. For this particular program, it was determined that claimed expenditures may be made or are made for undocumented individuals but none of the other items listed in A through T. This is explained below. All expenditures on these grants are related to grants to organizations to provide services to Individuals described in the Eligible Population section above. While the program allows up to $2.2 million in local administration costs, those expenditures are not included in the $48.1 million of EISEP expenses that NYS will claim under this program. NYSOFA State Office costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. Financial Participation (FFP) is not being claimed for the following as in accordance with the Special Terms and Conditions paragraph 15 sections A-F of Waiver 11-W-00114/2.

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - Very infrequent - AAAs may not refuse services to undocumented individuals if they meet program eligibility requirements. Disbursements are certainly within the agreed upon percentage will be applied to all claims.
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No.
Administrative costs - No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: New York State Office for the Aging

Program: Community Services for the Elderly (CSE)

Program Code: SFS 10379

Funding Sources: State General Fund

$48.1 million - State Appropriations

Chapter 53 of the Laws of 2014; Page No. 3, Line No. 14; Page No. 7 line Nos. 41 and 44 and Page No. 8, Line No. 27.

Process to identify the value of applicable offsets: To account for undocumented individuals, a ten percent reduction will be applied to all claims unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS. There are no additional offsets applicable to this program.

Brief Description:

CSE is a broadly defined program than can pay for virtually all non-medical service needed by NY residents aged 60 and over, in their communities. The local Area Agencies on Aging (AAA) have considerable flexibility within the general parameters of the program. Services funded through CSE include: case management; personal care; caregiver services; congregate and home delivered meals; information and assistance; referrals; social adult day care; transportation; respite; telephone reassurance and friendly visiting; health promotion and wellness activities; senior centers and other congregate programs; personal emergency response systems; residential repairs; and legal services. Long Term Care Ombudsman is the only service funded by NYSOFA is not allowable under CSE. This is because the Ombudsman program operates in institutional settings and not in the Community.

Medicaid will pay for a host of services to assist individuals remain in their homes and in their community. Some of the special services available to participants in waivers that are similar to those provided under CSE are, home care, personal care aides, adult day care, and transportation to medical care mental health services.

Eligible Population: Adults aged 60 or older.

Statewide Financial System (SFS)/Claiming Process:

Under the CSE Program, NYSOFA issues one-year grant award notices to our network of grantees. On an annual basis, NYSOFA allocates NYS General Fund monies appropriated in the enacted State Budget according to a population-based formula that is contained in NYS statute.

The allocations are made to our 59 Area Agencies on Aging (AAA). 52 AAAs are units of County Government, one AAA is a unit of NYC Government, two AAAs are units of Tribal Governments and four AAAs are not-for-profit agencies each serving a single NYS County. Based on these allocation amounts, the AAAs submit an annual plan and budget detailing the services that they will provide either directly or by contract with local service providers (mostly NFP). Before developing the annual plan, each AAA conducts a public hearing. Based on the public hearings and the AAA´s knowledge of older New Yorkers and the service needs in their

County, the plan is developed and submitted to NYSOFA for approval. NYSOFA then reviews and approves the plan before issuing the grant award notice. The AAAs and local NFP service providers then begin providing services to the older New Yorkers in their Counties.

The AAAs can spend up to $23.4 million for direct care services. NYS reimburses the AAAs for the net amount of their costs incurred, i.e., minus a required local match (25% or greater) and income generated by the program (generally participant contributions and cost share).

Step 3 The claims are then reviewed by NYSOFA Professional Staff with a focus on cost allowability, allocability and reasonableness as well as other grant program requirements.

Step 4 Once ready for processing, the claims are coded to Department 1010224, Program 10379, Fund 10000 and account 60301 as well as the budget reference which is currently 2014-15 into the Grants System.

Step 8 Through this process vouchers are paid by the OSC through the SFS system within 30 days of receipt. If requested by grantees, NYSOFA will advance up to 25% of the annual grant award*. These payments are also disbursed through the SFS system. For all providers, NYSOFA keeps the Claim for Payment with expenditure detail on file for six years after the annual grant is closed out as the claim of record.

Non-Match Able Expenditure List:

NYSOFA has reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Health Workforce Retraining Program. For this particular program, it was determined that claimed expenditures may be made or are made for undocumented individuals and federal maintenance of effort but none of the other items listed in A through T. This is explained below. All expenditures on these grants are related to grants to organizations to provide services to Individuals aged 60 and over. While the program allows up to $2.2 million in local administration costs, those expenditures are not included in the $23.4 million of CSE expenses that NYS will claim under this program. NYSOFA State Office costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. Financial Participation (FFP) is not being claimed for the following as in accordance with the Special Terms and Conditions paragraph 15 sections A-F of Waiver 11-W-00114/2.

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No

Services provided to undocumented individuals - Very infrequent - AAAs may not refuse services to undocumented individuals if they meet program eligibility requirements. Disbursements are certainly within the agreed upon percentage will be applied to all claims.

Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - Yes $5 million per year.
Administrative costs - No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

I. DSHP List 2 Program Details

A. The Department of Health (DOH)

State Agency: Department of Health

Program: Childhood Lead Poisoning Primary Prevention Program

Program Code: SFS Code 32426

Funding Sources: Special Revenue Funds - Other

$9,891,300 (Department of Financial Services sub-allocation)

Chapter 53 of the Laws of 2015; page436, lines 30-37

Process to identify the value of applicable offsets: The 10 percent undocumented immigrant offset is applied unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS.
Process to identify program net deficit payments -- N/A since there are no net deficit payments associated with this program group.

Brief Description:

In an effort to eliminate childhood lead poisoning in New York State, the Childhood Lead Poisoning Primary Prevention Program´s (CLPPP) goal is to increase the availability and number of housing units that are free of lead-based paint hazards in targeted communities identified with high incidence of childhood lead poisoning. Approximately 3,000 children are diagnosed with lead poisoning each year in New York State. This exposure could result in long-term adverse health effects and substantial costs to the State and local governments.

In New York State, housing built before 1980 presents the greatest risk of exposure for children, six years of age and under, to lead contaminated dust from deteriorating lead-based paint. New York has the nation´s highest number (5.7 million) and highest percentage (72.1%) of this older housing stock. However, the incidence of lead poisoning is not evenly distributed across the state but is concentrated in identified communities with a higher percentage of old housing stock.

The Primary Prevention Program has made a significant difference in the lives of children and their families and in the infrastructure for primary prevention of lead-based hazards. Since its inception, on October 1, 2007, over 10,000 children have been directly affected by the Primary Prevention Program through visits to their homes, and nearly 6,000 have been referred for blood lead testing as a result of those visits. Over 21,000 housing units have been inspected, and nearly 10,000 of them were found to have potential and/or confirmed lead-based paint hazards. Lead safe work practices training courses provided training to 2,700 contractors and property owners. The Childhood Lead Poisoning Primary Prevention Program services are delivered through contracts with counties.

Each county must submit a work plan detailing specific tasks, desired outcomes and performance measures to address program requirements. Examples of these tasks may include: identify high-risk housing, conduct outreach efforts to propose local partnerships with regard to primary prevention. Counties will also inspect, notify and remediate lead paint hazards.

Eligible Population:

Children living in high risk housing in targeted geographic areas. Children aged 9 months to <36 months are the primary target age. Children at risk up to six years old are also eligible.

Statewide Financial System (SFS)/Claiming Process:

A Contract Reporter Exemption Request (CRER) was filed for the Child Lead Primary Prevention Program requesting a sole source waiver. The CRER was subsequently approved by the NYS Office of the State Comptroller (OSC). Contracts were initiated with local health departments based on a formula that directed funds to areas of greatest need.

State General Funds and Special Revenue Other funds are appropriated annually in the NYS Budget to fund the CLPPP with the appropriations defined in the budget legislation. Claims are submitted quarterly by Local Health Departments along with a Budget Statement and Report of Expenditures form (BSROE), and quarterly report. The BSROE details the personnel salary (as a percentage of time performing grant-related activity from timesheets) and non-personal service expenses the county is vouchering for in that quarter. The quarterly report outlines the activities performed during that quarter which warrant the payment requested by the county. Once these are received, they are reviewed by program staff to ensure work is performed as specified for that quarter and then voucher information is entered into SFS for payment. Electronic quarterly payments are made via SFS.

Voucher, Budget Statement and Report of Expenditures form, and quarterly reports are kept. Recently, a favorable audit report (July 23, 2014) was issued reflecting the results of voucher processing for the Bureau of Community Environmental Health and Food Protection programs. The local health departments are also audited by program staff on a biannual basis.

The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Department of Health enters vouchers into this system for payment. The OSC reviews, approves and then processes such payments. The OSC process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

The Department of Health uses the SFS to process contracts and vouchers for the review, approval and payment by the State Comptroller. Multiple coding options are used, including department, program, account, budget reference and agency-described chart field 1 to easily identify where charges belong. When the State transitioned into SFS, DOH established unique program codes within SFS for all its programs in order to track disbursements. The CLPPP program is uniquely defined in SFS with the following Chart of Accounts coding:

Department Program Fund Account Bud Ref

CLPPP-DFS 3450000 32426 21994 60303 2015-16

Step 1 The program administrator enters coding into SFS to set up a purchase order (PO). This PO is given a unique ID that vouchers can be input into SFS to spend against. This program is coded to Department 3450000, Program 32426, Fund 21994, Account 60303 and Bud Ref 2015-16. Bud ref is updated annually.

Step 2 Upon receipt of the reimbursement claims, the program administrator compares the voucher to the approved budget. The focus is on cost allowability, allocability and reasonableness as well as other grant program requirements. Budget modifications may be required.

Step 3 Once ready for processing, the claims are entered into SFS and coded to the unique PO number.

Step 4 The claims are then reviewed by the fiscal officer who approves the claim.

Step 5 The payment is disbursed through the SFS system. For all providers, the program administrator keeps the Claim for Payment with expenditure detail on file for six years after the annual grant is closed out.

Step 6 Once paid, vouchers are scanned into the Bureau of Account Management´s File Net system and given a barcode to match SFS transactions.

Step 7 The state submits a claim to CMS for FFP based on the total computable expenditure incurred by the State in making the eligible payment to DSHP provider. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Step 8 The State attests expenditures used are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B.

The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

No advances of funding are processed for this program since all recipients of program funding are NYS governmental entities and therefore not advance eligible.

Non-Match able Expenditure List:

Relevant staff from the Center for Environmental Health and the Fiscal Management Group reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions Paragraph 15 sections A - F of Waiver 11-W-00114/2 of the waiver in relation to the Childhood Lead Poisoning Primary Prevention Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T below. All expenditures on these contracts are related to mitigating the impact of lead-based paint on the target population. DOH costs for administering the program, including the program reviews, are not included in the DSHP claims. No Federal Financial Participation (FFP) is being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: Healthy Neighborhood Program (HNP)

Program Code: SFS Code 29893

Funding Sources: General Fund

$1,872,800* - State Appropriation

Chapter 53 of the Laws of 2015; page 457, lines 4-5

Process to identify the value of applicable offsets: The 10 percent undocumented immigrant offset is applied unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS.
Process to identify program net deficit payments -- N/A since there are no net deficit payments associated with this program group.

Brief Description:

The New York State Healthy Neighborhoods Program seeks to reduce the burden of housing related illnesses and injury through a holistic, healthy homes approach. The program provides assessments and interventions for asthma, tobacco cessation, indoor air quality, lead, fire safety, and other environmental health hazards in selected communities throughout New York State. The program targets housing in high-risk areas that are identified using house, health and socioeconomic indicators from census and surveillance data. The HNP uses a combination of neighborhood canvassing and referrals to reach residents in these high-risk areas. During a visit, Local Health Departments assess the home for environmental health and safety issues such as lead, fire safety, indoor air quality, and pest control. For problems or potential hazards identified during the visit, an outreach worker provides education, referrals and products to help residents correct or reduce housing hazards. Twenty-five percent of homes receive a revisit to provide additional service and assess outcomes. Grants are awarded to local health departments through a competitive process.

Eligible Population:

Persons living in targeted low-income geographic areas with a high risk for unmet environmental health services in all age groups.

Statewide Financial System (SFS) / Claiming Process:

The Healthy Neighborhoods Program (HNP) was initiated with a Request for Applications (RFA) which resulted in contracts being executed with local county health departments. A scoring mechanism was devised, and the highest scoring applications were awarded funding.

A State General Fund appropriation is legislated annually in the budget for the HNP. Claims are submitted quarterly by Local Health Departments along with a Budget Statement and Report of Expenditures form (BSROE), and quarterly report. The BSROE details the personnel salary (as a percentage of time performing grant-related activity from timesheets) and non-personal service expenses the county is vouchering for in that quarter. The quarterly report outlines the activities performed during that quarter which warrant the payment requested by the county. Once these are received, they are reviewed by program staff to ensure the work is performed as specified for that quarter and then voucher information is entered into the Statewide Financial System (SFS) for payment. Electronic quarterly payments are made via SFS. Voucher, Budget Statement and Report of Expenditures forms as well as quarterly reports are kept. Recently, a favorable audit report (July 23, 2014) was issued reflecting the results of voucher processing for the Bureau of Community Environmental Health and Food Protection programs. The local health departments are also audited by program staff on a biannual basis.

The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Department of Health enters vouchers into this system for payment. The Office of State Comptroller (OSC) reviews, approves and then processes such payments. OSC´s process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

Step 1 The program administrator enters coding into SFS to set up a purchase order (PO). This PO is given a unique ID that vouchers can be input into SFS to spend against. This program is coded to Department 3450000, Program 29893, Fund 10000, Account 60303 and Bud Ref 2015-16 (Bud Ref to be update annually).

Step 3 Once ready for processing, the claims are entered into SFS and coded to the unique PO number.

Step 4 The claims are then reviewed by the fiscal officer who approves the claim.

Step 6 Once paid, vouchers are scanned into the Bureau of Account Management´s File Net system and given a barcode to match SFS transactions.

The Department of Health uses the SFS to process contracts and vouchers for review, approval and payment by the State Comptroller. Multiple coding options are used, including department, program, account, budget reference and agency-described chart field 1 to easily identify where charges belong. When the State transitioned into SFS, DOH established unique program codes within SFS for all its programs in order to track disbursements. The HNP program is uniquely defined in SFS with the following Chart of Accounts coding:

Department Program Fund Account Bud Ref HNP 3450000 29893 10000 60303 2015-16

To determine budgeted amounts, program staff awards requested funding amounts to counties based on submitted work plan activities.

No advances of funding are processed for this program since all recipients of program funding are NYS governmental entities and therefore not advance eligible.

Non-Matchable Expenditure List:

Relevant staff from the Center for Environmental Health and the Fiscal Management Group reviewed the attached list of non-matchable program expenditures included in the Special Terms & Conditions of the waiver in relation to the Healthy Neighborhoods Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T below. All expenditures on these contracts provide assessments and interventions for asthma, tobacco cessation, indoor air quality, lead, fire safety, and other environmental health hazards in selected communities throughout New York State. DOH costs for administering the program, including program reviews and audits, are not included in the DSHP claims. Federal Financial Participation (FFP) is not being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

*Match for HUD Healthy Homes Grant will be deducted from DSHP reporting - $257,000

State Agency: Department of Health

Program: Cancer Services Program

Program Codes: SFS Code 26926

Funding Sources: General Fund

$25,281,000 - State Appropriation

Chapter 53 of the Laws of 2014; page 555, lines 17-18

The Cancer Services Program initiative is funded from this appropriation as follows:

Infrastructure: $8,915,544

Clinical Reimbursement: $5,104,144

Cancer Services Program appropriations are used as current Partnership Plan match and also to match a Centers for Disease Control and Prevention (CDC) cooperative agreement (grant). The required 1:3 match for the current grant period (6/30/14-6/29/15 is $2,865,660 (this includes the match on the recently approved carryforward from the previous grant year). The match is documented using paid claims for payment from CSP infrastructure contracts.

Expenditures not used to match the CDC grant will be available for the MRT waiver when they are no longer used as current Partnership Plan match (i.e. after 12/31/14).

Process to identify program net deficit payments -- N/A since there are no net deficit payments associated with this program group.

Brief Description:

The goal of the NYSDOH Cancer Services Program is to increase the proportion of women and men in New York State who are up-to-date on recommended preventive cancer screenings. This is accomplished through:

Increasing public and health care provider awareness about the importance of guideline- concordant cancer screening;
Assisting underserved populations to access and navigate available cancer screening, diagnostic and treatment services through local service region contracts;
Integrating guideline-concordant cancer screening into the care received by men and women throughout NYS; and implementing evidence-based policy, systems and environmental change strategies to promote cancer screening.

The NYSDOH Cancer Services Program (CSP) oversees the delivery of comprehensive breast, cervical and colorectal cancer screening and diagnostic services to eligible uninsured and underinsured individuals in New York State through contracts with community-based organizations known as CSP contractors. Contractors develop relationships with regional providers (e.g., hospitals, clinics, health care providers) and community-based organizations to conduct outreach to priority populations, provide screening, diagnostic and case management services, public education, data management and quality assurance, as well as other activities outlined later in this document. The contractor and its partners also assist individuals diagnosed with breast, cervical, colorectal or prostate cancer in obtaining prompt, comprehensive treatment through the New York State Medicaid Cancer Treatment Program (MCTP), if eligible. Eligible individuals may receive full Medicaid coverage for the duration of their cancer treatment. NYSDOH does not support routine population-based screening for prostate cancer. However, men screened and/or diagnosed with prostate cancer through participating providers are eligible for treatment coverage through the MCTP.

Required Contractor Infrastructure Activities:

Program Management and Leadership
The lead organization (contractor) has overall responsibility for all contract activities and is the primary contact for the Department/HRI. They coordinate and administer the program to ensure that all required activities are implemented and that contractual obligations are met in a timely manner. The lead organization also ensures that any barriers to implementation of the required activities are promptly addressed to reduce potential effects on program performance.
Partnering, Coordination and Collaboration
The lead organization builds and maintains collaborative relationships with health, human service, education and other community organizations to provide and promote utilization of cancer screening services at the population level and among the eligible populations throughout the proposed service region.
Public Education, Targeted Outreach and In-Reach
The lead organization engages partners to implement evidence-based or evidence-informed strategies to promote the program, build public demand for cancer screening services, and identify eligible clients in priority populations, throughout the service region. In addition, the lead organization ensures coordination and implementation of client-oriented screening interventions and strategies as outlined in the Centers for Disease Control and Prevention Guide to Community Preventive Services.
Provision of Health Services: Screening, Diagnostic and Case Management Activities
The lead organization develops a network of medical care providers throughout the service region to provide eligible men and women with comprehensive, guideline-concordant breast, cervical and colorectal cancer screening and diagnostic services, and, when necessary, ensure access to treatment services.
Data Management
Data management is integral to the monitoring and evaluation of the program. The lead organization oversees the collection of all data required by the Department.
Fiscal Management
The lead organization will be responsible for all fiscal management activities.

Eligible Population:

The Cancer Services Program (CSP), through its contractors, provides patient education and targeted outreach to connect uninsured New Yorkers to cancer screening services. Case management services are provided to all individuals who need follow-up as a result of screening. In addition, population-based public health interventions implemented by the CSP are non- medical services. General program activities have the potential to reach all NYS residents. Screening services are available to women and men living at or below 250% of the Federal Poverty Guideline; and uninsured or underinsured. Average risk women 40+ years old; men 50- 64 years old; or, if not within these age ranges, individuals deemed as high risk for breast or colorectal cancer, or women with clinically significant findings for breast cancer. General program activities serve all New York State residents (19.5 million) and approximately 50,000 men and women receive CSP screening services yearly.

Statewide Financial System (SFS)/Claiming Process:

The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Department of Health enters vouchers into this system for payment. The Office of State Comptroller (OSC) reviews, approves and then processes such payments. The OSC process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

The Department of Health uses the SFS to process contracts and vouchers for the review, approval and payment by the State Comptroller. When the State transitioned into SFS, DOH established unique Chart of Account (COA) program codes within SFS for all its programs in order to track disbursements. The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

The Cancer Services program was developed from an RFA competitive procurement process. Funding for the program is provided by an annual State appropriation. Expenditure plans are submitted by program for planned expenditures of budget appropriations. These plans are approved through Division/CCH and are submitted to FMG for approval transmittal to DOB.

Cancer Services Program - Infrastructure:

State SFS Coding

4/1/14 - 3/31/15

Fundline	1
Department	3450266
Program	28471
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Cancer Services Program - Clinical Services:

State SFS Coding 4/1/14 - 3/31/15

Fundline	1
Department	3450266
Program	28470
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Cancer Services Program - Clinical Services:

State SFS Coding

4/1/14 - 3/31/15

Fundline	2
Department	3450266
Program	29497
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Services provided by the provider network of each contractor submit socio-demographic information about eligible clients along with information about cancer screening services provided to them. This data is entered into a web-based data system (built and maintained under a contract with INDUS Consultancy Inc.). At the end of each month, all data entered for that month are run through edit checks to assure services are being provided to eligible clients and that providers are licensed to perform the services indicated. Contractors download a Monthly Billing Report that details the services documented to eligible clients during the last month along with the allowable reimbursement for each service. Reimbursement rates are set annually by the DOH program at a value less than or equal to the Regional Medicare rate (as required by the CDC cooperative agreement) and no less than the NYS Medicaid rate. DOH program staff review the services provided to assure they are in line with expected patterns of care.

Contractors are required to submit monthly, certified reimbursement claims related to both their infrastructure and clinical contract with supporting documentation for key expenditures according to unit policies within 30 days after the end of each month. After being reviewed by the contract/regional manager, the claim is entered into SFS by a Fiscal Officer. The Division´s Fiscal Unit subsequently reviews and approves the claim for payment. An OSC Audit will then approve the claim in SFS and reimburse the contractor via Automatic Clearing House (ACH) or by check within 30 days of receipt of the claim, barring unforeseen delays. The Fiscal Unit retains copies of the claims, the budget statement report of expenditures, and all correspondence required for the approval process. Correspondence may include copies of paid invoices, payroll registers, sub-contract agreements, as well as correspondence between the contractor and DOH contract manager that are related to deficiencies in claims or back-up documentation.

Periodic on-site audits are completed every 18-24 months by the assigned contract/regional manager for the Department of Health. In addition, a fiscal in-house audit is made through a voucher trace process which covers a sample period selected by program. This audit requires the contractor to provide full supporting documentation for the respective claim period. These may include invoices, proof of payment for all NPS expenditures, and copies of payroll records and related fringe and payroll tax records.

Step 1 Upon receipt of the reimbursement claims, support staff that are responsible for mail distribution date stamp the claim with a receipt date and deliver it to the Regional Contract Manager responsible for that contractor.

Step 2 The claims are then reviewed by the Regional Contract Manager with a focus on cost allowability allocability and reasonableness as well as other grant program requirements.

Step 3 Once approved, the claims are sent to the Division of Chronic Disease Prevention Fiscal Management Unit (FMU) for processing. The claims are reviewed by FMU Fiscal Officers for cost allowabiltiy and allocability, coded with the appropriate COA coding (noted above) and keyed into SFS. The claims are advanced to the FMU management team for final review and approval in SFS for payment.

Step 4 FMU approved claims are advanced electronically in SFS to the Office of the State Comptroller for final audit and payment authorization. Through this process vouchers are paid by the OSC through the SFS system within 30 days of receipt.

Step 5 Payment is disbursed through the SFS system. For all contractors, the FMU scans the Claim for Payment with expenditure detail into the DOH Filenet system.

Step 6 The state submits a claim to CMS for FFP based on the total computable expenditures incurred by the State in making the eligible payment to DSHP provider. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F of Waiver 11-W-00114/2.

Step 7 The DOH Fiscal Management Group staff attests expenditures are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2. At the discretion of New York State, Not-for-Profit (NFP) organizations receiving grants under this program are eligible for contract written directive advances of up to 25% of the value of each of the budget periods (except the first year) of the multi-year contract. If a written directive advance request from a NFP is granted, upon execution of the contract, it is paid through the SFS by the processing of an advance claim voucher submitted by the NFP. As noted above, multi- year contracts have individual annual budgets, so the level of the advance in relation to the total contract value may vary. The advance is recouped by deducting 30 % of the gross value of each monthly claim. This continues until the entire advance amount is recouped. The recoupment amount is netted out of the full claim value for each month and the balance of the payment is processed through the SFS. For the current SFY 2014-15, there were 12 contractors that accepted contract advances for this program.

Non-Match Able Expenditure List:

Relevant staff from the Division of Chronic Disease Prevention (CDP) and the Fiscal Management Group (FMG) reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Cancer Services Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. There is a maximum of 25% of undocumented individuals receiving this program´s services, but FFP will not be claimed. Since no penalty is assessed for services provided to undocumented individuals, Medicaid Financial Management (MFM) staff will take a discount of 25% before such claims are submitted for federal funding, unless a more detailed accounting of actual costs can be made for each claim. No Federal Financial Participation (FFP) is being claimed for the following items:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines- No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - The program initiative´s community-based contractors serve all residents of New York State. There are no FFP matching expenditures, according to CDP. If there were, CDP staff would identify the vouchers to BAM, who produce the claims to CMS.
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - There is a Center for Disease Control cooperative agreement that requires an MOE of $1.0 million. There are no FFP matching expenditures according to CDP. If there were, CDP staff would identify the vouchers to BAM, who produce the claims to CMS.
Administrative costs - No. FFP is not claimed for administrative costs.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: Obesity and Diabetes Prevention Program

Program Codes: SFS Code 26925

Funding Sources: General Fund

State Fiscal Year 14/15: $6,803,300 - State Appropriation (with 5 program components)

Chapter 53 of the Laws of 2014; page 550, lines 44-45

State Fiscal Year 15/16: $7,463,300 - State Appropriation (with 10 program components)

14/15 State appropriations will be used for a Centers for Disease Control and Prevention (CDC) cooperative agreement (grant) match in the amount of $546,969. This is a one-time match requirement.

State expenditures are currently used for the Partnership Plan Demo and will be available for the MRT Waiver on 1/1/15.

Federal grant funds that support similar types of obesity and diabetes prevention programs are completely separate from state-appropriated funds. The Federal grant funds are administered by Health Research, Inc. a not-for-profit corporation. All fiscal and administrative processes are distinct, there is no risk of co-mingling of funds.

Process to identify program net deficit payments -- N/A since there are no net deficit payments associated with this program group.

Brief Description:

The Obesity and Diabetes Prevention Programs are designed to raise public and professional awareness of the twin epidemics of obesity and diabetes, reduce the prevalence of these diseases and their risk factors, and develop and implement programs to prevent or reduce these diseases and their complications.

The Obesity Prevention Program began in 2005 as a statewide education and prevention program to encourage New Yorkers to eat healthfully and be physically active to attain and maintain a healthy weight. The program is designed to reduce the epidemic of obesity and overweight, and thereby reduce and prevent chronic diseases such as diabetes, heart disease, arthritis and some cancers. The Diabetes Prevention Program works to prevent diabetes; educate people with diabetes and their health care providers about management techniques proven to prevent diabetes and its complications; increase the number of individuals with diabetes who receive guideline concordant care; and improve self- care and medical care of people with and at risk for diabetes. Both programs build upon ongoing efforts to promote increased physical activity, healthy eating, and development of healthy lifestyles. They support policy, systems and environmental change in worksites, communities, schools, health insurance, and health systems to achieve program goals.

According to a recent study, it is estimated that New York State will spend approximately $136.3 billion from 2011 to 2020 on adult obesity-related health problems. This accounts for approximately 5.6% of statewide health´s number one killer - cardiovascular disease - spending with approximately 80% of this cost paid through publicly funded health care programs, such as Medicare and Medicaid (Bending the Health Care Cost Curve in New York State: Options for Saving Money and Improving Care, New York State Health Foundation (July 2010). In 2010, New York State Medicaid program expenditures totaled approximately $8.6 billion for the nearly 307,000 beneficiaries with diabetes.

The Department issues competitive funding announcements: requests for applications (RFA) which result in grant contracts, or requests for proposals (RFP) which result in deliverables-based cost contracts. The RFA/P delineates the scope of work that the Department is procuring for, including program objectives. Eligible applicants submit proposals to the Department. Applications are scored by trained and experienced Department staff based on the technical merits and costs proposed by the applicants. As a result of the competitive bid process, the Department awards contracts which are inclusive of a structured budget and work plan.

Obesity/Diabetes Initiatives:

This section lists the SFY 14/15 (5 components) and SFY 15/16 (10 components). SFY 2014/15 Funding for Obesity/Diabetes Initiatives covers 5 distinct programs: 1) Creating Healthy Places to Live, Work and Play, 2) Healthy Schools New York, 3) Statewide Coalition and Center for Obesity Prevention, Healthy Eating and Active Living, 4) Statewide Coalition and Center for Obesity Prevention, Healthy Eating and Active Living, and 5) Obesity Prevention in Pediatric Health Care Settings. A brief program summary is provided below. A brief description is provided below and more detailed information is available here

Creating Healthy Places to Live, Work and Play - Creating Healthy Places to Live, Work, and Play is a NYS initiative to promote healthy lifestyles and prevent obesity and type 2 diabetes. This initiative works to improve the health and physical activity status for New York communities. These twenty-two contractors work to prevent obesity and type 2 diabetes through the implementation of sustainable policy, systems and environmental change in communities and worksites.

General Fund - Aid to Localities

Obesity and Diabetes Programs

State SFS Coding

4/1/14 - 3/31/15

Department	3450264
Program	28459
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Healthy Schools New York:

These programs to improve healthful eating and increase opportunities for physical activity in high need schools through the implementation of sustainable school health policy and practice changes. Programs are presently provided through 19 contractors

General Fund - Aid to Localities Obesity and Diabetes Program

State SFS Coding

Department	3450264
Program	28458
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Statewide Coalition and Center for Obesity Prevention, Healthy Eating and Active Living:

The Statewide Coalition and Center is funded to implement evidence-based policy, systems and environmental changes that prevent obesity, increase opportunities for healthful eating and physical activity, and decrease opportunities for sedentary activities and consumption of now nutrition foods. Specifically, the contractor:

Convenes, supports and coordinates the activities of a statewide coalition for obesity prevention;
Conducts policy research to inform the work of the coalition, Division of Chronic Disease Prevention (DCDP) staff, and contractors and partners; and
Provides training and technical assistance to the coalition, DCDP staff, contractors and partners on economic development, transportation and land use planning.

General Fund - Aid to Localities Obesity and Diabetes Programs

State SFS Coding

4/1/14 - 3/31/15

Department	3450264
Program	29483
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Obesity Prevention in Pediatric Health Care Settings:

Initiatives to promote the early recognition of pediatric overweight and obesity, building on a previously successful model for health care systems change in the primary care setting. Contractors (9) recruit pediatric practices to participate in the initiative and implement health systems changes to ensure that care delivered to children and adolescents is concordant with the Expert Committee Recommendations on the Assessment, Prevention and Treatment of Child and Adolescent Overweight and Obesity.

General Fund - Aid to Localities Obesity and Diabetes Programs

State SFS Coding

4/1/14-3/31/15

Department	3450264
Program	28455
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Prevention Agenda Training:

An MOU provides the mechanism to support the provision of training and technical assistance (TA) to local public health stakeholders (e.g., Local Health Departments, hospitals) and their community-based partners. The training and TA builds on previously held in-person and web- based training on evidence-based interventions to prevent chronic disease and aligns with the Prevention Agenda 2013-2017 Preventing Chronic Disease Action Plan. The MOU is held by the School of Public Health at the State University of New York at Albany (a "sister" state agency; www.nyspreventschronicdisease.com).

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/14 - 3/31/15

Department	3450264
Program	28460
Fund	10000
Account	60301
Bud Reference	2014-15
Project ID	N/A
Activity	N/A
Chartfield 1	11850

SFY 2015/16 Funding for Obesity/Diabetes Initiatives covers 10 distinct programs: A brief program summary is provided below.

Component 1: Comprehensive School Health Policies for Tobacco, Physical Activity and Nutrition (Healthy Schools NY):

Eighteen Healthy Schools NY contractors are funded to work with school districts implement comprehensive, district level policies related to physical education, physical activity and nutrition. These policies support schools in creating healthier environments for both students and adults. Policy efforts focus on increasing the quantity and quality of Physical Education and physical activity and healthful eating opportunities in schools.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Department	3450264
Program (Trans)	28458
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 2: Creating Healthy Places to Live, Work and Play:

Twenty-two contractors in 22 counties promote healthy lifestyles to prevent obesity and type 2 diabetes by implementing evidence-based and sustainable policy, systems and environmental change strategies in communities and worksites that increase opportunities for physical activity and healthy food choices. Examples of Community projects include: establishing and promoting the use of grocery stores in high needs areas; establishing and promoting the use of farmers markets; increasing physical activity, reducing sugary beverages and screen time in child care settings; and, developing transportation policies to ensure streets are safe and accessible for all users.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Department	3450264
Program (Trans)	28459
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 3: Statewide Center and Coalition for Obesity Prevention:

A vendor is funded to maintain a public web site in support of the activities of a statewide coalition for obesity prevention.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 9/30/15

Department	3450264
Program (Trans)	29483
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 4: Statewide Center and Coalition for Obesity Prevention:

This is a new procurement to convene, support and coordinate the activities of a statewide coalition for obesity prevention and conduct policy research issues that pertain to physical activity, healthy eating and obesity prevention.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

1/1/16 - 3/31/16

Department	3450264
Program (Trans)	29483
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 5: Creating Healthy Schools and Communities:

This is a new procurement to fund a coordinated, multi-sector effort to increase demand for and access to healthy food and opportunities for physical activity, to reduce the risk of obesity in high-need communities and school districts.

General Fund - Aid to Localities Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Department
Program (Trans) REQUESTED
Fund
Account
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1

Component 6: Obesity Prevention in Pediatric Health Care Settings:

Nine contractors covering 36 counties provide training, technical assistance and quality monitoring to health care provider organizations and primary care practices to improve clinical systems used to assess and prevent pediatric obesity. Contractors work with providers through local learning collaboratives to promote implementation of guideline-concordant care including screening using body mass index (BMI), providing a consistent response to the obesity epidemic statewide and improving the quality of pediatric primary care.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Department	3450264
Program (Trans)	28455
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 7: Chronic Disease Prevention Training:

The Department will fund an MOU with the State University of New York´s School of Public Health to convene and lead local improvement teams to implement priority chronic disease strategies from the Prevention Agenda 2013-17. Deliverables for the School of Public Health are: collaborate with DOH to develop and deliver trainings; coordinate all aspects of training; support local agencies, hospitals and community partners in developing required plans addressing chronic disease prevention; provide tools and resources to support implementation; and monitor and report progress to DOH.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Department	3450264
Program (Trans)	28461
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 8 - Eat Well Play Hard - Day Care Homes:

Building off the success of the nationally recognized Eat Well Play Hard in Child Care Settings (EWPH-CCS) initiative, Eat Well Play Hard in Day Care Homes (EWPH-DCH) is an educational and environmental change program to help day care home providers improve the nutrition and physical activity practices in their day care homes and communicate positive messages about eating healthy food and being physically active to children in care and their families. Day care home providers are trained and mentored to improve the nutrition, mealtime, physical activity and screen time practices in 128-day care homes serving approximately 1,000 children per year. The Child and Adult Care Food Program (CACFP) in the Division of Nutrition administers the program.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Department	3450264
Program (Trans)	28456
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

Component 9: The Behavioral Risk Factor Surveillance System (BRFSS):

This annual data collection program administered within BCDER. Data is collected through random-digit dial telephone surveys supported by the Department in collaboration with the federal Centers for Disease Control. The BRFSS collects statewide information on health practices, risk behaviors and preventable conditions, including obesity and diabetes, for the Department and its partners to use in program planning and evaluation.

Component 10: Breastfeeding Quality Improvement in Hospitals Expansion

One contract with the National Initiative for Children´s Health Care Quality (NICHQ) is established to assist NYSDOH and NYS maternity hospitals in improving breastfeeding exclusivity and duration rates and reducing formula supplementation rates by strengthening breastfeeding policies, systems and practices using proven quality improvement methods in 78 NYS maternity hospitals over the contract period.

General Fund - Aid to Localities

Obesity and Diabetes

State SFS Coding

4/1/15 - 3/31/16

Obesity and Diabetes State SFS Coding
4/1/15 - 3/31/16
Department	3450264
Program (Trans)	28460
Fund	10000
Account	60301
Bud Reference	2015-16
Project ID	N/A
Activity	N/A
Chartfield 1	11850

The Obesity/Diabetes programs and their initiatives serve all New York State residents regardless of age. Under this program, approximately 3.5 million clients are served annually. When this document was developed, the State and Federal parties agreed that approximately 10% of the population that is undocumented may benefit from the policy, systems and environmental change actions that are implemented statewide.

Statewide Financial System (SFS)/Claiming Process:

The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Department of Health enters vouchers into this system for payment. The Office of State Comptroller (OSC) reviews, approves and then processes such payments. The OSC process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

The Department of Health uses the SFS to process contracts and vouchers for the review, approval and payment by the State Comptroller. When the State transitioned into SFS, DOH established unique program codes within SFS for all its programs in order to track disbursements. The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

The Obesities and Diabetes Prevention program was developed from 3 types of processes: RFA competitive procurement processes; a competitive contract procurement; and an MOU with the School of Public Health. Funding for the program is provided by a State appropriation. Expenditure plans are submitted by program for planned expenditure of budget appropriations. These plans are approved through Division/CCH and are submitted to FMG for approval and transmittal to DOB.

Contractors are required to submit monthly, certified reimbursement claims with supporting documentation for key expenditures according to unit policies within 30 days after the end of each month. The program manager uses a performance management system to cross walk claims with monthly contract activity reports. After being reviewed by the program manager, the claim is entered into SFS by a Fiscal Officer. The Division´s Fiscal Unit subsequently reviews and approves the claim for payment. An OSC Audit will then approve the claim in SFS and reimburse the contractor via Automatic Clearing House (ACH) or by check within 30 days of receipt of the claim, barring unforeseen delays. The Fiscal Unit retains copies of the claims, the budget statement report of expenditures, and all correspondence required for the approval process. All contractors use a standard master budget template; categories of expense include: personal services (salary/fringe), non-personal services (contractual, travel, equipment, space/property/utilities, operating expenses or other). Correspondence may include copies of paid invoices, payroll registers, sub-contract agreements, as well as correspondence between the contractor and DOH contract manager that are related to deficiencies in claims or back-up documentation.

Step 1 Upon receipt of the reimbursement claims, support staff that are responsible for mail distribution date stamp the claim with a receipt date and deliver the claim to the Contract Manager.

Step 2 The claims are then reviewed by Contract Managers with a focus on cost allowability, allocability and reasonableness as well as other grant program requirements.

Step 3 Once approved, the claims are advanced to the Division of Chronic Disease Prevention Fiscal Management Unit (FMU) for processing. The claims are reviewed by FMU Fiscal Officers for cost allowability and allocability, coded with the appropriate COA coding (noted above) and keyed into SFS.

Step 4 The claims are advanced to the FMU management team for final review and approval in SFS for payment.

Step 5 FMU approved claims are advanced electronically in SFS to the Office of the State Comptroller for final audit and payment authorization. Through this process vouchers are paid by the OSC through the SFS system within 30 days of receipt.

Step 6 Payment is disbursed through the SFS system. For all contractors, the FMU scans the Claim for Payment with expenditure detail into the DOH Filenet system.

Step 8 The DOH Fiscal Management Group staff attests expenditures are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2.

At the discretion of New York State, Not-for-Profit (NFP) organizations receiving grants under this program are eligible for contract written directive advances of up to 25% of the value of each of the budget periods (except the first year) of the multi-year contract. If a written directive advance request from a NFP is granted, upon execution of the contract, it is paid through the SFS by the processing of an advance claim voucher submitted by the NFP. As noted above, multi- year contracts have individual annual budgets, so the level of the advance in relation to the total contract value may vary. The advance is recouped by deducting 30 % of the gross value of each monthly claim. This continues until the entire advance amount is recouped. The recoupment amount is netted out of the full claim value for each month and the balance of the payment is processed through the SFS. For the current SFY 2014-15, there were 15 contractors that accepted contract advances for this program.

Non-Match Able Expenditure List:

Relevant staff from the Division of Chronic Disease Prevention (CDP) and the Fiscal Management Group (FMG) reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Obesity and Diabetes Prevention Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. Undocumented individuals may receive the services of this program, but FFP will not be claimed. The 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual costs can be made for the claims. Financial Participation (FFP) is not being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines- No
School based programs for children - A number of initiatives under this funding support school-based interventions to reduce obesity in children. No matching Federal funds are being claimed, according to CDP.
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - The program initiative´s community-based contractors serve all residents of New York State. No matching Federal funds are being claimed, according to CDP. If there were, DCDP staff would identify the vouchers to BAM, who produce the claims to CMS.
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No. FFP is not claimed for administrative costs.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: Tuberculosis Control and Prevention

Program Codes: SFS Code 26839

Funding Sources: General Fund

$5,587,100 - State Appropriation

Chapter 53 of the Laws of 2014; page 551, lines 24-33

The appropriation will be used as Partnership Plan through the end of that waiver program. Disbursements for periods after that (1/1/15) will be used towards the MRT 1115 waiver.

Process to identify program net deficit payments--N/A since there are no net deficit payments associated with this program group.

Brief Description:

The Public Health Campaign funds support Tuberculosis (TB) contracts with twelve local health departments (including the New York City Department of Health) for maintenance of local public health infrastructure that provides direct patient care. These health departments are located in the city/counties with the highest TB morbidity in the State. Funds are awarded proportionally based on the average number of TB cases in county, above a threshold level.

The Department promotes early detection of active (TB) with timely diagnostic workups and ensures that all close contacts with infectious persons are evaluated for TB. Contracts with local health departments directly support staff and activities involved in providing case management for persons with active TB, including the use of directly observed therapy, and other TB-related services. The Department provides technical expertise, promotes targeted testing, and treatment of latent infection for persons at increased risk of disease.

Eligible Population:

All individuals of any age can be served. Approximately 14,000 clients are served annually and approximately 50% of adults fall below 200% of the Federal Poverty Level.

Statewide Financial System (SFS)/Claiming Process:

The Statewide Financial System (SFS) is the State of New York´s accounting system whereby the Department of Health enters vouchers into this system for payment. The Office of State Comptroller (OSC) reviews, approves and then processes such payments. The OSC process includes budget checks against appropriations / segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

Step 1 Upon receipt of the reimbursement claims, support staff that are responsible for mail distribution enter basic identifying information for each claim into the claim for payment form.

Step 2 The claims are then reviewed by two Professional Staff with a focus on cost allowability, allocability and reasonableness as well as other grant program requirements.

Step 3 Once ready for processing, the claims are coded to Department 3450252, Program 27669, Fund 10000 and account 60301 as well as the budget reference which is currently 2014-15.

Step 4 The claims are then reviewed by a higher-level supervisor who approves the claim.

Step 5 Upon approval, more detailed information is entered into the Grants system which contains many automated error checks to ensure that nothing critical got past the staff review.

Step 6 On a daily basis the transactions in the AP Module that are approved for payment are downloaded into a bulk load transaction file for encrypted electronic transfer to the SFS system where they are again audited by the Office of the State Comptroller (OSC). Through this process vouchers are paid by the OSC through the SFS system within 30 days of receipt.

Step 7 These payments are also disbursed through the SFS system. For all providers, the TB Program keeps the Claim for Payment with expenditure detail on file for six years after the annual grant is closed out.

Step 8 The state submits a claim to CMS for FFP based on the total computable expenditure incurred by the State in making the eligible payment to DSHP provider. The expenditures claimed must be in accordance with Special Terms and Conditions paragraph 15 sections A-F. of Waiver 11-W-00114/2.

Step 9 The State attests expenditures used are correct and verifiable as DSHP allowable. The program utilizes standard DOH approved DSHP Claiming Protocol referenced in Special Terms and Conditions paragraph 15 sections B of Waiver 11-W-00114/2.

For this particular program, the SFS Chart of Accounts values are:

Business Unit:	DOH01
Department:	3450252
Program:	27669
Fund:	10000
Account:	60301
Budget Reference:	Currently 2014-15
Chartfield 1:	11850

The Tuberculosis Control and Prevention program was developed from a non-competitive procurement process that administers contracts with local healthcare departments. Funding for the program is provided by a State appropriation. The local health departments must have a four- year average of at least five cases to be eligible for funding. The cost per capita is established by taking the available funding and dividing by the cumulative average number of patients. Awards are determined by multiplying the cost per capita by the average number of patients for each local health department.

The local departments may issue quarterly claims for reimbursement by New York State for services performed relating to the program within 30 days after the end of the quarter. These claims for reimbursement are reviewed and entered into SFS by State fiscal staff. The program´s director subsequently reviews each claim, recommends approval, and forwards to the Division´s Fiscal Unit for payment via Automatic Clearing House (ACH). The TB Program retains copies of the claims and all correspondence required for the approval process. Correspondence includes a quarterly report detailing performance towards contract objectives, payroll records for personal service expenditures, and appropriate records for travel and office supply expenditures.

For this particular protocol, the organizations that are in contract with the program are governmental, therefore New York State does not issue an advance on any contracts.

Non-Match Able Expenditure List for the Waiver:

Relevant staff from the Bureau of Tuberculosis Control (BTC) and the Fiscal Management Group (MFG) reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the TB Control and Prevention Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. Undocumented individuals may receive the services of this program, but FFP will not be claimed. The 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual costs can be made for the claims. Financial Participation (FFP) is not being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines- No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - Treatment must be provided to anyone with TB infection or disease, regardless of status. No individual claiming is involved for any patient. No federal funds are involved with the contracts, according to BTC. Article 22 directs the state appropriation to provide supplemental funds to Health Department entities to treat tuberculosis.
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - Treatment must be provided to anyone with TB infection or disease, regardless of status. No individual claiming is involved for any patient. No federal funds are involved with the contracts, according to BTC. Article 22 directs the state appropriation to provide supplemental funds to Health Department entities to treat tuberculosis.
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: Tuberculosis Directly Observed Therapy

Program Codes: SFS Code 29912

Funding Sources: General Fund

$565,600 - State Appropriation

Chapter 53 of the Laws of 2014; page 551, lines 51-52

Appropriation will be used in Partnership Plan through the end of that waiver program. Disbursements for periods after that (1/1/15) will be available for the MRT 1115 waiver.

Process to identify program net deficit payments--N/A since there are no net deficit payments associated with this program group.

Brief Description:

National Tuberculosis (TB) treatment guidelines strongly recommend using a patient-centered case management approach, including directly observed therapy (DOT), when treating persons with active TB disease. DOT is especially critical for patients with drug-resistant TB, HIV- infected patients, and those on intermittent treatment regimens (i.e., 2 or 3 times weekly). DOT decreases the chances of treatment failure and relapse and is highly effective at preventing the spread of TB and the development of multi-drug resistant (MDR) strains of TB.

DOT contracts are developed with New York City hospitals to provide salary and expense support for qualified health workers who:

Monitor patients swallowing their TB drugs
Check for side effects
Document the visit
Answer any questions the patient may have
Develop a reminder system to routinely remind patients to keep an appointment and take action when a patient fails to keep an appointment
Provide education and counseling to promote adherence to the treatment of active TB

Eligible Population:

Individuals of all ages are covered. Of the uninsured, underinsured, and indigent TB patients in New York City, approximately 90% of adults are under the 200% Federal Poverty Level. Approximately 180 clients are served annually.

Statewide Financial System (SFS)/Claiming Process:

The Department of Health uses the SFS to process contracts and vouchers for the review, approval and payment by the State Comptroller. When the State transitioned into SFS, DOH established unique codes within SFS for all its programs in order to track disbursements. The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

Step 1 Upon receipt of the reimbursement claims, support staff that are responsible for mail distribution enter basic identifying information for each claim into the claim for payment form.

Step 2 The claims are then reviewed by two Professional Staff with a focus on cost allowability, allocability and reasonableness as well as other grant program requirements.

Step 3 Once ready for processing, the claims are coded to Department 3450252, Program 27670, Fund 10000 and account 60301 as well as the budget reference which is currently 2014-15.

Step 4 The claims are then reviewed by a higher-level supervisor who approves the claim.

Step 5 Upon approval, more detailed information is entered into the Grants system which contains many automated error checks to ensure that nothing critical got past the staff review.

Step 7 These paymentsare also disbursed through the SFS system. For all providers, the TB Program keeps the Claim for Payment with expenditure detail on file for six years after the annual grant is closed out.

For this program the SFS Chart of Accounts coding is as follows:

Business Unit:	DOH01
Department:	3450252
Program:	27670
Fund:	10000
Account:	60301
Budget Reference:	Currently 2014-15
Chartfield 1:	11850

The Tuberculosis Directly Observed Therapy program was developed from a non-competitive procurement process that administers contracts with hospitals in high morbidity areas. Funding for the program is provided by the Public Health Campaign and Tuberculosis Hospital Based Grant (HBG) fund. In order to project the program´s budget amount, the cost per capita is established by taking the available funding and dividing by the cumulative average number of patients. Awards are determined by multiplying the cost per capita by the average number of patients for each hospital.

The hospitals may issue quarterly claims for reimbursement of salary and related expenses by New York State for services performed relating to the program within 30 days after the end of the quarter. These claims for reimbursement are reviewed and entered into SFS by State fiscal staff. The program´s director subsequently reviews each claim, recommends approval, and forwards it to the Division´s Fiscal Unit for payment via Automatic Clearing House (ACH). The TB Program retains copies of the claims and all of its correspondence required for the approval process. Correspondence includes a quarterly report identifying the number of patients served, payroll records for personal service expenditures, and appropriate records for travel and office supply expenditures.

For this particular protocol, the organizations that are in contract with the program are governmental, therefore New York State does not issue an advance on any contract.

Non-Match Able Expenditure List for the Waiver:

Relevant staff from the Bureau of Tuberculosis Control (BTC) and the Fiscal Management Group (MFG) reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the TB Directly Observed Therapy Program. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T. DOH costs for administering the program, including the program reviews and audits noted above, are not included in the DSHP claims. Undocumented individuals may receive the services of this program, but FFP will not be claimed. The 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual costs can be made for the claims. Financial Participation (FFP) is not being claimed for the following:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - No
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the Unites States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No. New York State costs for administering this grant program are not claimed.
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: Newborn Screening Program

Program Codes: SFS Code 32489

Funding Sources: State Miscellaneous Special Revenue Fund

$11,900,000 - State Appropriation

Chapter 50 of the Laws of 2014; page 306, lines 1-20

Process to identify the value of applicable offsets: The 10 percent undocumented immigrant offset is applied unless a more detailed accounting of actual costs for those individuals is provided and acceptable to CMS.
Process to identify program net deficit payments -N/A- since there are no net deficit payments associated with this program group.

Brief Description:

The Newborn Screening Program performs more than 12 million screens annually for 46 congenital disorders including exposure to the human immunodeficiency virus (HIV). The tests are conducted on approximately one-quarter of a million babies born each year in New York State.

Although most of the disorders are rare, they are usually serious. Some may be life threatening; others may slow down a baby´s physical development, cause intellectual disability or other problems if left undetected and untreated. None of the disorders can be cured, however, serious side effects can be lessened, and often completely prevented, if a special diet or other type of medical intervention is started early, before the infant displays any signs of the disease.

Screening for all of these conditions is performed on a sample of blood obtained by pricking the infant´s heel. The sample is usually taken within the first few days of life. The blood is collected and then dried on a special paper which is sent to the Newborn Screening Program in Albany. Results are reported to birthing facilities, physicians, and, when applicable, specialty care centers.

Eligible Population:

All infants, greater than 24 hours old, born in New York State are mandated by law to have this testing unless parental objection is documented for religious or other reasons.

Statewide Financial System (SFS)/Claiming Process:

The New York State Newborn Screening Program is mandated by legislation in NYS Public Health Law § 2500-a. Funding for the program is appropriated, by name, annually in the NYS Legislative Budget within the NYS Department of Financial Services. Funding is then sub allocated from the Department of Financial Services to the NYS Department of Health through the NYS Statewide Financial System (SFS). This sub allocation of funding is routed for approval through the Department of Financial Services, Department of Health, NYS Division of Budget, and the NYS Office of the State Comptroller. The funding for personal and non-personal services support the NYS employees that administer and perform the program´s requirements.

The Statewide Financial System is the State of New York´s accounting system whereby the Department of Health enters purchase requisitions for supplies, equipment and services into this system for procurement and payment. The Office of State Comptroller (OSC) reviews, approves and then processes payments for these items. The OSC process includes budget checks against appropriations/segregations to ensure appropriate authority for the expenditures has been provided by the NYS Division of the Budget through a journal process that is built into SFS.

Step 1 Upon receipt of the purchase requisition, support staff that are responsible for purchase requisition entry initiate the requisition in SFS.

Step 2 The requisitions are then reviewed and approved by staff that have expertise in procurement rules and laws with a focus on funding availability, completion of necessary documentation, and reasonableness as well as other fiscal requirements.

Step 3 Once ready for processing, the requisitions are coded to Department 3450319, Program 32489 and Fund 21994 as well as the budget reference which is currently 2015-16.

Step 4 The requisitions are then reviewed by a Budget Analyst in the Department who approves the requisition.

Step 5 A buyer at the Office of General Services Business Services Center approves the requisition which causes the dispatch of a purchase order to the vendor.

Step 6 The order and an invoice from the vendor are received and the invoice is approved in the FileNet system.

Step 7 On a daily basis the data in the FileNet system that is approved for payment is downloaded into a bulkload transaction file for encrypted electronic transfer to the SFS system where they are again audited by the Office of the State Comptroller (OSC). Through this process invoices are paid by the OSC through the SFS system within 30 days of receipt.

Step 8 These paymentsare also disbursed through the SFS system. The Department of Health keeps the purchase documentation on file for seven years.

Step 8(a) The Office of the State Comptroller processes payroll payments through their system called PayServe. Employees are paid 1/26 of their salary every two weeks, unless a payroll adjustment is made.

Step 10 The State attests expenditures used are correct and verifiable as DSHP allowable.

The Department of Health uses the SFS to process purchase requisitions for the review, approval and payment by the State Comptroller. When the State transitioned into SFS, DOH established unique codes within SFS for all its programs in order to track disbursements. The Newborn Screening Program is uniquely defined in SFS with the following Chart of Accounts coding:

Department Program Fund Budget Ref 3450319 32489 21994 2015-16 The SFS system budget tracks the transaction to ensure sufficient funds are available within the segregation/appropriation authority established through an approval process with the NYS Division of the Budget and the Office of State Comptroller. Such a mechanism also serves as a budget tool to track expenditures and provide a history of such expenditures by program that can be used for audit purposes.

Funding for this program is not used for advancements to supported entities.

Non-Match Able Expenditure List for the Waiver:

Pertinent staff from the Center for Community Health and the Fiscal Management Group reviewed the attached list of non-match able program expenditures included in the Special Terms & Conditions of the waiver in relation to the Newborn Screening Program stating that no Federal Financial Participation is being claimed for the following (below) in accordance with Special Terms and Conditions paragraph 15, iii sections A-F of Waiver 11-W-00114/2. For this particular program, it was determined that no claimed expenditures are made for the items listed in A through T below. All expenditures for Newborn Screening are related to performing more than 12 million screens annually for 46 congenital disorders. For item J, the 10% undocumented immigrant offset will be applied unless a more detailed accounting of actual costs can be made for the claims. For this program, it is anticipated that the percentage will be very low due to the services being provided are for newborns, most of which will be born in the U.S... Federal Financial Participation (FFP) is not being claimed for the following items:

Grant funding to test new models of care - No
Construction costs (bricks and mortar) - No
Room and board expenditures - No
Animal shelters and vaccines - No
School based programs for children - No
Unspecified projects - No
Debt relief and restructuring - No
Costs to close facilities - No
HIT/HIE expenditures - No
Services provided to undocumented individuals - Yes
Sheltered workshops - No
Research expenditures - No
Rent and utility subsidies normally funded by the United States Department of Housing and Urban Development - No
Prisons, correctional facilities, and services provided to individuals who are civilly committed and unable to leave - No
Revolving capital fund - No
Expenditures made to meet a maintenance of effort requirement for any federal grant program - No
Administrative costs - No
Cost of services for which payment was made by Medicaid or CHIP (including from managed care plans) - No
Cost of services for which payment was made by Medicare or Medicare Advantage - No
Funds from other federal grants - No

State Agency: Department of Health

Program: General Public Health Work (GPHW)

Program Codes: SFS Code 26815

Funding Sources: General Fund

$192,500,000 - State Appropriation

Chapter 53 Laws of 2014; pg. 548, ln. 12-49 & pg. 549, ln. 1-22

Please Note: Claiming protocol still under development.

|top of section| |top of page|

ATTACHMENT M

Final Evaluation Design and Final Evaluation Plan

Section	Page
Section A: Background/Introduction	1
A.1 Delivery System Reform Incentive Payment (DSRIP) Program	1
A.2 DSRIP Evaluation	1
A.3 DSRIP Special Terms and Conditions	1
A.4 Independent Evaluator (IE) Performance Standards/Expectations	1
A.5 Measures and Available Data	2
A.6 Study Populations and Sample Sizes	4
Section B: Time Series Design	5
B.1 Hypotheses	5
B.2 Research Questions	7
B.3 Determination of Cost Effectiveness	10
Section C: Qualitative Analysis	11
C.1 PPS Administrative Key Informant Interviews	14
C.2 Focus Groups with Project Associated Providers	18
C.3 Survey for Patients	19
C.4 Electronic Survey of Project-Associated Providers	20
C.5 Other Data Collection	22
C.6 Implementation/Process Analysis Summary	22
Section D: Comparative Analysis	23
D.1 Measures	28
D.2 Data	30
D.3 Clustering to Create PPS Comparison Groups	30
D.4 Difference In Difference (DID) Analysis	31
D.5 Patient Level Comparisons	31
D.6 Analytic Methods	32
D.7 Implementation/Process Evaluation	33
D.8 Triangulation of Data Analyses	34
D.9 Data Collection Plan	35
D.10 Anticipated Challenges and Mitigation Strategies	35
Section E: Detailed Table for Independent Evaluation of DSRIP	36
Section F: Timeline of Evaluation Activities	58
Section G: Reports/Meetings	60
Section H: Staffing Requirements	61
Section I: Limitation of the Design	67
Section J: Generalizability of Results	69
Section K: Analysis of DSRIP Dollar Allocation	69
Attachment 1: DSRIP Summary of Special Terms and Conditions	71

Section A. Background/Introduction

A.1 Delivery System Reform Incentive Payment Program:

The New York State Delivery System Reform Incentive Payment (DSRIP) Program is the main mechanism by which New York is implementing the Medicaid Redesign Team (MRT) Waiver Amendment. DSRIP´s purpose is to fundamentally restructure the health care delivery system by investing in the Medicaid program, with the primary goal of reducing avoidable hospital use by 25% over five years. Up to $6.42 billion is allocated to this program with payouts based upon achieving predefined results in system transformation, clinical management and population health. DSRIP provides incentives for Medicaid providers to create and sustain an integrated, high performing health care delivery system that can effectively and efficiently meet the needs of Medicaid beneficiaries and low income uninsured individuals in their local communities by achieving the MRT Triple Aim of improving care, improving health, and reducing costs.

Through DSRIP, the New York State Department of Health (NYSDOH) seeks to transform the health care safety net, reduce avoidable hospital use and make improvements in other health and public health measures at the system and state level, and ensure sustainability of delivery system transformation through leveraging managed care payment reform. DSRIP provides incentive payments to reward safety net providers that undertake projects designed to transform systems of care supporting Medicaid beneficiaries and low income, uninsured persons by addressing three key elements: safety net system transformation; appropriate infrastructure; and assuming responsibility for a defined population. Safety net providers who collectively participate in DSRIP are referred to as the 25 regional Performing Provider Systems (PPS).

A.2 DSRIP Evaluation:

An Independent Evaluator (IE), the Research Foundation of the State University of New York (SUNY), is implementing a multi-method, robust statewide evaluation of the DSRIP Program. The evaluation employs quantitative and qualitative methods in order to achieve a robust evaluation of DSRIP, and will achieve the following goals:

Assess program effectiveness on a statewide level with respect to the MRT Triple Aim;
Obtain information on the effectiveness of specific projects and strategies selected and the factors associated with program success; and
Obtain feedback from stakeholders including NYSDOH staff, PPS administrators and providers, and Medicaid beneficiaries served under DSRIP regarding the planning and implementation of the DSRIP Program, and on the health care service experience under DSRIP reforms. Evaluation results will be regularly reported to NYSDOH, the PPS and the Centers for Medicare and Medicaid Services (CMS).

A.3 DSRIP Special Terms and Conditions:

The evaluation will be consistent with the specifications outlined in the DSRIP Special Terms and Conditions (STC), Sections VIII.21 through VIII.33, as outlined in Attachment 1.

A.4 IE Performance Standards/Expectations:

The IE will address the following overarching Research Questions (RQs):

To what extent did PPS achieve health care system transformation?
Did health care quality improve as a result of clinical improvements in the treatment of selected diseases and conditions?
Did population health improve as a result of implementation of the DSRIP initiative?
Did utilization of behavioral health care services increase as a result of DSRIP?
Was avoidable hospital use reduced as a result of DSRIP?
Did DSRIP reduce health care costs?
What were the successes and challenges with respect to PPS planning, implementation, operation and plans for program sustainability from the perspectives of DSRIP planners, administrators and providers, and why were they successful and challenging?

A.5 Measures and Available Data:

A set of measures described in the "DSRIP Strategies Menu and Metrics" will be used to quantify facets of system transformation (Domain 2), quality of care through clinical improvements (Domain 3), and population health (Domain 4) using existing data sources, described below. Though the IE is not limited to the use of these measures in their evaluation, they may be used for purposes of the DSRIP evaluation in assessing statewide outcomes. The majority of these measures are well established with known measurement stewards (e.g., 3M, AHRQ) and are commonly used in health care quality improvement activities.

Regardless of outcome measures, the IE has access to a number of existing data sources that are maintained by, or are available to, NYSDOH. Given public health law and/or data use agreements that govern access to these data, the IE is aware that obtaining access may require substantial time and effort, which is a consideration of their evaluation timeline.

Medicaid Claims - This database contains billing records for health care services, including pharmacy, for approximately 5.7 million individuals enrolled in Medicaid in a given year. Also included are data on Medicaid enrollment status, diagnoses and provider associated with the billed services. The Medicaid claims database is updated on a monthly basis to include additional claims and modifications to existing claims. Given the claims processing, there is a six-month lag in the availability of complete and finalized Medicaid claims data, where data for a given year are considered final by June 30 of the following year.
Medicare Claims - For approximately 15% of Medicaid enrollees who are dually eligible for Medicare, Medicare claims will be used to ensure data completeness, as many of the services received by this group will be paid by Medicare and thus not appear in the Medicaid database. Medicare claims contains billing records for health care services, including pharmacy services, along with data on diagnoses and provider information. NYSDOH is working with an external entity specializing in the linking of Medicaid and Medicare claims data which will ensure timely access to Medicare claims through monthly data updates.
Statewide Planning and Research Cooperative System (SPARCS) - SPARCS is an all payer data reporting system established in 1979 as a result of cooperation between the health care industry and government. Initially created to collect information on discharges from hospitals, SPARCS currently collects patient level detail on patient characteristics, diagnoses and treatments, services, and charges for inpatient and outpatient (ambulatory surgery, emergency department, and outpatient services), hospital services and outpatient services from free-standing ambulatory surgery centers. SPARCS data may be used for medical or scientific research or statistical or epidemiological purposes. All entities seeking SPARCS identifiable or limited data must submit a request to SPARCS Operations using standard data request forms. Finalized SPARCS data for a given year are available in August of the following year.
Minimum Data Set (MDS) - MDS 2.0 and 3.0 data consist of federally mandated assessments collected at regular intervals on all nursing home residents in New York. Assessment data collected include diseases and conditions, nutritional status, resident physical and cognitive functioning (e.g., activities of daily living), medications received, and nursing home admission source and discharge disposition. These data have been shown to be adequately reliable and are widely used in research, and are available to NYSDOH under data use agreement with CMS. There is, approximately, a six-month lag in the availability of complete MDS data, where finalized data for a given year are available in June of the following year.
Consumer Assessment of Healthcare Providers and Systems (CAHPS®) - The Clinician & Group version of the CAHPS® survey will be administered by NYSDOH annually during the DSRIP demonstration period and will serve as the data source for selected outcome measures. The survey is administered by both mail and telephone, and assesses patients´ experiences with health care providers and office staff. This includes information on patient experience over the last 12-months including most recent visit to provider, ease of getting an appointment, and wait times while in the office. The survey includes standardized questionnaires for adults and children. The adult questionnaire can be used in both primary care and specialty care settings; the child questionnaire is designed for primary care settings, but could be adapted for specialty care. Users can also add supplemental items to customize their questionnaires. Surveys are administered in September of a given year, and are available for use in February of the following year. Given confidentiality agreements, only de-identified CAHPS data will be available for use.
New York Vital Statistics - Birth and death certificate data are maintained by New York, with New York City Department of Health and Mental Hygiene and NYSDOH comprising two separate jurisdictions in the reporting of birth and death records, which will likely necessitate separate data use agreements. NYSDOH has the responsibility for annual statewide reporting of vital statistics governed by the terms of a memorandum of understanding between the two jurisdictions. Birth records contain information such as maternal medical risk factors, prenatal care received, infant birth date, birth weight, and infant diseases/conditions including congenital malformations. Death certificate data include date of death, underlying and multiple cause of death, decedent demographics, county of residence, and county of death. While Vital Statistics data are received by NYSDOH on an ongoing basis, due to the process of updating and finalizing information from birth and death certificates (e.g., due to delayed receipt of lab results), data for a given year are not considered complete until the end of the following year.
Expanded Behavioral Risk Factor Surveillance System (eBRFSS) - eBRFSS augments the Centers for Disease Control and Prevention (CDC) BRFSS, which is conducted annually in New York. eBRFSS is a random-digit-dialed telephone survey among adults 18 years of age and older representative of the non-institutionalized civilian population with landline telephones or cell phones living in New York. The goal of eBRFSS is to collect county-specific data on preventive health practices, risk behaviors, injuries and preventable chronic and infectious diseases. Topics assessed by the eBRFSS include tobacco use, physical inactivity, diet, use of cancer screening services, and other factors linked to the leading causes of morbidity and mortality. The 2013-14 eBRFSS survey will be used as the baseline for DSRIP for measures derived from these data, and contains a question to identify Medicaid respondents. Repeat eBRFSS surveys to be used in support of the DSRIP evaluation will be conducted in 2016-17, and again in 2019-20.
New York HIV/AIDS Case Surveillance Registry - This registry contains information on new cases of HIV and AIDS, as well as persons living with HIV or AIDS. Data include date of diagnosis, HIV exposure category, county of residence at diagnosis, and whether or not diagnosis was made while individual was incarcerated.
Uniform Assessment System (UAS) - UAS contains assessment data on individuals receiving home or community-based long-term care (e.g., adult day health care, long-term home health care). Data include patient functional status, health status, cognitive functioning, and care preferences.
US Census - These data are publicly available from the United States (US) Census Bureau, and contain estimates of population size, and data on population characteristics. The latter include housing status, income, employment status, educational level, and health insurance coverage. US census data are gathered on an ongoing basis from a number of surveys including the Decennial Census, the American Community Survey, and the Economic Census.
Medical Record - Measures that are derived from medical records will be reported by PPS, or their participating providers.
Medicaid and Medicare Claims - These data, as well as SPARCS data, are available from the Office of Health Insurance Programs (OHIP) Data Mart. Implemented in 1998, the OHIP Data Mart serves as a data repository to support analytical reporting and applications for NYSDOH, the Office of the Medical Inspector General, and the Office of the Attorney General. It supports analytics and ad hoc user queries and supports a number of projects including Medicaid Claims History, the Medicaid Drug Rebate Application, and MRT Performance Analytics.

The IE will use a mixed methods strategy to meet the project objectives. This strategy offsets the weaknesses inherent in single method approaches and allows them to confirm, cross-validate, and corroborate the findings (Creswell, et al., 2003; Teddlie and Yu, 2007). See Sections B, C and D for more detailed explanations and strategy rationales.

A.6 Study Populations and Sample Sizes:

In November 2017, NYSDOH responded to CMS´s request to present a sampling strategy that explains how the 25 distinct PPS offer a distinct form of the intervention and the intensity of the intervention is expected to vary greatly across the state. CMS suggested the plan provide details on how the treatment and comparison groups will be identified in each data source and what identifiers will be used to match records across sources (e.g., SSN, Medicaid ID, name and address, provider numbers). CMS further stated that the plan should also document the ability to identify attributed or served patients and include a discussion of challenges in obtaining and integrating data and strategies for overcoming them. NYSDOH responded that the IE will receive information needed to identify members attributed to each PPS, and members not attributed to any PPS. The IE has further specified that they will create exact matches using available data on social security number, date of birth, and name across non-Medicaid datasets identified in this evaluation plan and will incorporate the whole population in their analysis. Other methods for matching will be utilized (e.g., propensity score matching) as needed based on availability and reliability of the measures. Their analysis will be carried out at different levels of the population (aggregated view of impact at the state level, then at the PPS and intra-PPS level, and finally at a more individual level). Any issues with sample selection and missing data will be resolved using statistical methods (e.g., Heckman correction, MICE imputations). After data cleaning, the research hypothesis may be tested by changing the control and treatment groups; the IE has further specified that they will explore other options for exact matching or propensity score matching and performing sensitivity analysis.

NYSDOH also responded to CMS´s suggestion that the evaluation include a plan for assessing the use of DSRIP funds for non-Medicaid populations (e.g., the uninsured to be included in the PPS implementing the "11th project.") CMS stated that if these funds represent a significant share of DSRIP funding, the design should include a sampling strategy documenting how this population is served and samples sufficient to estimate the impacts/benefits. Initially, NYSDOH stated that assessing the use of DSRIP funds for non-Medicaid populations such as the uninsured is outside of the scope of the evaluation (IE contract) and that a (contract) amendment would not be feasible with the timely submission of the Draft Interim Evaluation Report and Preliminary Summative Evaluation Report.

This has been revisited by the IE. SPARCS data would be able to be utilized to determine, in a limited capacity, utilization patterns of the uninsured in the inpatient and emergency department settings based upon the patient discharge dataset since those beneficiaries will not appear in the Medicaid claims. To examine whether the uninsured for the 14 PPS that are participating in the 11th Project are representative of the larger DSRIP population, the IE will examine hospital discharge records of the uninsured and compare the 14 PPS to the remainder.

Section B:

Time Series Design:

As stated in Attachment 1 regarding the STC, quantitative analysis to assess the effect of DSRIP on a statewide level will use a time series approach to the comparison of health outcomes following the implementation of DSRIP, to a time period prior to DSRIP´s implementation.

B.1 Using this approach, the IE will test the following hypotheses:

Health care service delivery will show greater integration.
Health care coordination will improve.
Primary care utilization will show a greater upward trend.
Expenditures for primary care services will increase.
Utilization of, and expenditures for, behavioral health care service will increase.
Expenditures for emergency department and inpatient services will decrease.
Primary care, behavioral health, and dental service utilization will increase among the uninsured, non-utilizing, and low-utilizing populations, while emergency department use will decrease.
Through clinical improvements implemented under DSRIP, health care quality in each of the following areas will increase:
1. Behavioral health
2. Cardiovascular health
3. Diabetes care
4. Asthma
5. HIV/AIDS
6. Perinatal care
7. Palliative care
8. Renal care
Population health measures will show improvements in the following four areas:
1. Mental health and substance abuse
2. Prevention of chronic diseases
3. Prevention of HIV and STDs
4. Health of women, infants, and children
Avoidable hospital use will be reduced.
Costs associated with hospital inpatient and ED services will show reductions or slowed growth.
Total cost of care will show reductions or slowed growth.

The IE will emphasize comparison of health care service delivery, health improvements, and cost to the Medicaid program at the state level over the study period. They will also do an inter-PPS analysis to identify components that posed success or challenges for implementation and outcomes by difference-in- differences (DID) analysis. Possible improvement in 12 broad categories of health care under four (4) domains is envisioned.

The IE will use the interrupted time series design with segmented regression on the following statewide times series to evaluate the statewide impact of DSRIP with quarterly observations ending in April 2019. Using the dataset starting from 2005 and defining 2014 as DSRIP Year 0, the IE will have 10 years of pre-DSRIP data to control for existing trends in performance measures due to concurrent health care reforms, both nationally and statewide. The IE will examine if post DSRIP values are better than those of the pre-DSRIP period from the standpoint of utilization, spending, and change in outcome measures when compared to the newly designed Medicaid program.

Even though the IE will use the interrupted time series (ITS) design as the main component of their analysis, the ITS assumes that, without the intervention, trends in the outcome are not affected. The ITS design does not require the use of a comparison group, but is limited in controlling for external shocks (i.e., Medicaid expansion, individual mandate, overall changes in medical practice). This motivates the IE to explore if an appropriate non-DSRIP control group of patients (using propensity score or exact matching) or a comparison group of non-DSRIP providers (using cluster analysis to find similar hospital sites) can be identified for conducting DID analysis using time series and panel data. The IE realizes that a non-Medicaid population as a control group will be hard to identify because it would likely differ in many ways from the Medicaid population in terms of socio-demographic, and more importantly, by health. It will experiment with creating these comparison groups from the non-Medicaid population by matching all payer SPARCS data with DSRIP network information for at least a subset of its research questions. In addition, the IE will use a full-scope, Medicaid-enrolled, non-DSRIP attributed population as a control if the data are available. For research questions pertaining to performance in specific projects, PPS not selecting the project can also be used as a control group. The DID estimator only requires that in the absence of the treatment, the average outcomes of the treated and control groups would have followed parallel paths over time and that responses to "common shocks" (i.e., Medicaid expansion, individual mandate, overall changes in medical practice) are similar. Even this assumption may not be reasonable because the pre-treatment characteristics may be associated with the dynamics of the outcome variable that can affect the control and treatment groups asymmetrically. In this situation, the IE will experiment with Abadie´s (2005) simple two-step semi-parametric strategy to estimate the average treatment effect of the treated. These methods will have to be corrected for serial correlation in the outcome variable by using appropriate cluster analysis. The IE plans to experiment with the aforementioned ideas during the current year and use statistical tests to decide whether a comparison group can be identified for each of its research questions. For those questions where a suitable comparison group could not be identified, the IE will use the ITS to study the effect of DSRIP. These results will be reported in the "2019 Statewide Annual Report" and the "CMS 2019 Interim Evaluation Report."

Research Questions

Research Question	Hypotheses
To what extent did PPS achieve health care system transformation, including increasing the availability of behavioral health care?	Health care service delivery will show greater integration. Health care coordination will improve. Primary care utilization will show a greater upward trend. Expenditures for primary care services will increase. Utilization of, and expenditures for, behavioral health care service will increase. Expenditures for emergency department and inpatient services will decrease. Primary care, behavioral health, and dental service utilization will increase among the uninsured, non-utilizing, and low-utilizing populations, while emergency department use will decrease.
Did health care quality improve as a result of clinical improvements in the treatment of selected diseases and conditions?	rough clinical improvements implemented under DSRIP, health care quality in each of the following areas will increase: Behavioral health Cardiovascular health Diabetes care Asthma HIV/AIDS Perinatal care Palliative care Renal care
Did population health improve because of implementation of the DSRIP initiative?	Promote mental health and prevent substance abuse (MHSA) Prevent chronic diseases Prevent HIV and STDs Promote healthy women, infants and children
Did utilization of behavioral health care services increase as a result of DSRIP?	Utilization of, and expenditures for, behavioral health care service will increase.
Was avoidable hospital use reduced because of DSRIP?	Avoidable hospital discharges and emergency department utilization will be reduced. Costs associated with hospital inpatient and ED services will show reductions or slowed growth.
Did DSRIP reduce health care costs?	Health care expenditures associated with services under DSRIP will show a reduction or slowed growth
What were the successes and challenges with respect to PPS planning, implementation, operation and plans for program sustainability from the perspectives of DSRIP planners, administrators and providers, and why were they successful and challenging?	This RQ is not applicable to the Time Series Analysis. See Section C.

The IE will consider two possible "comparisons." One is a patient-level control group made up of Medicaid beneficiaries who were not exposed to any PPS intervention for a certain amount of time, so they are direct controls for the intervention group of patients. The IE can match using propensity scores from Medicaid enrollment and claims, plus geography if possible. There is another "comparison" that is at the hospital level; which is hospitals that did not participate in the DSRIP but have similar characteristics. This may be a challenge or actually be impossible to identify, because most safety net hospitals in New York are in a PPS. In that case, the IE would use the average rates of hospitalizations, Medicaid spending, ER visits, etc. and compare those hospital-level outcomes from the comparison hospitals to the PPS hospitals. In both cases, the IE could use DID, Time Series, or ITS. There are limitations for each. In DID, the IE is relying on two assumptions: 1) parallel trends, and 2) common shocks external to the intervention. In ITS, the IE is assuming that pre-intervention trends continue. The IE does not know which is true at this point because the IE does not have the data, so they will assess the utility of both methodologies. The IE can do DID with propensity score matching with patient level data, because there are enough non-PPS patients to find matches. DID with hospital comparison is tougher, because there will not be a good match one-to-one with PPS hospitals. Finally, DID analysis with another state is not feasible. It would not be within the scope of the IE contract for the IE to perform DID with comparisons from a non-DSRIP state.

As described in Section B.1, during the current year, the IE will explore creating a control group of non- DSRIP patients and a comparison group of non-DSRIP hospitals and assess whether it is feasible to use them given that DSRIP is so far reaching. Most Medicaid beneficiaries are receiving care and being exposed to PPS even if not technically attributed under the 50% threshold.

The Time Series Analysis will use a "global" comparison group to develop a state-wide control group of hospitals.

Additional information regarding the above RQs:

Sub research questions were added or expanded for those noted below but others are not possible within the current scope of the contract.

RQ3. Also, racial and ethnic disparities will be addressed with respect to the following metrics: premature deaths, newly diagnosed cases of HIV, preterm births, adolescent pregnancy rate per 1,000 females aged 15-17, percentage of unintended pregnancy among live births, and infants exclusively breastfed while in the hospital. Disparities on these outcomes will be measured as ratios and will be treated as additional outcomes at the statewide level with the prediction that these ratios will show improvement (i.e., will be reduced) following DSRIP implementation.

RQ6. It is hypothesized that following the introduction of DSRIP, the health care of the Medicaid patients has become better and also the program has become economically more efficient. Due to small sample size and multiple hypotheses testing, correct significance levels have to be determined by controlling the false discovery rate (FDR), rather by conventional Bonferroni bounds.

Supplemental RQ: Was DSRIP cost effective in terms of New York State and federal governments receiving adequate value for their investments?

A set of measures described in the "DSRIP Strategies Menu and Metrics" will be used to quantify the performance measures. Because a large number of hypotheses will be tested, the problem of the inflated type I error will be mitigated by replacing the conventional Bonferroni methods with the control of the false discovery rate (FDR), defined as the expected proportion of errors (i.e., null hypotheses that are actually true) among a set of null hypotheses that have been rejected. In addition, a comparative analysis will be conducted for efficiency and effectiveness based on the chosen projects on alternative domains using a DID methodology.

B.3 Determination of Cost Effectiveness:

Cost-effectiveness analysis, in the simplest terms, calculates the ratio of the amount of "effect" a program achieves for a given amount of cost or investment in the program incurred, or conversely, the amount of cost required to achieve a given impact. For program evaluation such as the DSRIP evaluation, this means measuring the impact of a program on achieving a given policy goal (e.g., the additional reduction in avoidable hospital use as a result of DSRIP against the cost of the program). This ratio, when calculated for a range of alternative programs addressing the same policy goal, conveys relative impacts and costs of these programs in an easily understandable and intuitive way.

The value of cost-effectiveness analysis is two-fold: first, its ability to summarize a complex program in terms of an illustrative ratio of effects to costs, and second, the ability to use this common measure to compare multiple programs evaluated in different contexts and in different years. The first requires technical correctness with respect to the program´s actual costs of administration of the program and impacts as they were evaluated, while the second requires adherence to a common methodology for estimating costs and effectiveness across various programs. For cost-effectiveness analysis to be a useful tool, it is necessary to agree on an outcome measure that would be the key objective of many different programs and policymakers. In this evaluation, there are two obvious contenders: the reduction in avoidable hospital readmissions - a goal of the DSRIP intervention, and the improvement in health outcomes for the population— public health goal. Since this is a summative evaluation method, the entire pre-post DSRIP time horizon will be taken for the analysis. The DSRIP policy is compared to a baseline policy of do-nothing, or status quo—traditional Medicaid for New York State. The incremental costs of each life-year gained or of hospital readmissions of the traditional and DSRIP Medicaid programs will be calculated, and the incremental cost per life year gained for each scenario will then be elicited. Sensitivity analysis will be conducted to assess the robustness of the results due to other policy changes in the system or a change in case-mix of the beneficiaries. The uncertainty surrounding the effectiveness of the program to reduce hospital admissions and readmissions and to improve life years gained, and their impact on total costs per life year gained or reduction in hospital readmissions, will be calculated using the minimum and maximum effectiveness values from the literature review currently in process by the IE concerning these outcomes.

The Time-Series component of the evaluation will focus on the macro-level cost-effectiveness analysis with the counterfactual being addressed in the pre-post DSRIP comparison. The comparative component will focus on the variations among PPS in the achievement of the effects noted above among the various programs and projects initiated across the State under the DSRIP program. Working closely with the NYSDOH in order to determine yearly costs of administering the Medicaid program in New York State prior to, and after the DSRIP incentive program, these costs will be compared to the yearly measures pre- and post-DSRIP in avoidable hospital readmissions and health outcomes such as life years gained. Cost effectiveness thresholds will be determined with the NYSDOH prior to the cost effectiveness evaluation and sensitivity analysis will be performed given that there are many health policy changes that affect the Medicaid population during this period of the DSRIP intervention as well as some provider changes within DSRIP. The complexity of this analysis will depend on the type and richness of the data acquired from the Assessor and NYSDOH. This macro-level analysis builds on findings at the state level of RQs 3 through 5. Since this is a pre-post comparison of costs and effects at the macro or PPS-level of analysis, the measures will be discounted for time value and adjusted for uncertainty and risk-attitude as noted above. Further, marginal cost-effectiveness will be calculated since the programs reflect an on-going decision-making process.

Challenges for the Cost-effectiveness Analysis
There are obstacles to obtaining the cost-effectiveness determinations, which include difficulties in obtaining costs of the DSRIP intervention by PPS or over time as the PPS learn about the best methods to deliver their project workflows to the targeted population. In order to mitigate this issue, sensitivity analysis will be performed in order to determine the robustness of the outcomes over time and for various policies that simultaneously affect the Medicaid population over the period considered.

Section C: Qualitative Analysis:

Qualitative information obtained from DSRIP planners, administrators, providers, and beneficiaries is expected to play a vital role in the DSRIP evaluation. The IE´s qualitative methods will:

Identify facilitators and barriers to PPS achieving progress on pay-for-reporting/pay-for- performance metrics using feedback from PPS administrators, providers, and patients, as well as to identify these issues that are characteristic of particular strategies or projects.
Conduct PPS case study evaluation by obtaining information from DSRIP stakeholders on an ongoing basis on program planning, implementation, operation, and effectiveness to guide quality improvement through project refinements and enhancements.

Qualitative methods to be used include key informant interviews, focus groups, and surveys, with issues to be investigated qualitatively to include notable program outcomes and challenges, effectiveness of governance structure and provider linkages, contractual and financial arrangements, challenges in the delivery of patient care, the effect of other ongoing health care initiatives (e.g., New York Prevention Agenda, Affordable Care Act) on DSRIP implementation and operation, and patient experience and satisfaction with services. In the qualitative component of the evaluation, the IE will develop qualitative instruments to address the central evaluation questions and to augment results of quantitative analysis. This will include the determination of interview or survey questions with appropriate review and pre- testing to ensure that questions are comprehensive, understandable, and reliable, a plan and schedule for data collection, and a plan for analysis.

The IE´s qualitative data collection will be designed to address the RQs, objectives, and aims presented in several of the main research questions, the broad objectives and issues to be addressed in this section. Qualitative data will provide context for the quantitative questions assessing RQ 1-4, which focus on system transformation, clinical improvement, and population wide projects (Domains 2-4). These questions focus on the implementation of projects initiated with the DSRIP program. Qualitative data will also address RQ 7, which asks about successes and challenges related to different aspects of the DSRIP program.

Gaining an understanding of these RQs, aims, and objectives will provide integral information on the implementation and operation of DSRIP, the successes and challenges of PPS and projects within DSRIP, and guidance on sustaining programming going forward.

The IE will use four major data sources to collect qualitative information from a number of relevant stakeholders in order to reach a diverse perspective and maximize the information collected. Interviews with PPS administrators, surveys with patients, and surveys with project-associated providers will be completed once over the course of data collection for each PPS. While it would be helpful to survey non- engaged providers for comparative purposes, this additional survey component is largely not feasible because the PPS will not have accurate contact information for partners from which they are no longer engaged. This component is also outside of the scope of the research questions. These data sources will be used to collect qualitative data on three major focal points: the DSRIP program overall, individual projects, and patient experience. In general, interviews and focus groups will be the major data source of patient satisfaction and experience, and surveys of providers will be the major source of project specific data. These methods of data collection were selected to be able to efficiently and thoroughly address all of the areas of inquiry described in the table below.

Areas of Inquiry	Interviews with PPS Administrators and Staff	Focus Groups with Providers on Projects	Surveys with Patients	Surveys with Providers on Projects
DSRIP Program Overall
Program planning, operation, and effectiveness	X	X		X
Program outcomes and challenges	X	X		X
Plans for program sustainability	X		X
Effectiveness of governance structure and provider linkages	X	X
Facilitators and barriers to PPS achieving progress on pay-for-reporting/pay-for- performance metrics	X	X		X
Contractual and financial arrangements including provider transformation to Value Based Payments	X	X		X
Challenges in the delivery of patient care	X	X		X
The effect of other ongoing health care initiatives (e.g., New York Prevention Agenda, Affordable Care Act) on DSRIP implementation and operation	X	X		X
Project Specific
Progress/effectiveness of projects focused on system transformation	X	X		X
Progress/effectiveness of projects focused on behavioral health	X	X		X
Progress/effectiveness of projects focused on clinical improvement and population			X	X
Identify the issues that are characteristic of particular strategies or projects (in terms of metrics)	X			X
Patient Experience
Patient satisfaction and experience			X

Prior to collecting data through surveys, focus groups, and interviews, there are a number of preparatory actions that will occur, including identifying participants, preparing protocols, and working with state and local Institutional Review Boards to ensure compliance with human subjects´ requirements.

The IE will work closely with the PPS staff and administrators to identify the appropriate stakeholders needed for interviews and focus groups. PPS will aid the IE by providing lists of names and contact information for appropriate PPS planners and administrators for interviews. In addition, lists of names and contact information (including email addresses) will be sought from PPS identifying relevant providers that are associated with and knowledgeable of each of their DSRIP projects. This information is necessary for the administration of surveys addressing specific projects. Because provider lists are so vast within the PPS, identifying the appropriate stakeholders is important as it will guide recruitment efforts for focus groups, with the goal of recruiting a diverse group of perspectives.

Another preparatory activity is developing question sets and protocols. Because the goal is to hear diverse perspectives on research objectives, numerous questions will be asked from multiple stakeholders to gain a holistic understanding of all areas of inquiry. Question sets will be developed for each method of data collection. Interviews and focus group question sets will be semi-structured, such that all respondents (PPS administrators or providers) will be asked the same questions; however, some items may elicit probing for additional information. Survey items will be selected using existing measures whenever possible to ensure psychometrically rigorous measures are employed. Questions will be developed for any question areas without existing measures. All items will be carefully reviewed and pre-tested, which will ensure that all items are easily understandable and thorough. All data collection protocols will be approved by the Institutional Review Board (IRB) at the SUNY at Albany, for human subjects´ research. Changes to interview and focus group questions may be necessary based on responses during early data collection. Any changes will be carefully reviewed by the IE and approved by the IRB, as needed before use. In additional, all IE staff involved in data collection will be trained on the handling and storage of confidential information.

Once approaches are developed and participants are identified, focus groups, interviews, and surveys will be scheduled and conducted.

Data sources by population, method and time

Population	Method	Cycle 1 April 2017- Dec. 2017	Cycle 2 April 2018- Dec. 2018	Cycle 3 April 2019- Dec. 2020
PPS Administrators	Telephone Interviews	25		25
PPS Team Leaders	Telephone Interviews		125
DSRIP-Associated Providers	Focus Groups	8 Groups	8 Groups	8 Groups
DSRIP-Associated Providers	Web Survey	2400 w/response rate goal 50-60%	2400 w/response rate goal 50-60% %	2400 w/response rate goal 50-60%
Patients	Phone/Mail Survey	CAHPS Survey Data from DY1-5

C.1. PPS Administrative Key Informant Interviews

Sample Selection
Key informant interviews will be conducted with administrators and staff annually in each of the 25 PPS located throughout the four regions of New York State. In the first year of data collection, interviews will be conducted with PPS administrators. Using purposive sampling (Bryman 2012; Creswell 2013; Patton, 2002), PPS administrators chosen for interviews will be individuals who are most knowledgeable about DSRIP start-up, implementation, ongoing processes, administrative components, and challenges. Specifically, the sample will include the chief executive officer, chief operating officer, or the individual currently responsible for all operations; someone with authority who was involved in PPS startup; the fiscal officer or individual involved in financial transactions; and others identified by either the NYSDOH or the PPS who are vital to the ongoing operations of the PPS. Each PPS had leadership join at different junctures, and many will have leaders with specialized knowledge in certain areas. In the second year of data collection (DSRIP DY4), the research team will schedule interviews with PPS leaders responsible for the implementation and operation of their selected projects. Each PPS has selected up to 11 DSRIP projects from the DSRIP Project Toolkit (e.g., the integration of primary care and behavioral health services, development of community-based health navigation services). These interviews will shed light on factors related to the successful implementation of various DSRIP projects. The sample will include all PPS staff members with professional experience launching or running PPS projects. In the third year of data collection, the research team will again schedule interviews with PPS senior leadership for follow up.

Data Collection Procedures Telephone interviews will be scheduled at the convenience of the PPS staff and administration and will be conducted with PPS staff and administrators annually in these periods:

Research Cycle 1 (July - December 2017): Senior Leadership
Research Cycle 2 (July - December 2018): PPS Staff Responsible for Projects
Research Cycle 3 (July - December 2019): Senior Leadership

The interviews will be guided by a semi-structured interview protocol and should take no more than two hours to complete. A core set of questions will be asked of all key informants, and a subset of questions and probes will be developed based on each key informant´s roles, knowledge, and responsibilities.

Interviewers will be trained by experienced staff at the Center for Human Services Research who have many years of experience in qualitative interviewing. Trained interviewers will study and review the semi-structured interview protocol at length prior to interviewing to ensure that adequate interview structure is maintained, and interviewing is conducted seamlessly. Interviews will be recorded electronically to preserve the content and ensure that each interviewee perspective is accurately captured. Interviews will be transcribed manually during the course of the interview by a research assistant with the Center who will later review the recording and transcribe any missing content.

In the first year of data collection (DSRIP DY3) with the senior leadership team of the PPS, the interview questionnaire will be designed to address the following topics:

Initial formation of the PPS - exploring the development of the relationships required to form the PPS as well as the project selection.
Challenges during years 0-2 of DSRIP implementation - exploring launching of the projects, workflow, and engagement with community partners. The IE will also ask about resources required to operate projects.
Successes during years 0-2 of DSRIP implementation - exploring the application process, project workflow, community partner engagement, and projects.
Committees - exploring effectiveness of the PPS´ governance related-committees and modifications to the committees over time. Also explores challenges and successes related to committees.
Data - exploring what specific data (quality, financial, utilization, and/or population health measures) the PPS thinks is most important to evaluating progress and success.
Account Support - exploring the account support provided by NYS for the PPS and the projects.
Value based payment - preparatory activities and sustainability plans for the future
Viewpoint - exploring changes to the healthcare system from DSRIP and other interventions in NY.
Other issues - comments on areas the IE may have missed.

In the second year of research collection (DSRIP DY4), the research team will schedule telephone interviews with PPS staff responsible for projects. The topics to be discussed in the interview are:

Initial planning of the projects - exploring effectiveness of project selection and planning.
Major outcomes and challenges of the projects - exploring project launch, major milestones achieved and missed, barriers to project implementation, and methods barriers were overcome (or plans for overcoming).
Program sustainability - exploring plans for project sustainability (i.e., continuing projects post- DSRIP).
Structure and provider linkages on projects - exploring the effectiveness of the project governance structure and provider participation in reaching project milestones.
Facilitators and barriers to PPS achievement of progress on pay-for-performance metrics related to project milestones - exploring the ways in which PPS are working toward pay-for-performance and the facilitators and barriers for particular projects that are excelling or falling behind on milestones.
Contractual and financial arrangements - exploring how PPS financial contracts and planning contribute to project milestones and success or failure.
Changes in the delivery of patient care - exploring the way DSRIP projects have affected the way patients are treated in terms of quality and delivery of care.
Other ongoing health care initiatives - exploring whether other ongoing initiatives (e.g., NY Prevention Agenda, ACA, Value Based Payments) have had an effect on specific project implementation and operation.
Progress/effectiveness of projects focused on system and VBP transformation.
Progress/effectiveness of projects focused on behavioral health.
Other issues - comments on items we may have missed.

In the third year of data collection, DSRIP DY5, the research team will again schedule interviews with PPS senior leadership. Anticipated topics for the final key informant interviews are:

Challenges during years 3-5 of DSRIP implementation - explores launching the projects and other workflows including engagement with community partners. We will also ask about resources required to operate projects.
Successes during years 3-5 of DSRIP implementation - explores project implementation and workflows, and provider and community partner engagement.
Pay-for-performance - a lookback at the shifts related to pay for performance from DY3 forward.
Committees - explores effectiveness of the PPS´s governance related-committees and modifications to the committees over time. Also explores challenges and successes related to committees.
Data - explores what specific data (quality, financial, utilization, and/or population health measures) the PPS thinks is most important to evaluating progress and success.
Account Support - explores the account support provided by NYS for the PPS and the projects.
Value based payment - successes and challenges to date.
Viewpoint - changes to the healthcare system from DSRIP and other interventions in NY and future PPS Sustainability plans.
Other issues - comments on items we may have missed.

Challenges
There are a number of challenges to key informant research of this scale. First, engaging the study population to participate in interviews may be difficult. The research team is requesting time from busy professionals. The research team will mitigate this challenge in several ways. The IE will craft a well-structured communications plan that carefully lays out what is expected of the PPS professionals at each juncture in terms of content, time, and its impact on their performance. Having this communications plan in place will streamline the interviewing process, instill participant confidence in the researchers´ methods, and increase the likelihood of participation. In addition, researchers will also communicate the extrinsic rewards of participating in the research to interviewees (e.g., input from interviews will be communicated to policy makers who have the power to foster meaningful changes at the system level). The communication plan, combined with a thorough explanation of the extrinsic rewards, will combat the difficulties of participant engagement.

Another challenge is that because the evaluation begins in the middle of the demonstration, there may be difficulties in recall of initial startup and implementation phases of DSRIP. The research team will resolve these challenges by using the first research cycle (operational in DSRIP DY3) to ask retrospective questions on the DSRIP initiative to date to glean a broad characterization of DSRIP process and progress. The questionnaire was designed with this lookback procedure in mind and consequently tailored to contain probing questions to enhance participant recall. We will also recruit individuals who have historical knowledge of the program to the key informant interviews so that recollection is augmented. Retrospective data collection is not ideal, but it is commonly used to capture perceptions of change from participants. In addition, qualitative data for the remaining 2.5 years of the demonstration project will be collected in real time, which will provide context and information regarding both the present operation and planned sustainability of projects.

Focus Groups with Project-Associated Providers:

Focus groups will be conducted with select project-associated providers. The sample will be selected based on geographic location and provider type. Focus groups function best when groups are somewhat homogenous, which fosters greater cooperation, greater willingness to communicate, and less conflict among group members (Stewart & Shamdasani, 2015). The creation of groups based on provider types ensures that each focus group is comprised of individuals whose work is similar, allowing for more candor and in-depth participation from individuals. Drawing from research on best practices for conducting focus groups, the number of participants for each focus group will be limited to 10-12 individuals; this group size allows participants sufficient time to share insights yet is large enough to provide a diversity of perspectives. The focus groups will be guided by a focus group category, with questions tailored to each PPS group. Each focus group will last approximately one to 1.5 hours. Focus group participants will be informed of the research protocol regarding confidentiality before the session begins. This includes reporting the findings as a group and not associating anyone with individual remarks. With the permission of the participants, all qualitative focus groups and interviews will be audio-recorded and transcribed verbatim, and field notes will be taken to document the process. Planned topics for the focus groups include:

Engagement of providers with DSRIP activities and projects
DSRIP transformation of professional responsibilities
Integration of projects with other projects or services received by patients
Characterization of DSRIP to-date
The effect of other ongoing healthcare initiatives on DSRIP, such as NY Prevention Agenda and the ACA
Progress of the DSRIP projects and impact on provider´s area of work
Factors that influence achieving pay-for-performance
Barriers that influence achieving pay-for-performance
Transformative efforts toward Value based payment
Characterization of the contractual and financial arrangements
Other changes the project partners would recommend

Challenges
A critical challenge for conducting these focus groups includes establishing a sampling frame that captures the allocation of provider types across PPS groups and counties. The research team developed the hybrid focus of balancing provider types by geographic areas after significant consultation with key leaders at NYSDOH and the DSRIP PPS Account Support team. The hybrid focus will allow researchers to combat these challenges to the utmost extent possible.

A challenge that is inherent to conducting standard focus groups includes difficulty recruiting busy professionals from their demanding clinical responsibilities. Using a communication strategy that includes support from the PPS entities and DOH, the research team will convey information on the benefits of participation to all providers and provide flexible scheduling times, such as early morning or evening times if necessary. Focus groups also face challenges in terms of gathering retrospective data. As the IE is conducting focus groups across three time points, the IE will only ask lookback questions to the groups held in research collection year 1. To address this challenge, the IE will supplement the data collected via this method by also collecting lookback data from the DSRIP-associated provider survey respondents. The IE will be able to ask more detailed questions about progress, successes, and challenges to date via survey techniques.

C.3. Survey for Patients: In 2015, in response to the NYSDOH Request for Proposals (RFP), the IE proposed to collect patient surveys. The original evaluation plan described that each PPS would collaborate with researchers to identify patients who were eligible to participate. Planned criteria included patients age 18 and older who had not opted out of DSRIP-related data collection. Research cycle 1 was slated to begin in March 2017 (DSRIP Demonstration Year (DY) 2) and end December 2017 (DSRIP DY3). The survey was planned to repeat for three research cycles, ending in 2019. Planned survey topics included patient satisfaction, reactions to changes to care, and patient experience overall. NYSDOH is currently fielding a CAHPS survey that will be provided to the IE, rather than requiring the creation and administration of a separate DSRIP-specific survey.

In order to obtain adequate response rates for this difficult-to-reach population, researchers planned on using a hybrid mail and web-based approach. The target sample size for the survey was anticipated to be 1,500 patients surveyed with a response sample of 450 per research cycle.

After a comprehensive review of challenges to an IE-sponsored patient survey and the current data collection burdens on Medicaid members, the IE received approval to perform secondary analysis on the NYSDOH-sponsored Consumer Assessment of Healthcare Providers and Systems (CAHPS©) Health Plan survey for Medicaid enrollees that has been run since DSRIP DY1. After DSRIP was launched the NYSDOH tailored the report to assist NYSDOH and participating PPS in pinpointing opportunities to improve Medicaid members´ experiences. The survey, the CAHPS© C&G Adult Medicaid core survey (Primary Care, version 3.0), is a nationally vetted tool designed to measure patient experiences. The survey was customized to include 18 supplemental questions concerning health literacy, health promotion, and care coordination. The NYSDOH has run the CAHPS © survey each year since year 1 of DSRIP. The survey is sent to 1,500 patients from each of the 25 PPS for a total sample size of 37,500.

The IE´s original evaluation questions for patients were focused on how the patients were experiencing change and their satisfaction with that change. As we now know, since patients do not know that they are in a PPS or part of DSRIP, the scope of the RQs questions has changed. The IE is now interested in reviewing trends and changes to access to care and experiences with care. They are aware that there is no information from before participation in DSRIP or from a control group. As this is an implementation sub-study of the larger IE study, the IE can integrate their findings with the rest of their data without a control group. They will measure change through displays of descriptive statistics from both individual questions and composite measures. They will display the trends from each PPS and statewide.

For questions related to access to primary care, the IE will use:

Q2. Provider is usual source of care
Q3. Length of provider relationship is at least 1 year or longer

For questions related to experiences with care, the IE will use:

Q25. Rating of Provider
Composite: Getting Timely Appointment, Care, and Information
Composite: How Well Doctors Communicate with Patients
Composite: Care Coordination
Composite: Helpful, Courteous, and Respectful Office Status

Challenges
The IE had always planned to use descriptive statistics for any patient survey data. While they cannot view or analyze data to the individual level from the CAHPS © reports, they can look at the breakdown of composite measures across the state and within each individual PPS. The IE will also provide response rates for each PPS. These data points are appropriate for their planned reports including both the annual statewide and PPS reports as well as the interim and final Independent Evaluator reports slated for 2019 and 2021, respectively.

In addition to the data from these comprehensive, representative surveys, the IE may also explore patient focus groups. The IE would hold six to eight patient focus groups centered around changes from DSRIP project 3.a.i. Integration of primary care and behavioral health services. Recruitment of these patients is dependent on the PPS staff linking the IE with medical facilities and providers that would be open to hosting focus groups. Development of these groups is also dependent on the rollout of project 3.a.i. and that project´s patient engagement. The IE will work closely with the NYSDOH in DSRIP DY4 to determine feasibility of this approach and the types of data that would be appropriate to collect from the consumer facing group. The IE may also request to review findings from ongoing STC-required Consumer Education Campaign focus groups that the NYSDOH is running in complementary efforts.

Survey for Project Providers

Electronic Survey of Project-Associated Providers
Sample Selection

In order to gather uniform information on the functioning of individual projects, an electronic survey will be administered annually to project-associated providers. The sample will be drawn from lists maintained by PPS administrators of providers who are associated with each of their projects and known as "engaged providers." The IE anticipates the survey will target 2,400 providers annually, a number that is based upon response rates in past research with health care professionals that have generally yielded response rates between 50 and 60% (McLeod et al. 2013; Nielsen et al., 2009; Podichetty et al., 2006). A sample of 1,200 health care providers will allow researchers to examine the data by various subgroups (e.g., provider type) and allow for analyses based on geographic location. Researchers at the Center for Human Services Research are immersed in the literature on best practices in survey collection, have extensive experience in this area, and have specifically investigated approaches for maximizing participation in electronic surveys among health care professionals to ensure an adequate sample is achieved (e.g., McLeod et al. 2013).

Data Collection Procedures
Surveys will be conducted with DSRIP-associated providers once per year in the following periods:

Research Cycle 1 (July - December 2017)
Research Cycle 2 (July - December 2018)
Research Cycle 3 (July - December 2019)

The electronic survey will utilize Qualtrics Survey Software to ensure accurate data capture and preserve participants´ responses in a confidential manner. Qualtrics Survey Software is known for its elegant design that will mitigate any difficulties that generally arise in web navigation with electronic surveys. In addition, the survey length will be as short as possible while collecting all relevant information so as to encourage participant responses and reduce respondent fatigue.

The link to the survey designed in Qualtrics will be emailed to individuals from the list of engaged providers. The sample of engaged providers will be developed from the DSRIP MAPP Provider Import Tool and its hybrids used by each PPS. Contact information may need to be validated from a second contact database but the Provider Import Tool or the similar tool being used by the PPS will be the determination of how providers are designated as "engaged." Providers will have ample time to respond and gentle reminder follow up emails will be sent to encourage providers who have not yet participated to complete the survey.

The survey questions will focus specifically on progress within individual projects, barriers and facilitators to project implementation, and perceived effectiveness. The survey will generate user-based responses that will allow the IE to provide individualized feedback to each PPS for quality improvement of their projects (Bate & Robert, 2007). Topics will include:

Service provision within each project dimension
Project operation compared to the planned model and reflection of this change over implementation years
Future anticipated changes to project models
Factors of each project that have helped or hindered with implementation
Challenges faced in working with the PPS entities
Challenges faced with specific projects and corrective actions (if any)
Changes to project(s) or DSRIP operation
Level of satisfaction with planning process
Reflections on what worked well and less well during the planning process
Value based payment readiness and change
Changes to program planning processes for specific projects
Satisfaction with current operation
Overall perception of DSRIP
Overall perception of projects

Project Providers Survey Challenges
One major challenge to survey data collection includes identification of the sample. The research team will work with NYSDOH and the DSRIP Account Support team to develop a method to pull the sample and ensure its accuracy. It is anticipated that the sample will be pulled by the research team manually from the MAPP Network tool. The challenge with sample identification in this case is that it will require collaboration with these entities as well as the PPS to identify potential providers who will participate; however, the research team at the Center for Human Services Research is poised to meet this challenge based on the team´s extensive experience in coordinating data collection endeavors of this nature through other quantitative research projects. This challenge will be mitigated both through the experience of the research team as well as the planning that went into utilizing the MAPP Network tool to pull the sample.

A second anticipated challenge is accurate categorization of provider type. Upon receipt of feedback from the PPS entities during the DSRIP Mid-Point Assessment, NYSDOH allowed PPS to broaden their own categorization tools in early 2017. This tool replaces the Provider Import Tool (PIT) and allows for greater customization. As not every PPS will broaden provider categorization. The survey must be designed to collect responses on categorization type and should mirror the language that the PPS entities use to define their providers. Another challenge is that providers may engage with multiple PPS entities on the same or different projects. The survey will be designed to allow for separate responses for project questions per PPS entity.

C. 5 Other Data Collection

To reflect the real-world nature of this evaluation and to gather data from all stakeholders, the IE will explore the addition of other surveys or interviews.

Managed Care Representatives - The IE will explore the addition of a survey with managed care representatives in DSRIP DY5. The sample would include representatives from the 18 mainstream plans. Topics to be covered in the survey include successes and challenges of DSRIP related initiatives to date, engagement with PPS and transformative efforts of DSRIP toward managed care plan value-based payment contracting.

Project Approval and Oversight Panel (PAOP) - The IE will survey the members of the Project Approval and Oversight Panel in DSRIP DY4 to gather their perspectives on the implementation and process progress of DSRIP. They will also collect their feedback and suggestions.

C.6 Implementation/Process Analysis Summary

Analysis will focus on identifying usable feedback for improvement for each of the 25 PPS. An additional focus will be identifying common and unique themes that arise in the data to inform the evaluation of DSRIP implementation as a whole. Any quantitative survey data will be analyzed using SPSS statistical software. The qualitative data obtained through key informant interviews, focus groups, and open-ended survey questions will be transcribed and analyzed, using a qualitative data software program.

Coding and analysis of qualitative data will follow the strategies described by Bradley, Curry and Devers (2007). Once data are organized and reviewed, the IE will use an integrated approach to identify and categorize the data according to concepts, relationships between concepts, and evaluative participant perspectives. Categorization based on setting and participant characteristics will also be completed, as appropriate. This categorization process facilitates the development of taxonomies, themes and theory, and comparisons. Responses will then be reviewed independently by at least two IE staff utilizing the finalized coding structure. Any coding discrepancies between reviewers will be resolved with discussion to achieve consensus. Coded data will be analyzed and interpreted to identify major concept domains and themes. Analysis will focus on understanding of the DSRIP initiative as a whole, as well as on understanding of each individual PPS.

Progress on qualitative data collection and analysis will be included in quarterly progress reports, as well as any changes in implementation strategies that have occurred based on feedback to the PPS and project sites. In addition, results from the qualitative data analysis will be reported in the overall annual reports. Information on individual PPS will be presented in annual case study reports to each PPS to be used to guide quality improvement through project refinements and enhancements. Qualitative data will also contribute to the interim and final summative reports.

Section D:

Comparative Analysis:
To address questions pertaining to the effects of type of projects adopted by the PPS, the relative effectiveness of specific strategies employed within project types, and the contextual factors associated with PPS success or failure to demonstrate improvement in the metrics associated with each domain, quantitative and qualitative comparatives may include the following:

Where there is variation in the strategies selected per the PPS project requirements described in the STC above, assess the effect on the pertinent outcome of PPS having selected a particular strategy. For example, a comparison would be made in the improvement in diabetes care (Domain 2) between PPS that implement a project to address this issue and PPS that do not.
The relative effectiveness of particular projects intended to produce the same outcome. For example, among PPS that opt for a strategy to improve asthma care, compare such improvement between those PPS that chose to implement a project to expand asthma home-based self- management programs to those PPS that chose alternative projects to improve asthma care.
Identification common to those PPS receiving or not receiving maximum payment based on project valuation.
Comparisons between PPS operating in different regions of New York to identify successes and challenges associated with local resources or procedures.
Patient-level comparisons by factors such as age, sex, race, presence of selected chronic conditions, and mental health/substance abuse status to obtain information on variations in service experience and satisfaction under DSRIP, by patient characteristics.

The comparative analysis will be designed by the IE to address the seven (7) research questions (RQ)(see Section B.2). The IE´s approach is to apply quantitative techniques to assess relative PPS performance on domain-specific metrics over time and supplement this work with qualitative data collection to provide further contextualization of the findings. Specifically, the IE will supplement their quantitative analyses of publicly available data sets by analyzing other primary data, such as 1) focus groups, 2) semi- structured key informant interviews with PPS administrators and staff, 3) surveys of providers with semi- structured interview follow-up, and 4) surveys with patients, to provide further contextualization of results. The approach will include clustering PPS to create comparison groups according to project selections, the uses of DID methodology, as well as multi-level modeling.

Further the IE will develop a compendium of domain projects across all DSRIP PPS that includes information important to the comparative analysis. The compendium will include information on timeline (start and end dates of implementation), planning decisions (changes that occurred prior to implementation or during implementation), fidelity of the intervention to its original intent (ranked low to high), relative success to internal expectations (low to high), and previous work (was the program new or building upon existing, pre-DSRIP activity). This compendium will allow the IE to examine variation between PPS within projects and across domains in a way that will contribute to the IE´s understanding of DSRIP and exploit less apparent differences between the programs and projects to drive analyses. For example, if two projects look the same "on paper" but one is new and one is based upon existing initiative, the IE might see differential outcomes (if the IE is looking at change over time).

The comparative analysis will be designed to address the seven RQs with specific emphasis on the five specific issues in this section above. The research aims for comparative analysis are:

To compare PPS performance on domain-specific metrics for those that did/did not adopt specific DSRIP projects.
To evaluate the relative effectiveness of specific strategies employed within specific projects.
To examine contextual factors related to PPS successes and failures in demonstrating improvement in domain-specific metrics.

The conceptual framework below depicts the factors that are expected to impact health outcomes in the broader context of the DSRIP program. System Transformation (Domain 2), Clinical Improvement (Domain 3), and Population-wide Strategies (Domain 4) are all anticipated to impact patient-level outcomes. Moreover, broad external factors, such as economic conditions, immigration, and unemployment, are also likely to influence patient outcomes. To this point, issues related to beneficiary eligibility and the frequency of patients going in and out of the Medicaid system tend to play a role in influencing health outcomes. In addition, the varying performance levels and culture related to organizations that are early adopters versus late adopters of DSRIP projects and strategic initiatives also are likely to play a role in determining patient-level outcomes.

Conceptual Framework:

Evaluating DSRIP, given the multiple PPS networks, partnerships, and projects within each domain, is a complex endeavor. The IE will leverage both qualitative and quantitative data to inform the evaluation design by embracing the variation across and within PPS interventions and the varied goals of each.

Early analyses will focus on the direct relationship between domain projects and the ultimate outcome measure. Analyses will be descriptive in nature when examining broader PPS outcomes, but additional multivariate analysis will be used to control for differences between populations, regions, providers, and other characteristics of the PPS that exist beyond the intervention or within the intervention project.

Descriptive Analysis Example for Domain 2 Impact on Emergency Department Visits:

In the example above, the underlying hypotheses are that specific Domain 2 projects will result in reductions in the percentage of emergency department (ED) visits per 1,000 total visits over time (from pre-DSRIP to post-DSRIP) in aggregate. Testing this hypothesis will simply use the inventory of DSRIP projects across PPS and use descriptive statistics to understand if the percent change in ED visit use was reduced in the five PPS that had a Patient Centered Medical Home (PCMH)/Advanced Primary Care intervention when compared to sites without a PCMH/Advanced Primary Care intervention, and separately calculate whether the 22 PPS with an integrated delivery system intervention experienced a reduction in ED visits when compared to those without an integrated delivery system intervention. These descriptive tables will give a general sense of what happened for the groups of sites that opted into a specific Domain project versus those that did not but does not address multiple interventions in the same domain or control for underlying PPS characteristics. The unit of analysis will be the PPS site and data will be pulled from the PPS project list and administrative records (Medicaid claims for ED visits) and/or PPS Quarterly Implementation Project Plan Reports (from the PPS to NYSDOH). The resulting table is likely to appear in the evaluation report in the following format:

Example Output for Bivariate Analysis by Project:

Domain 2 Project	Number of PPS Participants	Measure 1: Percentage Change in ED Visits per 1,000
Domain 2 Project	Number of PPS Participants	Baseline Rate	Year 1	Year 2	Year 3	Year 4	Year 5
1. Integrated Delivery System	22	1.3 per 1,000 visits	−0.2	−0.3	−0.4	−0.3	−0.5
2. PCMH/Advanced Primary Care	5	1.1 per 1,000 visits	−0.1	−0.2	−0.3	−0.2	−0.4

The second stage of descriptive analysis will focus on interactions between Domains and Projects between PPS networks, to better understand the impact of the customizability and flexible nature of the DSRIP interventions the IE is tasked with evaluating. The additive relationship of implementing a PCMH/Advanced Primary Care project along with an integrated delivery system project can be better understood and incorporated into the evaluation approach. The table below is likely to appear in the evaluation report in the following format:

Example Output for Bivariate Analysis by Project Combinations:

Domain 2 Project	Number of PPS Participants	Measure 1: Percentage Change in ED Visits per 1,000
Domain 2 Project	Number of PPS Participants	Pre-DSRIP Rate	Year 1	Year 2	Year 3	Year 4	Year 5
1. Integrated Delivery System only	19	1.2 per 1,000 visits	−0.2	−0.3	−0.4	−0.3	−0.5
2. PCMH/Advanced Primary Care only	2	1.0 per 1,000 visits	−0.1	−0.2	−0.3	−0.2	−0.4
2 & 3. PCMH/Advanced Primary Care + Integrated Delivery System	2	1.4 per 1,000	−0.2	−0.2	−0.3	−0.4	−0.5

In both of the examples above, the unit of analysis is the PPS, with the projects aligned with aggregate measures of ED visits reported or calculated at the PPS level. However, the IE also plans to leverage the individual level data when possible to understand the independent effects of each project on patient-level outcomes by controlling for individual patient characteristics for the beneficiaries nested within each PPS, and developing multivariate models to predict ED use over time using the Medicaid claims data to understand ED use for each individual. The regression analysis could focus on the rate of change in ED use over time, but because ED use is a fairly rare outcome at an individual level (more than half of subjects may have no ED use at all in a given year [Kaiser Family Foundation]), it would make more sense to use a two-step model predicting ED use (binomial logistic regression) and a conditional model (log-link Poisson or GLM model) for those with any ED use predicting the number of ED visits over time for each individual. Each individual would be nested in a PPS based on where they are attributed according to administrative records, and the qualitative data or progress reporting would be used to assign PPS values to capture categories of projects and/or variation in the interventions within project. While there are not sufficient degrees of freedom to do regression analysis at the PPS level, the individual level data would provide substantial data to test hypotheses about population health outcomes and measure change as a result of the DSRIP overall and individual projects or combinations of projects. The resulting regression equations would be based upon the distribution of the data and variables from the multiple data sources available to the IE. The two-step model would be based upon the following general theory: Step 1: Binomial Logistic Regression Predicting any ED Use

Y1pt Bo+ Bp1D2PROJ1t+ Bp1D2PROJ2t+ BiRACE1+ IS11AGEit + BiGENDER1 + B1ILLNESS1 + B1tAIDCODEit + B1tMONTHS1t + E

where:
y =Presence of any Emergency Department visit during year

D2PROJ1 =Domain 2, Project 1 (Integrated Delivery System)
D2PROJ2 =Domain 2, Project 2 (PCMH/Advanced Primary Care)
ILLNESS =Presence of a chronic illness
AIDCODE =Medicaid aid code assigned by eligibility worker for a 12-month period
MONTHS =total number of months enrolled in Medicaid in a given year
i= individual
p= Performing Provider System Setting
t =year
e =error term

Step 2: Log-Link Poisson Regression Predicting Number of ED Visits

N;pt= B0+ BµtD2PROJlt+ BptD2PROJ2t+ B;RACE;+ B;tAGE;t + B1GENDER;+B;ILLNESS; + B;tAIDCODE;t + B;tMONTHS1t + e

where:
N =Count of Emergency Department visits in year D2PROJ1 =Domain 2, Project 1 (Integrated Delivery System)
D2PROJ2 =Domain 2, Project 2 (PCMH/Advanced Primary Care)
ILLNESS =Presence of a chronic illness
AIDCODE =Medicaid aid code assigned by eligibility worker for a 12-month period
MONTHS =total number of months enrolled in Medicaid in a given year
i= individual
p =Performing Provider System Setting
t =year
e =error term

D.1 Measures:

To ground the IE´s comparison of PPS, they have identified a number of measures that have broad- ranging implications on the overall success of the DSRIP program. These measures were chosen based on their potential relevance to the overall DSRIP goals (e.g., reducing avoidable hospital use by 25 percent over five years) and the four most notable disease areas based on DSRIP project selections and the overall burden of disease in New York State. The IE will use these metrics as the basis for their comparative analysis of PPS. Additional metrics can be added based upon priorities of the NYSDOH and project resources.

Domain/Category	Measure Name	Measure* Steward	Data Source*	National Benchmark Available
Domain 2, A	Potentially avoidable ER visits	3M		MACPAC Report (preferably with Medicaid)
Domain 2, A	Potentially avoidable readmissions	3M		No
Domain 2, A	PQI suite - composite of all measures	AHRQ		No
Domain 2, A	PDI suite - composite of all measures	AHRQ		No
Domain 2, A	CAHPS measures (various)	AHRQ		Only with other state reports. There is no national CAHPS for Medicaid only.
Domain 2, B	CAHPS measures (care coordination with provider…)	AHRQ		Only with other state reports. There is no national CAHPS for Medicaid only.
Domain 3, A (BH)	All claims and MDS- based metrics (see DSRIP Strategies Menu and Metrics)	3M, NCQA, CMS	Medical Record, MDS	No
Domain 3, B (CVD)	All claims metrics listed in DSRIP Strategies Menu and Metrics	AHRQ, NCQA, CAHPS	Claims, Survey, Medical Record	No
Domain 3, C (Diabetes)	All claims metrics listed in DSRIP Strategies Menu and Metrics	AHRQ, NCQA, CAHPS	Claims, Medical Record, Survey	No
Domain 3, D (Asthma)	All claims metrics listed in DSRIP Strategies Menu and Metrics	AHRQ, NCQA	Claims	No
Domain 4	Age-adjusted preventable hospitalizations rate per 10,000-aged 18+ years		SPARCS	Yes
Domain 4	Asthma ED visit rate per 10,000		SPARCS	Yes
Domain 4	Asthma ED visit rate per 10,000 (aged 0-4)		SPARCS	No
Domain 4	Age-adjusted heart attack hospitalization rate per 10,000		SPARCS	Yes
Domain 4	Rate of hospitalizations for short-term complications of diabetes per 10,000 (aged 6-17 years)		SPARCS	No
Domain 4	Rate of hospitalizations for short-term complications of diabetes per 10,000 (aged 18+ years)		SPARCS	No

*Note: information in the above table is taken directly from the DSRIP Strategies Menu and Metrics, when completed.

D.2 Data:

Given the IE´s interest in the above variables, they have identified the following data sets that will aid in their comparative analysis:

Medicaid and Medicare Claims. These data will be the primary source of data for their analyses. These data will house the details related to many of the metrics referenced above.
SPARCS. The data related to a number of the aforementioned measures is stored in the SPARCS database. Use of these data will allow the IE to investigate key metrics and compare across PPS.
MDS (long-term care). For measures specific to long-term care (e.g., Domain 3, Behavioral Health, percent of long stay residents who have depressive symptoms).
CAHPS ©. The use of CAHPS ©data will allow the IE to learn about variations in service experience and patient satisfaction during the DSRIP program and examine the linkage between organization-level patient experience and individual-level outcomes.

D.3 Clustering to create PPS comparison groups.

The IE´s approach will begin by clustering PPS to compare those that have adopted specific domains and projects within those domains versus those that did not. More specifically, this will allow the IE to understand broadly, the impacts of PPS that elected projects addressing asthma care to those that did not. A second approach the IE will use is to cluster PPS based on their Domain 2 and Domain 3 selections. For example, several PPS selected 2.b.iv. (Care Transitions to reduce 30-day readmissions) and 3.b.i (Evidence-based strategies for disease management in high-risk/affected populations), whereas others selected one of the above or neither. The IE would cluster these groups of PPS to create comparison groups and examine specific metrics, such as readmission rates. This approach will identify the potentially most impactful Domain 2 and 3 projects.

Tests of statistical significance will be used to determine whether material differences exist between PPS. For measures available at the aggregate level for each PPS, the IE can only examine the bivariate association between the presence of a specific domain or project (or the level of implementation for that project) and the outcome variable. In that case, the IE will employ chi-square analysis to understand if differences are significant. However, in the case that outcome variables are available at the individual level (e.g., from Medicaid claims), the IE can control for patient characteristics via multivariate, multilevel modeling because they will have individuals nested via attribution in each PPS.

Then, to provide further context for these findings, the IE will use key informant interview and survey data previously gathered by the IE to contextualize "how" certain PPS have implemented project-specific plans and better understand "why" certain strategies may have been more or less effective in the context of comparative analysis.

D.4 DID.

The IE will use a Difference In Difference (DID) estimation methodology to examine specific performance measures in the time before and after the implementation of the DSRIP program comparing PPS involved in specific interventions to those that were not engaged in those projects. This estimation strategy adjusts for time-based variations in outcomes, helping determine program impacts from other phenomena. Moreover, this approach will give the IE an aggregate understanding as to whether the overall picture has changed for specific domains based on key measures of interest defined in the New York State DSRIP Strategies Menu and Metrics.

This approach will also require the use of risk-adjusted measures. This will be ideal because it would level the playing field in terms of dual-eligible and SSI patients as these individuals tend to seek care at distinct locations and are typically high-utilizers of care. Also, prior to carrying out this analysis, the IE will endeavor to identify patients and providers (hospitals and medical groups) who were not involved in any DSRIP PPS and understand the trends in use, quality, and spending over time in a separate DID analysis.

D.5 Patient-level comparisons.

The IE will examine trends within and across PPS with respect to patient- level outcomes. In particular, the IE will focus such comparisons on factors including age, sex, race, presence of chronic conditions, and mental health/substance abuse to inform their understanding of patients´ service experience and satisfaction during the DSRIP program. Such analyses will require the use of CAHPS data to examine patient satisfaction scores. However, because CAHPS scores/responses are typically not attributed to specific patients and are only available at the department, hospital, medical group, physician, or health plan level, the IE will need to examine the organizational-level CAHPS scores and their relationship to patient-level outcomes for populations attributed to the specific organization (at multiple levels). To effectively conduct such an analysis, the IE will build upon the approach set forth by Sequist, et al. (2008) to deal with the lack of individual-level outcome data linked to CAHPS scores.

Because the IE knows the Medicaid population can be vulnerable to income status changes and other reasons for disenrollment, they will determine inclusion criteria based upon months enrolled over each 12-month time period for specific measures (e.g., HEDIS-based quality measures often require 11 months of enrollment) and gaps in coverage. When considering other measures (e.g., spending and patient experience), all Medicaid members will be included for the months they were enrolled over the 36-month program and the 12-month look-back period for pre-DSRIP data.

D.6 Analytic Methods:

NYSDOH responded in November 2017 to CMS´s request to show what specific hypotheses will be tested, what data and analytic methods will be employed to address each research question, samples to employed, statistical or qualitative evidence to be examined, and how conclusions will be drawn. CMS suggested possible comparison strategies of a a.) Medicaid comparison group, b.) comparison based on differences in intensity of the intervention, c.) compare Medicaid and non-Medicaid trends in New York, and d.) compare trends in state and federal spending for the uninsured. NYSDOH responded that the IE will explore comparison groups as noted in a. and b. above, but some of the requested analysis is outside of the scope of the evaluation (contract), and that data sources are not available to address c. and d.

Clustering to create PPS comparison groups. The IE´s approach will begin by creating PPS-specific comparison groups by clustering PPS to compare those that have adopted specific domains and projects within those domains versus those that did not. More specifically, this will allow the IE to understand broadly, the impacts of PPS that elected projects addressing asthma care to those that did not. A second approach the IE will use is to cluster PPS based on their Domain 2 and Domain 3 selections. For example, several PPS selected 2.b.iv. (Care Transitions to reduce 30-day readmissions) and 3.b.i (Evidence-based strategies for disease management in high-risk/affected populations), whereas others selected one of the above or neither. The IE would cluster these groups of PPS to create comparison groups and examine specific metrics, such as readmission rates. This approach will identify the potentially most impactful Domain 2 and 3 projects.

Tests of statistical significance will be used to determine whether material differences exist between PPS.

For measures available at the aggregate level for each PPS, the IE can only examine the bivariate association between the presence of a specific domain or project (or the level of implementation for that project) and the outcome variable. In that case, the IE will employ chi-square analysis to understand if differences are significant. However, in the case that outcome variables are available at the individual level (e.g., from Medicaid claims), the IE can control for patient characteristics via multivariate, multilevel modeling because they will have individuals nested via attribution in each PPS.

DID. The IE will use a DID estimation methodology to examine specific performance measures in the time before and after the implementation of the DSRIP program comparing PPS involved in specific interventions to those that were not engaged in those projects. This estimation strategy adjusts for time- based variations in outcomes, helping determine program impacts from other phenomena. Moreover, this approach will give the IE an aggregate understanding as to whether the overall picture has changed for specific domains based on key measures of interest defined in the New York State DSRIP Strategies Menu and Metrics.

Patient-level comparisons. The IE will examine trends within and across PPS with respect to patient-level outcomes. In particular, the IE will focus such comparisons on factors including age, sex, race, presence of chronic conditions, and mental health/substance abuse to inform their understanding of patients´ service experience and satisfaction during the DSRIP program. Such analyses will require the use of CAHPS ©data to examine patient satisfaction scores. However, because CAHPS ©scores/responses are typically not attributed to specific patients and are only available at the department, hospital, medical group, physician, or health plan level, the IE will need to examine the organizational-level CAHPS ©scores and their relationship to patient-level outcomes for populations attributed to the specific organization (at multiple levels). To effectively conduct such an analysis, the IE will build upon the approach set forth by Sequist, et al. (2008) to deal with the lack of individual-level outcome data linked to CAHPS scores.

D.7 Implementation/Process Evaluation:

To assess the implementation of DSRIP initiatives, the IE will conduct a mixed method (quantitative and qualitative) evaluation. This evaluation will focus on the existing structures prior to DSRIP, process factors that shaped each program/project, program implementation strategies utilized by each site, and will complement the comparative and time series analyses. Quantitative data will be obtained through enrollment data, program data, and Medicaid claims data to determine how many participants are receiving services, whether the target populations are being reached by the initiatives, which services are being provided, the amount of services provided, and how these services are integrated. Qualitative data will be collected to extend and contextualize the quantitative measures. Sources include focus groups, semi-structured key informant interviews with PPS administrators and staff, and surveys of providers with semi-structured interview follow-up.

Quantitative and qualitative data will be used to aid in the understanding of several outcomes of interest. Outcomes of interest are based on the required RQs above. Quantitative and qualitative measures will be derived from different sources (e.g., qualitative data are based on analysis of patterns and responses via Atlas-TI, a qualitative data software program).

Outcome	Data
Quantitative:
Avoidable hospital use	3M, AHRQ, Medicaid Claims
Health care cost	Change in spending over time from Medicaid claims, compared to national Medicaid spending growth trend
Qualitative:
PPS achievement of health care transformation	Interviews with administrators, focus groups with providers, surveys with providers
Health care quality improvement	Interviews with administrators, focus groups with providers, surveys with providers
Population health improvement	Interviews with administrators, focus groups with providers, surveys with providers
Use of behavioral health care services	Interviews with administrators, focus groups with providers, surveys with providers
Successes and challenges of planning, implementation, and operation	Interviews with administrators, focus groups with providers, surveys with providers, surveys with patients

D.8 Triangulation of Data Analyses:

In the final stage of the IE´s analysis, findings from the different analyses and sources (quantitative and qualitative) will be triangulated to develop an integrated analysis. Such data will be derived from multiple sources including Medicaid and Medicare claims, SPARCS, MDS, focus groups, key informant interviews, surveys, etc. Building on the findings from the time-series analysis, qualitative analysis, and comparative analysis, the IE will synthesize the results and present interim and final summary reports that will provide insight into the effectiveness of the DSRIP program.

The IE designed the evaluation to specifically address the diversity of initiatives under the DSRIP program. The implementation/process evaluation will provide a detailed description of the programs to set the context for the time series and outcomes analyses. The IE will also address the methodological challenges of evaluating initiatives that differ in focus and target population by carefully refining the evaluation plan based on further information provided by NYSDOH. In the design, the IE selected comparison groups based on the information available at the time of the competitive procurement but will reevaluate this and other components of the evaluation based on updated and detailed information from NYSDOH. The IE will leverage the relationships and experiences that the University at Albany (UA) research team has with the Boston University School of Public Health (BUSPH) and University of Maryland School of Public Health (UMSPH) team to facilitate a responsive, comprehensive evaluation for NYSDOH that provides timely, useful information to guide future decisions.

D.9 Data Collection Plan:

Quantitative Data. All datasets are available through NYSDOH. The process of accessing the data (e.g., Medicaid claims, SPARCS) will begin immediately following the start date of the IE´s contract with NYSDOH. Once obtained, data cleaning, management, and analyses will begin and continue throughout the duration of the evaluation.

Qualitative Data. These data will look at the overall planning and implementation of DSRIP, operation of each PPS as a whole, as well as successes and challenges of projects within the PPS. The comparative analysis will be conducted jointly with the data collection activities of the IE, as to not duplicate efforts and to ensure alignment between the comparative analysis goals and the variables created via the qualitative data collection activities.

Data collection will occur annually, coinciding with each demonstration year of the DSRIP program (April 1 to March 31). It is important that the evaluation timeline follow the project timeline in order to provide appropriate and meaningful annual feedback to PPS. In addition, maintaining this timeline is important for comparative analysis based on funding, etc. Each year, the IE plans to collect information from data sources (interviews, survey with patients, and survey with providers) for each of the PPS. Focus groups will be conducted once each year over the course of data collection for each PPS. Data collection will include researchers visiting the PPS for data collection (e.g., focus groups and interviews), as well as online and telephone data collection (e.g., surveys and interviews).

D.10 Anticipated Challenges and Mitigation Strategies:

Like any empirical project of this depth, the IE is anticipating several challenges and roadblocks. Given the nature of this project, challenges may be associated with

Matching large datasets
Handling missing data.

These first two challenges are common when dealing with large and complex data sets. The IE´s subject matter experts and programmers will write algorithms based on common identifiers to link the datasets for challenge #1. To mitigate challenge #2, the IE will assess the issues as they present and determine what, if any, imputation approaches may be necessary.

Medicaid beneficiaries who frequently go in and out of covered status
Medicaid beneficiaries who move across PPS throughout the demonstration period
Initiation of interventions (DSRIP projects) as some PPS may have started earlier than others
Distinct differences in culture and outcomes between early adopters and late adopters of specific activities and/or projects

To address the challenges in #3-6, the triangulation of analyses will overcome many of these challenges. For example, with respect to #3 and 4, these issues may be mitigated by using the individual level observations as some of these variations over time will not be apparent at the PPS unit of analysis. Moreover, challenge #6 can be addressed during key informant interview with program managers and PPS leadership, as well as surveys of each PPS.

Recruiting and connecting with stakeholders for participation in data collection
Methodological challenges of evaluating PPS with different projects and strategies
Evaluating the full implementation of a five-year demonstration project, when data collection is starting in the middle of the demonstration period

Since many providers are very busy with their work, it may be a challenge to recruit participants for focus groups and/or key informants. The IE will mitigate #7 by explaining the purpose of the group to the providers and emphasizing how important their input is to the evaluation. Because providers have an interest in improvement of their projects and achieving the highest-level payment attainable, evaluation is likely to be of interest to them. In addition, the IE has designed data collection to be more flexible by incorporating a survey into the data collection methods that can be completed when it is convenient for providers, rather than having to convene providers for additional groups.

Comparisons across PPS may be challenging because all of the PPS are implementing different projects and strategies. One way to mitigate #8 is to focus on similarities between PPS and cluster PPS by projects or disease foci. For instance, all PPS are implementing behavioral health projects. Across all PPS, the IE can consider aspects of this project type, such as what strategies were successful, what challenges were specific to a strategy or were pervasive across all projects in the same domain.

Given that the DSRIP program has already started, joining mid-stream may present challenges to the IE (see #9). Ideally, program evaluations occur concurrently with the development and operation of a program. This way, data prior to the implementation of programs is compared to data during and after the implementation of programs to assess change. To mitigate this strategy, NYSDOH has insured the IE will have comprehensive access to Medicaid claims, SPARCS, and other data reported by the PPS participants. However, because the evaluation is beginning in the middle of the demonstration project, this presents a challenge for qualitative data collection focusing on the initial implementation of the DSRIP program and the individual projects. This may introduce bias when seeking to learn about the initial steps in DSRIP project development and implementation. One way to mitigate this issue is to ask respondents how things have changed since before the implementation and since the earlier stages of implementation. Retrospective data collection is not ideal but is still able to capture perceptions of change from participants. In addition, qualitative evaluation for the remaining 2.5 years of the demonstration project will be collected in real time, which will provide context and information regarding the operation and planned sustainability of projects.

Section E:

Detailed Table for Independent Evaluation of the New York DSRIP Demonstration (7/24/17):

The Independent Evaluation is built to investigate the DSRIP demonstration goals. The table below represents the three arms of the evaluation with clarification on how the arms will investigate their own RQs and hypotheses that correspond to the demonstration goals. The table is presented in this format to provide clarity of the investigation approach. Sections B, C, and D provide more detailed information regarding exact approaches the IE will pursue in the evaluation of the DSRIP demonstration goals. The summary of the evaluation questions, measures, data, and methods is below.

Research Question and Hypotheses	Outcome measures used to address the research question	Sample or population subgroups to be compared	Data Sources	Analytic Methods
Time Series Analysis
To what extent did Performing Provider Systems achieve health care system transformation, including increasing the availability of behavioral health care? Hypothesis 1: Integration of service delivery will improve under DSRIP as seen in increased availability of primary and behavioral health services for Medicaid beneficiaries. Hypothesis 2: Care coordination will increase under DSRIP as seen through increased utilization of primary care services among Medicaid beneficiaries. Hypothesis 3: Expenditures for primary care will increase under DSRIP among Medicaid beneficiaries. Hypothesis 4: Use and expenditures for outpatient behavioral health will increase under DSRIP among Medicaid beneficiaries. Hypothesis 5: Medicaid utilization and expenditures for ED and inpatient services will decrease under DSRIP. Hypothesis 6: Utilization and expenditures for ED and inpatient services among the uninsured will decrease under DSRIP.	- Use and expenditures for Primary Care Services for N1351Medicaid beneficiaries - Use and expenditures for behavioral health services for Medicaid beneficiaries - Medicaid expenditures and utilization for emergency department (ED) and inpatient services. Utilization and expenditures for ED and inpatient services for the uninsured Project specific outcomes to be selected from Att. J, pages 10-21	-All attributed DSRIP control beneficiaries as can be identified and uninsured who have ED or inpatient utilization -Intra and Inter- PPS	- Medicaid Claims Data, SPARCS data	-Descriptive Statistics over time to see trends -Comparative Interrupted Times Series Analysis & Interrupted Time Series Analysis to study the mechanics behind the trends
Did health care quality improve because of clinical improvements in the treatment of selected diseases and conditions? Hypothesis 1: Through clinical improvements under DSRIP, health care utilization in the inpatient and ED settings will decrease for all conditions examined for Medicaid beneficiaries. Hypothesis 2: Through clinical improvements under DSRIP, post discharge mortality rates will decrease for all conditions considered for Medicaid beneficiaries. Hypothesis 3: Through clinical improvements under DSRIP post discharge mortality rates will decrease for all conditions considered for the uninsured.	Hospital admissions and readmissions for: -Behavioral Health -Cardiovascular Health -Diabetes -Asthma -HIV/AIDS -Renal disease -Perinatal care -Palliative care ED utilization for: -Behavioral Health -Cardiovascular Health -Diabetes -Asthma -HIV/AIDS -Renal disease -Perinatal care -Palliative care Mortality rates post discharge from inpatient and ED settings for: -Behavioral Health -Cardiovascular Health -Diabetes -Asthma -HIV/AIDS -Renal disease	-All attributed Medicaid Beneficiaries affected by DSRIP and control Medicaid beneficiaries who can be identified -Intra- and inter- PPS analysis -Uninsured who utilize services in the inpatient or ED settings	- Medicaid Claims Data, SPARCS data, VR (death) data	-Descriptive statistics -Comparative Interrupted Time Series Analysis
RQ: Did population health improve as a result of implementation of the DSRIP initiative? Hypothesis 1: Preventive mental health and substance use services will increase under DSRIP. Hypothesis 2: Preventive HIV and STD services will increase under DSRIP. Hypothesis 3: Maternal mortality rates of Medicaid beneficiaries will decrease under DSRIP. Hypothesis 4: Infant mortality rates of Medicaid beneficiaries will decrease under DSRIP.	-Outpatient mental health or substance use services -Outpatient screening for HIV/AIDS and STDs -Outpatient services and expenditures for HIV/AIDS and STDs -Mortality rates for mothers and infants	-All attributed Medicaid beneficiaries affected by DSRIP and possible control beneficiaries -Mortality rates for Medicaid and general population	Medicaid Claims data, VR (death data)	-Descriptive statistics -Comparative Interrupted Time Series Analysis
RQ: What is the role of DSRIP in promoting behavioral health care? Hypothesis 1: Utilization and expenditures for outpatient behavioral health services for Medicaid beneficiaries will increase under DSRIP. Hypothesis 2: Utilization and expenditures for inpatient behavioral health services for Medicaid beneficiaries will decrease under DSRIP. Hypothesis 3: Utilization and expenditures for ED behavioral health services for Medicaid beneficiaries will decrease under DSRIP. Hypothesis 4: Utilization and expenditures for inpatient behavioral health services for uninsured will decrease under DSRIP. Hypothesis 5: Utilization and expenditures for ED behavioral health services for uninsured will decrease under DSRIP.	-Percentage of adults with poor mental health and substance use disorders in Medicaid and general population -Outpatient mental health and substance use services Inpatient mental health and substance use services ED visits for mental health and substance use services	-All attributed Medicaid Beneficiaries affected by DSRIP and possible control beneficiaries Uninsured in inpatient and ED settings -Inter-PPS analysis	-Medicaid Claims Data, SPARCS data, BRFSS	-Descriptive statistics -Interrupted Time Series Analysis
RQ: Was Avoidable Hospital Use Reduced because of DSRIP? Hypothesis 1: Expenditures for inpatient and ED visits will be slowed our decreased under DSRIP. Hypothesis 2: Utilization of ED and inpatient services will decrease under DSRIP. Hypothesis 3: Post-hospital death rates will decrease under DSRIP	-Potentially- preventable ED visits -Potentially- preventable hospital readmissions -Potentially- preventable hospital admissions -Post-hospital mortality rates -Various claims metrics listed in Attachment J, p. 10-21, for matching the intervention and control groups as feasible	-All attributed Medicaid beneficiaries affected by DSRIP - Inter-PPS analysis -Medicaid and Non-Medicaid subpopulations	-Claims data, SPARCS data, VR (death) data	-Descriptive statistics -Interrupted Time Series Analysis -Propensity Score matched DID for comparing Medicaid and Non-Medicaid populations
RQ: Did DSRIP reduce health disparities? Hypothesis 1: The mortality rates among racial/ethnic classes will be more equal under DSRIP. Hypothesis 2: The percentage of beneficiaries with mental health or substance use disorders will be more equal under DSRIP. Hypothesis 3: Avoidable inpatient utilization will become lower among all racial/ethnic classes under DSRIP. Hypothesis 4: Avoidable ED visits will become lower among all racial/ethnic classes under DSRIP.	-Mortality rates by racial/ethnic class -Percentage with mental health or substance use disorder by racial/ethnic class -Avoidable hospital utilization by racial/ethnic class -Avoidable ED visits by racial/ethnic class	-All attributed Medicaid Beneficiaries affected by DSRIP and possible control beneficiaries by racial/ethnic class -Adult population in NYS -Inter-PPS analysis	-Claims data, SPARCS data, BRFSS, VR (death) data	-Descriptive statistics -Interrupted Time Series Analysis
RQ: Did DSRIP reduce health costs? Hypothesis 1: Health care expenditures associated with services under DSRIP will be reduced or lowered.	-Medicaid Spending in total	-All attributed Medicaid beneficiaries affected by DSRIP and possible control beneficiaries - Inter-PPS analysis	-Medicaid Claims Data, SPARCS data	-Descriptive statistics -Interrupted Time Series Analysis
RQ: Was DSRIP cost effective in terms of NYS and federal government receiving adequate value for their investment?	-Medicaid expenditures pre- and post-DSRIP -Costs of implementing DSRIP by PPS and total over time -Costs of Medicaid program pre- DSRIP	-All attributed Medicaid beneficiaries affected by DSRIP and control beneficiaries in pre- and post- periods -Providers for PPS and non-PPS groups	Medicaid claims data, Independent Assessor information on costs of implementing DSRIP, Medicaid budget appropriations for non-DSRIP Medicaid program	Incremental Cost Effectiveness Analysis
Qualitative Analysis
What services are being provided in each project dimension?	-Categorization and itemization of services in each dimension -PPS and provider-led identification of services in each project dimension -Provider assessment of projects	-Engaged DSRIP Providers (defined as providers who are contractually involved with one or more PPS sponsored DSRIP projects) who have email based contacts with the PPS, PPS administrators	-Surveys with engaged providers (sampling frame of 2,400 providers who are engaged in projects with every PPS in all provider categories) -Key informant interviews with PPS administrators (year 1: 25 interviews with administrators year 2: 25 interviews with PPS project leads; year 3: 25 interviews with administrators).	Descriptive statistics of survey responses; Qualitative analysis of interview material
What are the most critical components of each project?	PPS and provider-led assessment of projects - Critical component case studies	-Engaged DSRIP Providers; PPS administrators (see definitions and sample frame above)	-Focus groups with engaged providers with probing for examples -Surveys with engaged providers with open ended space for examples -Key informant interviews with PPS administrators with probing for examples.	-Descriptive statistics of survey responses - Qualitative analysis of interview material, open-ended survey questions, and focus groups
Have the selected projects been implemented as designed/intended (e.g., modifications or adaptions, consistency with program design, fidelity to a model?)	-PPS and Provider-led assessment of fidelity to project operation and implementation -Identification of adaptations to design -- Challenges and successes with implementation -Utility of scale and speed items -Utility of IA assessments - Utility of other DOH milestones.	Engaged DSRIP Providers, PPS administrators	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How well does the program connect with other programs and services received by participants?	-PPS and partner- led assessment of program integration -Composite ratings of program -Examples and case studies of integration -Patient rating of coordination of care	Engaged DSRIP Providers, PPS administrators, patients	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators; patient survey	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What are the key factors in the project´s environment (e.g., the larger community, the network of services, community-based organizations) that influence project implementation?	-Categorization and itemization of factors in project environment; -PPS and Partner led assessment of those factors -Case studies and examples of those factors	Engaged DSRIP Providers, PPS administrators, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators, surveys with managed care organizations	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What barriers or challenges been encountered during service delivery?	-Categorization and itemization of barriers and challenges -PPS-led assessment of barriers and challenges -Partner-led assessment of barriers and challenges -Examples and case studies	Engaged DSRIP Providers, PPS administrators	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What strategies have been utilized? What were there outcomes?	-Success and challenges of planning, implementation and operation Categorization of strategies -PPS-led assessment of those strategies -Partner-led assessment of those strategies -Examples and case studies of those strategies	Engaged DSRIP Providers, PPS administrators	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How have other health care initiatives impacted DSRIP?	-Itemization of other health care changes -PPS and partner-led assessment of other initiatives -Case studies and examples of impacts	Engaged DSRIP Providers, PPS administrators, patients, managed care organizations	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators; surveys with managed care organizations	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How satisfied are DSRIP stakeholders with program planning?	-Rating of satisfaction with program planning from PPS, Partners, Patients, Managed Care - Case studies and examples of satisfaction and dissatisfaction with program planning - PPS achievement of healthcare transformation	Engaged DSRIP Providers, PPS administrators, patients, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators, patient surveys, surveys with managed care organization representatives	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How satisfied are DSRIP stakeholders with program implementation and operation?	-Rating of satisfaction with program implementation from PPS, Partners, Patients, Managed Care -Rating of satisfaction with program operation from stakeholders -Case studies and examples of satisfaction and dissatisfaction of both implementation and operation	Engaged DSRIP Providers, PPS administrators, patients, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators, patient surveys, surveys with managed care organization representatives	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What changes have there been to health care system overall?	-Itemization of changes to health care system over demonstration years - PPS and provider led assessment of changes to health care -Patient-led assessment of changes to l health care - Managed care-led assessment of changes to health - PPS achievement of healthcare transformation	Engaged DSRIP Providers, PPS administrators, patients, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators, patient surveys, surveys with managed care organization representatives	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What changes have there been behavioral health care?	-Itemization of changes to behavioral health care over demonstration years -PPS and provider led assessment of changes to behavioral health care - Patient-led assessment of changes to behavioral health care - Managed care-led assessment of changes to behavioral health	Engaged DSRIP Providers, PPS administrators, patients, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What changes has there been to population health?	-Itemization of changes to population health over demonstration years - Case studies of population health projects at each PPS -PPS and provider-led assessment of changes to population health -Patient-led assessment of changes to population health -Managed care-led assessment of changes to public health	Engaged DSRIP Providers, PPS administrators, patients, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators, patient surveys, surveys with managed care organization representatives	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How effective do DSRIP stakeholders perceive the projects to be? Perceive DSRIP to be overall?	-PPS achievement of healthcare transformation -Success and challenges of planning, implementation and operation -Rating of projects and DSRIP overall -PPS-led Rating of effectiveness of projects and DSRIP -Partner assessment of projects	Engaged DSRIP Providers, PPS administrators	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
Which participants seem to be benefiting the most and the least? Why?	-PPS and provider-led assessment of benefits from DSRIP -Examples of major changes; Patient assessment of care and changes to care	Engaged DSRIP Providers, PPS administrators, patients, managed care organization representatives	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators, patient surveys, surveys with managed care organization representatives	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
What recommendations are offered regarding DSRIP improvement?	-PPS and provider-led project-specific improvements, DSRIP improvements	Engaged DSRIP Providers, PPS administrators	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How has the patient experience changed?	-Patient reported assessment of experiences of changes to care -Changes to patient reported rating of provider compared to DSRIP milestones over project (e.g. VBP) -Changes to patient reported assessment of doctor communication -Changes to patient reported care coordination -Provider level assessment of changes to patient care -Changes to patient reported assessment of doctor communication	Engaged DSRIP Providers, PPS administrators; Patients who use Medicaid	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators; surveys with patients	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
How satisfied are patients with the change?	-Patient reported assessment of experiences with care -Patient reported rating of provider -Patient reported assessment of doctor communication -Patient reported care coordination; -Provider level assessment of patient satisfaction with change	Engaged DSRIP Providers, PPS administrators; Patients who use Medicaid	Focus groups with engaged providers, Surveys with engaged providers; key informant interviews with PPS administrators; surveys with patients	Descriptive statistics of survey responses; Qualitative analysis of interview material, open-ended survey questions, and focus groups
Comparative Analysis
RQ: Where does variation exist in the strategies implemented by PPSs when a similar strategy(s) were selected? Hypothesis 1: PPS that implement projects in a specific area of a domain (e.g., asthma, Domain 2) will experience comparatively better performance on related outcomes than PPS that did not implement projects in this area of a domain. Hypothesis 2: PPS that implement projects in a specific area of a domain (e.g., asthma, Domain 2) will experience comparatively better performance following the intervention.	-Potentially avoidable ER visits -Potentially avoidable readmissions -various claims metrics listed in Attachment J -Quantitative Data: claims data, SPARCS data, vital records data	-Qualitative Data: Key informant interviews, focus groups, surveys -PPS Characteristics to be used to identify comparison sub-groups: The primary characteristics that will be used to distinguish between PPS sub-groups will be project selections. Additional controls that will be included in the models may include: attribution size, number of hospitals and physicians, aggregate patient characteristics such as average age, % race, etc.		- Directed content analysis -Interrupted Time Series Design
RQ: How does the relative effectiveness of particular projects intended to produce the same outcome differ among the PPSs? Hypothesis: PPS that select certain projects for a specific domain (e.g., asthma, Domain 2) will experience comparatively better performance on related outcomes than those PPS that selected other projects.	-Potentially avoidable ER visits -Potentially avoidable readmissions -various claims metrics listed in Attachment J; project specific outcomes to be selected from Attachment J, pages 10-21	- Quantitative Data: claims data, SPARCS data, vital records data -Qualitative Data: Key informant interviews, focus groups, stakeholder surveys		- Directed content analysis -ITS
RQ: What similarities exist among those PPSs receiving (or not receiving) maximum payment based on project valuation? Hypothesis: PPS that achieve a higher percentage of their maximum payment based on project valuation will have higher overall performance on similar outcomes.	-Potentially avoidable ER visits -Potentially avoidable readmissions -various claims metrics listed in Attachment J, pages 10-21	- Quantitative Data: claims data, SPARCS data, vital records data -Qualitative Data: Key informant interviews, focus groups, surveys		- Directed content analysis -ITS
RQ: What regional differences exist between PPS´s operating in different regions of New York? RQ: What successes and challenges are associated with local resources or procedures? Hypothesis: PPS in the NYC boroughs will have made greater improvements during the demonstration period among similar outcomes than other regions of NYS.	-Potentially avoidable ER visits -Potentially avoidable readmissions -various claims metrics listed in Attachment J	- Quantitative Data: claims data, SPARCS data, vital records data -Qualitative Data: Key informant interviews, focus groups, surveys		- Directed content analysis -ITS
RQ: What patient-level differences exist in terms of service experience and satisfaction? Hypothesis 1: Older adults will have comparatively lower scores in service experience and satisfaction than younger adults on similar DSRIP- related outcomes. Hypothesis 2: Female patients will report higher-levels of satisfaction than males on similar DSRIP-related outcomes.	CAHPS Measures (various)	-Surveys of patients using CAHPS survey data		Qualitative analysis of survey data

Section F:

Timeline of Evaluation Activities:

Research Activities	2016	2017				2018				2019				2020				2021
	Q4	Q1	Q2	Q3	Q4	Q1	Q2	Q3	Q4	Q1	Q2	Q3	Q4	Q1	Q2	Q3	Q4	Q1	Q2	Q3
	DY2		DY3				DY4				DY5
Develop/ design protocols for IRB submission	X
IRB submission	X
DUA for Medicaid and other data executed	X
Schedule & perform key informant interviews			X	X	X		X	X	X		X	X	X
Schedule & perform focus groups					X	X	X	X	X	X	X	X	X
Transcribe, code, & analyze interview & focus group text					X	X	X	X	X	X	X	X	X	X	X	X	X	X
Design web-based survey		X
Administer web-based survey			X	X	X		X	X	X		X	X	X
Analyze web-based survey data					X	X			X	X			X	X	X	X	X	X
Receive Medicaid claims data			X		X		X		X		X		X		X		X
Submit request for SPARCS & other data	X
Receive SPARCS & other data				X				X				X				X
Data cleaning & preparation				X	X	X	X	X	X	X	X	X	X	X	X	X	X
Data analysis					X	X	X	X	X	X	X	X	X	X	X	X	X	X

Expanded Timeline for Evaluation Milestones:

Milestone	Target Date
Qualitative Analysis:
Finalize key informant interview guides	4/28/17
Introduce recruitment of key informant interviews to PPS staff via email blast	5/22/17
Introduce web-based survey to PPS staff and DSRIP associated providers via email	6/9/17
Begin scheduling of key informant interviews via telephone and hold key informant interviews	6/14/17
Finalize focus group guides	7/30/17
Finalize content of web-based survey for DSRIP associated providers	7/30/17
Introduce recruitment of DSRIP-associated providers for focus groups via email	8/14/17
Begin analyses of incoming data from focus groups, key informant interviews, surveys with DSRIP-associated providers, and surveys with patients	8/15/17
Complete research cycle 1 key informant interviews with PPS staff	9/22/17
Launch web-based survey for DSRIP associated providers	9/25/17
Launch focus groups at 8 PPS sites with DSRIP-associated providers	11/9/17
Finalize patient survey content	11/1/17
Launch patient survey	1/1/18
Complete cycle 1 web-based survey with PPS staff/community partners	12/21/17
Complete evaluation year 1 focus groups with DSRIP-associated providers	2/21/18
Complete cycle 1 web-based survey with patients	2/15/18
Complete analyses of cycle 1 data	2/28/18
Prepare for launch of cycle 2 research activities (key informant interviews, focus groups, and surveys)	3/15/18
Prepare for launch of cycle 3 research activities (key informant interviews, focus groups, and surveys)	3/15/19
Mixed Methods Analysis:
Meet with NYSDOH to explore data needs and uses of Salient data, etc.	5/15/17
Gain access to Medicaid, quality metric data, and other data (MDW)	11/17/17
Gain access to SPARCS data	11/28/17
Gain access to Vital Records	11/28/17
Training on MDW data for staff using data	8/9/17
Receive MDW, SPARCS, and Vital Records data (through most recent data available) via VPN	TBD
Begin establishing baseline data prior to start of DSRIP	TBD
Perform descriptive statistics on baseline data prior to start of DSRIP for all PPS	TBD
Receive data from qualitative team collected from initial key informant interviews, focus groups, and surveys	1/31/18
Begin comparative analysis examining first two demonstration years data to baseline	3/31/18
Conduct mixed methods analysis of quantitative and qualitative data for Comparative Analysis.	9/31/18 9/31/19 9/31/20
Quantitative Analysis:
Acquire access to MDW	1/31/18
Establish HCS accounts for all DSRIP evaluators	6/29/17
MDW data training	Ongoing
Gain access to MDW via VPN provided by NYSDOH (phase 2)	4/1/18
Get access to NYSDOH "sandbox" for availability of SPARCS, Vital Records, MDW, and DSRIP on same framework	1/31/18
Clean available datasets conforming to research questions	Ongoing
Obtain descriptive statistics and trend of main indicators pertaining to research questions	Ongoing
Begin time series analysis	5/15/18
Obtain preliminary results for time series RQs 1-6	12/31/18
Begin data collection for cost effectiveness analysis	1/1/19
Obtain results for time series analyses	12/31/19
Preliminary results for cost effectiveness analysis	1/1/20
Final results for time series analyses	8/30/20
Conclusions for cost effectiveness analyses	8/30/20

Section G:

Reports/Meetings:

Interim Evaluation Report - Per agreement between NYSDOH and CMS, this report must contain evaluation results from quantitative and qualitative data available for reporting and is due from the IE as follows:

Draft due to NYSDOH for review 2/15/2019
Draft due to CMS for review 3/30/2019
Final due to NYSDOH for review 5/15/2019
Final due to CMS 6/30/2019

Summative Evaluation Report - Per agreement between NYSDOH and CMS, this report must cover the entire five-year demonstration, and contain the major results and conclusions with respect to DSRIP´s operation and effectives. This will be the final report from the DSRIP evaluation. Content of the report is described in the STC above.

Preliminary report due to NYSDOH for review	5/15/2020
Preliminary report due to CMS	6/30/2020
Draft final report due to NYSDOH for review	11/15/2020
Final draft due to CMS	12/28/2020
Final due to NYSDOH for review	2/15/2021
Final due to CMS	3/28/2021

Annual Statewide Reports - For the first four years of the demonstration, annual summaries of major DSRIP evaluation results to be shared with state policymakers, PPS planners, administrators and providers in order to highlight areas of success and those in need of improvement, and to guide any needed program modifications and enhancements.

Each demonstration year´s annual report is due on March 31 of the following year. No annual statewide report is due for DY 5, as it will be replaced by the Summative Evaluation Report.
Annual PPS Reports - The IE will, on an annual basis for each of the five demonstration years, distribute results from interviews and surveys administered on the PPS level back to those PPS, with the expectation that receipt of information that is specific to their own projects will assist their ongoing quality improvement efforts.

Each demonstration year´s PPS report is due on March 31 of the following year.
Quarterly Reports - The IE will provide quarterly reports with updates to NYSDOH on data collection, analysis, and the status of written products, including activities completed during the quarter, and any difficulties encountered. These reports are due March 31, June 30, September 30, and December 30 of each year.
Meetings with CMS - The IE will, as necessary, participate in meetings/conference calls with CMS pertaining to New York´s DSRIP evaluation.
Cooperation with Federal Evaluation - The IE will cooperate with any federal evaluation activities that may be undertaken by CMS.

Section H:

Staffing Requirements:

Though there are no specific staffing requirements, the appropriateness of the staffing plan was reviewed by NYSDOH according to the competitive procurement:

Staffing is to adequately meet the project activities and deliverables. The staffing should demonstrate that project staff have appropriate training and experience in program evaluation, quantitative data analysis using large and complex data systems, survey and interview development, qualitative data collection and analysis, and report preparation. The IE provided a description of roles for each staff person, including the lead evaluator.
Job descriptions are to detail staff qualifications for the position and are to include total hours per week and estimated hours dedicated to each major task. Where possible, a resume for each staff person is to be provided.
A description of how internal management will be conducted for the DSRIP evaluation. Management oversight should be adequate to ensure integrity of products throughout the course of the DSRIP evaluation.

Appropriately staffing this project is a critical task and requires the coordination of subject matter experts and support staff from the University at Albany (UA), Boston University School of Public Health (BUSPH), and University of Maryland School of Public Health (UMSPH). The IE´s staffing plan is organized according to the activities involved in this evaluation (e.g., time series design, qualitative analysis, and comparative analysis). The internal management of evaluation activities is coordinated within each unit by the lead for that unit and the total evaluation is coordinated by the lead unit and by Diane Dewar, PhD, Principal Investigator.

The Research Foundation of the SUNY, Institute for Health System Evaluation (IHSE) will function as the coordinating entity for the entire evaluation. Dr. Dewar´s team of support staff will be comprised of four (4) individuals who will function as the adhesive that will connect the research activities going on across the evaluation. Brian Fisher, PhD, who is a Senior Research Associate within the IHSE, will also serve in a data preparation role and collaborate regularly with the Research Foundation of the SUNY, Econometrics Research Institute (ERI) and BUSPH teams. Two additional support staff will be used to manage daily activities and support the work of Dr. Fisher. This team will also ultimately be responsible for coordinating and submitting quarterly and annual reports to NYSDOH and the PPS.

UA IHSE Staffing:

Team Member	Job Description (Key Tasks)	Level of Effort (as a % of 100% or 40 hours)
Diane Dewar, PhD, Principal Investigator, Director of IHSE and Associate Professor	Oversee all project components of entire contract Coordinate and oversee data analysis and triangulation of methods and sources in comparative analysis Oversee report writing	40% in Y1-Y5
Brian Fisher, PhD, Senior Research Associate	Work with ERI in data cleaning and data gathering for time series design Serve as IT liaison Coordinate with BUSPH in comparative analysis Assist with report writing	45% in Y1-Y5
Sharleen Brittell Project Coordinator	Coordinate meetings Secure locations Organize all documentation	50% in Y1-Y5
Graduate Research Assistant TBD	Compile documents Assist in data cleaning and programming Assist in meeting and documentation organization	50% in Y1-Y5

The UA Center for Human Services Research (CHSR) will serve as the qualitative team that will oversee all activities related to surveys, key informant interviews, and focus groups. Given the labor-intensive nature of the tasks inherent in this work, a number of qualified and trained staff is needed by the IE. Paloma Luisi will maintain oversight of these activities. Moreover, support staff including qualitative researchers, survey specialists, and graduate assistants will be included in the plan to ensure that the survey and protocol design is developed appropriately, surveys are administered and analyzed in a timely manner, and that key informant interviews and focus groups are conducted, transcribed, and analyzed properly.

UA CHSR Staffing:

Team Member	Task	Level of Effort (as a % of 100% or 40 hours)
Paloma Luisi, MPH	Oversee all project components, including participant recruitment, conduct interviews and focus groups, analysis Develop interview and focus group protocols Develop surveys Conduct key informant interviews and focus groups Coordinate and oversee data analysis and triangulation of methods and sources Oversee report writing Pilot interviews and focus group protocols Develop and pilot surveys Conduct key informant telephone interviews Conduct focus groups Code qualitative data Administer surveys Analyze data Assist with report writing	100% Y1-Y5
Denise Carner, Project Staff Associate	Coordinate travel plans Assist with scheduling meetings Secure locations Organize all documentation	10% in Y1-Y5
Erin Berical, Senior Research Support Specialist	Conduct key informant phone interviews Conduct focus groups Transcribe data Code qualitative data using qualitative software Create PowerPoint slides and charts	40% in Y1-Y4
Jay Robohn, IT	Program surveys Oversee transmissions of data Ensure data security	10% in Y1-Y4
Rose Greene, MS, Center Director	Conduct staff training on focus groups and interviews Review all project reports Ensure timely submission of all required products	10% in Y1-Y5
Graduate Research Assistants (1 position)	Compile documents Schedule interviews and focus groups Coordinate travel plans Transcribe data Assist with coding Assist with report writing	50% in Y1-Y5

The UA ERI, led by Kajal Lahiri, PhD will be responsible for activities related to time series design. Dr. Lahiri will provide oversight to a graduate research assistant who will be the main support for this research. Dr. Lahiri will also plan, coordinate, and execute such analyses in coordination with Dr. Fisher.

UA ERI Staffing:

Team Member	Task	Level of Effort (as a % of 100% or 40 hours)
Kajal Lahiri, PhD, Distinguished Professor and Institute Director	Formulate, plan and execute the time series and DID analysis. Responsible for writing the relevant documents based on quantitative analysis. Coordinate with IHSE for comparative analysis and data accuracy.	20% in Y1-Y5
Soumyadeb Chatterjee Graduate Research Assistant	Compile diverse data sets Clean and organize data for statistical analysis	100% in Y1-Y5

Finally, subcontractors from BUSPH and UMSPH will be used to perform several functions. The role will be to lead the comparative analysis, and function as active, regular participants in the time series design and qualitative analysis. Christopher Louis, PhD will function as the lead for all subcontractors and manage/prioritize the activities of each subcontractor in collaboration with Dr. Dewar. Moreover, the team of subcontractors will collaborate with UA in the qualitative and time series components of this evaluation. For example, Dr. Louis, Roby, and Drainoni, will collaborate with the UA CHSR in survey and qualitative study design.

BUSPH and UMSPH Staffing:

BUSPH & UMSPH Subcontractors	Task	Level of Effort (as a % of 100% or 40 hours)
Chris Louis, PhD, Clinical Assistant Professor	Lead for all BUSPH subcontractors with responsibility for project management Participate in comparative analysis study design and planning Collaborate with qualitative research team in study and survey research design Provide leadership for support/programming staff to conduct comparative analysis study Provide support and leadership for report and content development	30% in Y1-Y5
Sally S. Bachman, PhD, Chair and Associate Professor	Lead for comparative analysis study design and planning Participate in comparative analysis study design Provide leadership for quantitative analysis	10% in Y1-Y5
TBD	Participate in comparative analysis study design Participate in time series design study design Provide oversight and subject matter expertise for quantitative analysis in all phases Provide leadership and subject matter expertise for PPS and state-level report development	10% in Y1 5% in Y3-Y5
TBD	Participate in qualitative research design and analysis of key informant interviews Provide subject matter expertise in planning of qualitative analysis Assist in report design and development	10% in Y1-Y2 5% in Y3-Y5
Mari-Lynn Drainoni, PhD, Associate Professor and Director, CIIS	Participate in qualitative research design Provide subject matter expertise in planning of qualitative analysis; specifically related to Implementation Science Assist in report design and development	10% in Y1-Y2 5% in Y3-Y5
Dylan Roby, PhD, Assistant Professor	Participate in comparative analysis study design Provide leadership for quantitative analysis Participate in qualitative research design Participate in survey research design Technical Assistance on DSRIP domains, project fidelity investigation, and claims data analysis	20% in Y1-Y2 15% in Y3-Y4 10% in Y5
Lily Chen, MD, MPH - Programmer/Data Management support	Provide support for comparative analysis Provide programming support, data management expertise Collaborate with BUSPH faculty and UA faculty to analyze data Participate in statewide and PPS report generation	20% in Y1 50% in Y2-Y4 25% in Y5

The figure below reflects the individuals who will participate in evaluation activities. This figure is organized according to:

overall evaluation project oversight and coordination,
time series analysis,
qualitative analysis and
comparative analysis.

Some individuals may participate in more than one area, and thus, their name appears multiple times.

The figure below reflects the individuals who will participate in evaluation activities.

Section I:

Limitation of the Design:

NYSDOH responded in November 2017 to CMS´s request to include limitations of the design in the evaluation design. NYSDOH responded that, as part of the STC, the IE is required to use controls and adjustments for, and reporting of, the limitations of data and their effects on results. As evaluation results are reported, this will be monitored by NYSDOH.

The evaluation will leverage data from multiple sources, including available administrative data like hospital discharge records, Medicaid claims, Medicaid enrollment, DSRIP attribution and enrollment, and hospital-supplied measures. In addition, the evaluation team will obtain quarterly PPS progress report data to capture detail about PPS implementation and phase-in of programs that are likely to affect the outcomes of interest. The evaluation team will attempt to control for important independent variables at the individual-level (i.e. age, gender, race/ethnicity, attribution length, language) as well as geographic or provider-level variation. However, the IE is aware that the number of PPSs are limited and there are not sufficient degrees of freedom to accurately estimate the independent effect of the PPS using regression analyses, so they will be able to control for individual level characteristics of those nested within each PPS. The evaluation team should also be able to examine the impact of different projects or clusters of projects (if not restricted to PPS location) to assess the impact of the DSRIP´s projects on population health outcomes and spending.

Two of the key complicating factors of the New York DSRIP design are the selection of a control group of enrollees and identification of non-participating hospital or provider sites that serve as adequate comparisons for the provider participants in the DSRIP. Due to the nature of New York´s Medicaid managed care enrollment, payer mix at participating hospitals versus non-participating hospitals, and the geographic areas where PPS have been implemented, the evaluation team will explore identification of a control group using propensity score matching from non-attributed Medicaid enrollees over the same time period, and also identifying hospitals in the state that are not participating in PPS networks. That will be a challenge and exploratory analyses will be required to assess whether either method is appropriate. In this endeavor to explore the data, the IE is far more skeptical of the ability of the non-DSRIP provider world to provide adequate comparisons. The inclusion safety net and non-safety net funding criteria for DSRIP participation explicitly limited the types of providers able to participate in DSRIP, and therefore the comparison hospitals available in the state may look fundamentally different. One of the IE-team members, Dr. Dylan Roby, was a co-PI of the California DSRIP and led efforts to identify comparison hospitals to the DSRIP hospitals in the state. Despite more than 300 general acute care facilities in the state, it was virtually impossible to identify unique hospitals to act as comparison sites due to differences in operations, size, payer mix, DSH and supplemental payments, and case mix. The IE anticipates that the same problems will occur in finding non-DSRIP hospitals to serve as adequate comparisons given the "safety net" nature and reach of the DSRIP participants. Given the broad reach of the DSRIP PPS and the inclusion criteria related to Medicaid caseload required by the DSRIP, it is difficult to find appropriate comparison hospitals that look similar to the DSRIP-participating hospitals. However, the IE considers following these exploratory steps to adequately create a control group using Medicaid data, and identify comparison sites using hospital-level data and Medicaid claims.

As stated throughout the evaluation plan, the IE will employ different analytic methods for the different sections (time series, comparative, qualitative process/implementation). In all of the sections, the IE will explore the best way to select control patients from the non-Medicaid population (when making statewide comparisons in trends in utilization, spending, etc.) using exact or propensity score matching to identify Medicaid beneficiaries in New York who were not exposed to the DSRIP and can serve as adequate controls. At the same time, the IE will explore methods for selecting similar hospital/providers from Medicaid claims data that were not instrumentally impacted by the DSRIP and can serve as comparison sites for DSRIP participating hospitals/providers. The IE will explore the non-participating sites to identify potential matches using cluster analysis based upon important variables (i.e., risk mix, payer mix, size, services, etc.) and will provide feedback to the NYSDOH and CMS on feasibility. There is a second set of comparison and control groups that will be primarily used by the comparative analysis team. Rather than attempting to draw comparisons across the state among DSRIP and non-DSRIP sites, the IE will instead draw from project selections and clustering of sites around specific goals to identify within DSRIP controls (patients) and comparisons (PPS) to analyze claims, CAHPS survey, and other data sources to understand the impact of project selections and clusters of projects on patient outcome and hospital/provider metrics. The IE will use a difference-in-differences estimation methodology to examine specific performance measures in the time before and after the implementation of the DSRIP program comparing PPSs involved in specific interventions to those that were not engaged in those projects. This estimation strategy adjusts for time-based variations in outcomes, helping determine program impacts from other phenomena. Moreover, this approach will give the IE an aggregate understanding as to whether the overall picture has changed for specific domains based on key measures of interest defined in STCs Attachment J.

This approach will also require the use of risk-adjusted measures. This will be ideal because it would level the playing field in terms of the dual-eligibles and SSI patients as these individuals tend to seek care at distinct locations and are typically-high utilizers of care. Also, prior to carrying out this analysis, the IE will endeavor to identify patients and providers (hospitals and medical groups) who were not involved in any DSRIP PPS and understand the trends in use, quality, and spending over time in a separate difference-in-differences analysis.

Patient-level Comparisons. The IE will examine trends within and across PPS with respect to patient-level outcomes using claims data and NYSDOH patient CAHPS surveys. In particular, the IE will focus such comparisons on factors including age, sex, race, presence of chronic conditions, and mental health/substance abuse to inform their understanding of patients´ service experience and satisfaction during the DSRIP program. Such analyses will require the use of CAHPS data to examine patient satisfaction scores. However, because CAHPS scores/responses are typically not attributed to specific patients and are only available at the department, hospital, medical group, physician, or health plan level, the IE will need to examine the organizational- level CAHPS scores and their relationship to patient-level outcomes for populations attributed to the specific organization (at multiple levels). To effectively conduct such an analysis, the IE will build upon the approach set forth by Sequist, et al. (2008) to deal with the lack of individual- level outcome data linked to CAHPS scores.

Because the IE knows the Medicaid population can be vulnerable to income status changes and other reasons for disenrollment, they will determine inclusion criteria based upon months enrolled over each 12-month time period for specific measures (for example, HEDIS-based quality measures often require 11 months of enrollment) and gaps in coverage. When considering other measures, like spending and patient experience, all Medicaid members will be included for the months they were enrolled over the 36-month program and the 12-month look-back period for pre-DSRIP data.

Section J:

Generalizability of Results:

NYSDOH responded in November 2017 to CMS´s request to include generalizability of results in the evaluation design. NYSDOH responded that, as part of the STC, the IE is required to discuss generalizability of results. As evaluation results are reported, this will be monitored by NYSDOH.

The comparative evaluation team, which includes Dr. Chris Louis (BU) and Dr. Dylan Roby (University of Maryland) are experts on state DSRIP interventions and the results available to date in California, New Jersey, and Texas. Dr. Roby was a co-PI of the DSRIP evaluation in California, while at UCLA. The comparative evaluation team will consider the scope, details of each DSRIP model, and explain the advantages and disadvantages of comparing other state DSRIP programs to New York´s implementation, what variation existed that might impact the overall impact of DSRIP waivers, and how findings from New York inform their understanding of DSRIP program effects overall. Evaluating the NYS DSRIP, given the multiple PPS networks, partnerships, and projects within each domain is a complex endeavor. The evaluation team will leverage both qualitative and quantitative data to inform the evaluation design by embracing the variation across and within PPS interventions and the varied goals of each. The evaluation team acknowledges that broad external factors, such as economic conditions, immigration, unemployment, Medicaid expansion decisions, and health care market factors will impact results of the DSRIP in different states, and they will address how those factors may differ and limit or help generalizability of the New York DSRIP.

Section K:

Analysis of DSRIP Dollar Allocation:

In November 2017, NYSDOH responded to CMS´s request to include analysis of the distribution of funding both across and within PPS, including a description of how DSRIP funds were used, distribution to downstream providers, which DSRIP projects received the greatest resources, and how many patients benefited from each type of project. NYSDOH answered that the DSRIP Independent Assessor and Account Support team (Public Consulting Group [PCG]) is collecting information regarding how DSRIP funds are used. However, information regarding allocation of DSRIP funds to various providers is not available throughout the DSRIP project in a standardized fashion. This has been further considered by the IE and they will explore funds flow to various providers via the publicly available PPS Implementation Progress Plans.

Additionally, NYSDOH responded in November 2017, that the requested analysis related to patients benefiting from each type of project is outside of the scope of the IE contract and a contract amendment would not be feasible with the timely submission of the Draft Interim Evaluation Report and Preliminary Summative Evaluation Report. This has been further considered by the IE and they will explore the patient engagement information publicly available from the PPS in their quarterly Implementation Progress Plans.

DSRIP Summary of Special Terms and Conditions (STC)

The DSRIP evaluation will be consistent with the specifications outlined in the DSRIP Special Terms and Conditions (STC), Sections VIII.21 through VIII.33, as summarized below:

Evaluation Requirements. The state shall engage the public in the development of its evaluation design. The evaluation design shall incorporate an interim and summative evaluation and will discuss the following requirements as they pertain to each:

The scientific rigor of the analysis;
A discussion of the goals, objectives and specific hypotheses that are to be tested;
Specific performance and outcome measures used to evaluate the demonstration´s impact;
How the analysis will support a determination of cost effectiveness;
Data strategy including sources of data, sampling methodology; and how data will be obtained;
The unique contributions and interactions of other initiatives; and
How the evaluation and reporting will develop and be maintained.

The demonstration evaluation will meet the prevailing standards of scientific and academic rigor, as appropriate and feasible for each aspect of the evaluation, including standards for the evaluation design, conduct, and interpretation and reporting of findings. The demonstration evaluation will use the best available data; use controls and adjustments for and report of the limitations of data and their effects on results; and discuss the generalizability of results.

The state shall acquire an independent entity to conduct the evaluation. The evaluation design shall discuss the state´s process for obtaining an independent entity to conduct the evaluation, including a description of the qualifications the entity must possess, how the state will assure no conflict of interest, and a budget for evaluation activities.

Evaluation Design. The Evaluation Design shall include the following core components to be approved by CMS:

Research questions and hypotheses: This includes a statement of the specific research questions and testable hypotheses that address the goals of the demonstration, including:
1. Safety net system transformation at both the system and state level;
2. Accountability for reducing avoidable hospital use and improvements in other health and public health measures at both the system and state level; and
3. Efforts to ensure sustainability of transformation of/in the managed care environment at the state level.
The research questions will be examined using appropriate comparison groups and studied in a time series.
The design will include a description of the quantitative and qualitative study design (e.g., cohort, controlled before-and-after studies, interrupted time series, case-control), including a rationale for the design selected. The discussion will include a proposed baseline and approach to comparison. The discussion will include approach to benchmarking and should consider applicability of national and state standards. The application of sensitivity analyses as appropriate shall be considered.
Performance Measures: This includes identification, for each hypothesis, of quantitative and/or qualitative process and/or outcome measures that adequately assess the effectiveness of the Demonstration in terms of cost of services and total costs of care, change in delivery of care from inpatient to outpatient, quality improvement, and transformation of incentive payment arrangements under managed care. Nationally recognized measures should be used where appropriate. Measures will be clearly stated and described, with the numerator and dominator clearly defined. To the extent possible, the state will incorporate comparisons to national data and/or measure sets. A broad set of metrics will be selected. To the extent possible, metrics will be pulled from nationally recognized metrics such as from the National Quality Forum, Center for Medicare and Medicaid Innovation, meaningful use under HIT, and the Medicaid Core Adult sets, for which there is sufficient experience and baseline population data to make the metrics a meaningful evaluation of the New York Medicaid system.
Data Collection: This discussion shall include: A description of the data sources; the frequency and timing of data collection; and the method of data collection. The following shall be considered and included as appropriate:
1. Medicaid encounter and claims data in TMSIS;
2. Enrollment data;
3. EHR data, where available;
4. Semiannual financial and other reporting data;
5. Managed care contracting data;
6. Consumer and provider surveys; and
7. Other data needed to support performance measurement
Assurances Needed to Obtain Data: The design report will discuss the state´s arrangements to assure needed data to support the evaluation design are available.
Data Analysis: This includes a detailed discussion of the method of data evaluation, including appropriate statistical methods that will allow for the effects of the Demonstration to be isolated from other initiatives occurring in the state. The level of analysis may be at the beneficiary, provider, health plan and program level, as appropriate, and shall include population and intervention-specific stratifications, for further depth and to glean potential non-equivalent effects on different sub-groups. Sensitivity analyses shall be used when appropriate. Qualitative analysis methods shall also be described, if applicable.
Timeline: This includes a timeline for evaluation-related milestones, including those related to procurement of an outside contractor, if applicable, and deliverables.
Evaluator: This includes discussion of the state´s process for obtaining an independent entity to conduct the evaluation, including a description of the qualifications that the selected entity must possess; how the state will assure no conflict of interest, and a budget for evaluation activities.

Interim Evaluation Report. The state is required to submit a draft Interim Evaluation Report 90 days following the completion of DY 4 of the demonstration. The Interim Evaluation Report shall include the same core components as identified for the Summative Evaluation Report (below) and should be in accordance with the CMS approved evaluation design. CMS will provide comments within 60 days of receipt of the draft Interim Evaluation Report. The state shall submit the final Interim Evaluation Report within 30 days after receipt of CMS´s comments.

Summative Evaluation Report. The Summative Evaluation Report will include analysis of data from DY The state is required to submit a preliminary summative report in 180 days of the expiration of the demonstration including documentation of outstanding assessments due to data lags to complete the summative evaluation. Within 360 days of the end for DY 5, the state shall submit a draft of the final summative evaluation report to CMS. CMS will provide comments on the draft within 60 days of draft receipt. The state should respond to comments and submit the Final Summative Evaluation Report within 30 days.

The Final Summative Evaluation Report shall include the following core components:

Executive Summary. This includes a concise summary of the goals of the Demonstration; the evaluation questions and hypotheses tested; and key findings including whether the evaluators find the demonstration to be budget neutral and cost effective, and policy implications.
Demonstration Description. This includes a description of the Demonstration programmatic goals and strategies, particularly how they relate to budget neutrality and cost effectiveness.
Study Design. This includes a discussion of the evaluation design employed including research questions and hypotheses; type of study design; impacted populations and stakeholders; data sources; and data collection; analysis techniques, including controls or adjustments for differences in comparison groups, controls for other interventions in the state and any sensitivity analyses, and limitations of the study.
Discussion of Findings and Conclusions. This includes a summary of the key findings and outcomes, particularly a discussion of cost effectiveness, as well as implementation successes, challenges, and lessons learned.
Policy Implications. This includes an interpretation of the conclusions; the impact of the demonstration within the health delivery system in the state; the implications for state and federal health policy; and the potential for successful demonstration strategies to be replicated in other state Medicaid programs.
Interactions with Other State Initiatives. This includes a discussion of this demonstration within an overall Medicaid context and long-range planning and includes interrelations of the demonstration with other aspects of the state´s Medicaid program, and interactions with other Medicaid waivers and other federal awards affecting service delivery, health outcomes and the cost of care under Medicaid.

State Presentations for CMS. The state will present to and participate in a discussion with CMS on the final design plan at post approval. The state will present on its interim evaluation report (described above). The state will present on its summative evaluation (described above).

Public Access. The state shall post the final approved Evaluation Design, Interim Evaluation Report, and Summative Evaluation Report on the State Medicaid website within 30 days of approval by CMS.

CMS Notification. For a period of 24 months following CMS approval of the Summative Evaluation Report, CMS will be notified prior to the public release or presentation of these reports and related journal articles, by the state, contractor or any other third-party. Prior to release of these reports, articles and other documents, CMS will be provided a copy including press materials. CMS will be given 30 days to review and comment on journal articles before they are released. CMS may choose to decline some or all of these notifications and reviews.

Electronic Submission of Reports. The state shall submit all required plans and reports using the process stipulated by CMS, if applicable.

Cooperation with Federal Evaluators. Should CMS undertake an evaluation of the demonstration or any component of the demonstration, or an evaluation that is isolating the effects of DSRIP, the state and its evaluation contractor shall cooperate fully with CMS and its contractors. This includes, but is not limited to, submitting any required data to CMS or the contractor in a timely manner and at no cost to CMS or the contractor.

Cooperation with Federal Learning Collaborative Efforts. The state will cooperate with improvement and learning collaboration efforts by CMS.

Evaluation Budget. A budget for the evaluation shall be provided with the evaluation design. It will include the total estimated cost, as well as a breakdown of estimated staff, administrative and other costs for all aspects of the evaluation such as any survey and measurement development, quantitative and qualitative data collection and cleaning, analyses, and reports generation. A justification of the costs may be required by CMS if the estimates provided do not appear to sufficiently cover the costs of the design or if CMS finds that the design is not sufficiently developed.

Deferral for Failure to Provide Summative Evaluation Reports on Time. The state agrees that when draft and final Interim and Summative Evaluation Reports are due, CMS may issue deferrals in the amount of $5,000,000 if they are not submitted on time to CMS or are found by CMS not to be consistent with the evaluation design as approved by CMS.

______________________________________________

^{1. Medically needy refers to those who have the option of spousal impoverishment budgeting, including post eligibility when it is more beneficial. Medically needy is defined as an individual who is not eligible for, or in receipt of public assistance or SSI (or the state supplement), because his/her income and/or resources are in excess of cash assistance standards, but who has insufficient income and/or resources to meet the cost of his/her necessary medical and remedial care (42 CFR §435.320 (aged), §435.322 (blind) and §435.324 (disabled)). 1}
^{2. All beneficiary protections apply to MMMC, MLTC and HARPs, unless otherwise noted in Section V 2}
^{3. Throughout these STCs, the term "Health Home," unless otherwise noted, only refers to Health Homes approved under section 1945 of the Act and consistent with approved NY Health Home state plan benefits for Health Homes SPA for IDD, Health Homes SPA for children, and/or Health Home SPA for Chronic Medical and SSI Health Home program. 3}
^{4. Note: for all children, HCBS Non-Medical Transportation (NMT) is paid through the State´s transportation broker. 4}
^{8. The terms ´visits´, ´services´, and ´qualifying services´ are used interchangeably throughout Attachment I. 8}

|top of section| |top of page|

New York 1115 Waiver Amendment: Children´s Program Evaluation Design

Revised March 20, 2020

Start Date of Demonstration Period (Children´s Design): August 2, 2019 End Date of Demonstration Period: March 31, 2021

Overview

On August 2, 2019, New York (the State) began implementation of the Children´s Design, an approved amendment to the existing 1115 waiver program. This amendment streamlined the model of care for children and youth under 21 years of age with behavioral health and home and community-based services (HCBS) needs, including the following children covered under the 1915(c) Children´s Waiver: medically fragile children, children with a behavioral health diagnosis, children with medical fragility and developmental disabilities, and children with developmental disabilities who are in foster care. The Children´s Design includes the following authority under the Medicaid Redesign Team Waiver:

Authority to mandatorily enroll the children receiving HCBS under the State´s newly consolidated 1915(c) Children´s Waiver in managed care (implemented 10/1/2019, except for foster care children, for whom it will be implemented 7/1/2020);
Authority to include current fee-for-service (FFS) 1915(c) Children´s HCBS authorized in managed care organization (MCO) benefit packages (implemented 10/1/2019);
Authorities to target eligibility to medically needy "Family of One" children (Fo1 children) who meet the risk factors, targeting criteria, and clinical eligibility standard for the Children´s Waiver but are not otherwise enrolled in the 1915(c) Children´s Waiver. Children under this authority receive Health Home Comprehensive Care Management and no HCBS, or are eligible under a non-Supplemental Security Income category;
Authority to institute an enrollment cap for Fo1 children who attain Medicaid eligibility via the 1115 waiver (implemented 8/2/2019); and
Authority to provide customized goods and services, and financial management services, under the Demonstration´s Health and Recovery Plan´s self-direction pilot for Fo1 children (will be implemented no earlier than 9/30/2020).

This amendment created a streamlined children´s model of care for children and youth under 21 years of age with behavioral health and HCBS needs. A streamlined children´s model of care will improve clinical and recovery health outcomes for children and youth with behavioral health and HCBS needs; improve timely access to services that address needs early in childhood and before they escalate and become more costly and complex in adulthood; improve access to integrated Health Homes and managed care models that integrate the delivery and care planning of behavioral health, health services, and community supports; and increase access and network capacity to deliver community-based recovery-oriented services and supports.

This amendment improves the continuity of care for transition-age youth and preserves Medicaid eligibility for many medically needy Fo1 children who would otherwise lose their Medicaid eligibility because they no longer received at least one 1915(c) service because case management is now covered outside of the 1915(c) Children´s Waiver or are eligible under a non-Supplemental Security Income eligibility category.

This document outlines an overall evaluation plan for the Children´s Design. This includes specifications for what can be included in the upcoming 1115 Waiver Interim Evaluation Report (due September 2020). Given the short time between the start of the Children´s Design and the due date for the Interim Evaluation Report, the report will include only high-level metrics and qualitative data. The overall evaluation plan also includes a more comprehensive evaluation design that can be implemented as a Summative Evaluation Report to be completed in the future with the agreement of both the State and the Centers for Medicare & Medicaid Services (CMS).

Background

The New York Medicaid Redesign Team (MRT) Demonstration (formerly known as "Partnership Plan") allows New York to implement a managed care delivery system to provide benefits to its Medicaid recipients, create efficiencies in the Medicaid program, and enable the extension of coverage to many individuals needing long-term services and supports (LTSS). The Demonstration was originally approved in 1997 to enroll most of the state´s Medicaid recipients into MCOs and it has been amended numerous times, including through the following notable amendments:

In 2010, an HCBS expansion program was added.
In 2012, an improved care coordination model of managed LTSS was added.
In 2013, modifications were approved to coordinate with the Medicaid expansion and other changes under the Affordable Care Act-including a) transitioning childless adults and parents and caretaker relatives with incomes up to, and including, 133 percent of the federal poverty limit (FPL) into State Plan coverage; and b) mandating them into managed care arrangements.
In 2014, a Delivery System Reform Incentive Payment (DSRIP) program was added.
In 2015, Health and Recovery Plans were approved to integrate physical, behavioral health, and HCBS for beneficiaries diagnosed with severe mental illness and/or substance use disorder.
In 2019, a waiver of comparability was added to exempt Medicaid Mainstream Managed Care (MMMC) enrollees from cost sharing-except for applicable pharmacy co-payments.
In 2019, CMS approved the Children´s 1115 MRT waiver amendment concurrently with the 1915(c) Children´s Waiver amendments to consolidate and streamline children´s HCBS services delivery in New York.

Larger Comprehensive Design Timeframe Limitations

One primary limitation to the evaluation methodology is the timing of the approval, the Children´s Design implementation, and the overall Demonstration evaluation.

The MRT Waiver Special Terms and Conditions (STCs) require that a draft Interim Evaluation be submitted with the 1115 renewal application. Because of the timing of the approval and implementation of the amendment,¹ New York intends to stratify existing performance metrics for children´s Health Home populations to the extent that the metrics are relevant to the larger Comprehensive Design. Other Interim Evaluation metrics for children´s populations will not include stratification for HCBS and Fo1 populations because of timing and the limited availability of data at this time. The Interim Evaluation will be analyzed and interpreted by a conflict-free independent evaluator.

MRT Waiver Evaluation Target Dates:

August 2019: January 2020: Contractor orientation, data and other preparatory steps take place.
January 2020: Dataset for evaluation period becomes available and can be provided to contractor. Review and analysis begin.
June 2020: Draft Interim Evaluation Report is submitted to the New York State Department of Health.
July 2020: State publicly posts Interim Evaluation Report.
September 2020: 1115 renewal application is submitted with Interim Evaluation Report to CMS.

Due to delays with the approval of the Children´s Design authorities, the State´s implementation timeline was significantly compressed. The State recognizes this Interim Evaluation Report has an aggressive timeline associated with the 1115 renewal application given the schedule necessary to procure an independent evaluator. Updates and risks will be shared with CMS via the quarterly 1115 report.

Other descriptive analyses directly relevant to the Children´s Design-as noted in this design-will be added to the Interim Evaluation Report in an addendum, once the design is approved by CMS. The addendum will be primarily descriptive in nature because of the timing and data limitations. Most children will be newly enrolled during the Interim Evaluation data collection period; most measures will require participation over a longer time period in order to observe outcomes. This means any quantitative data in the Interim Evaluation Report will be baseline data only. No data showing effects of the waiver will be reported at that time. The addendum will be reviewed and analyzed by a conflict-free independent evaluator.

Consumer Assessment of Healthcare Providers and Systems (CAHPS®) baseline data collected in 2018 is currently available for the Interim Evaluation Report. This survey will be re-administered in fall 2020. The Demonstration ends in 2021. Because the implementation of the Children´s Design will only have begun in fall 2019, the impact of the Children´s Design will have limited time to affect the satisfaction of members in managed care, as measured by the CAHPS®.

The Summative Evaluation Report will incorporate all metrics and the entire evaluation design plan and will be submitted consistent with Section XI of the approved STCs. The State will include any changes in the evaluation design in the 1115 quarterly report to CMS.

Research Questions and Hypotheses

The following section outlines the research questions, organized by the six goals outlined in the original Children´s Design application. Each question includes one or more hypotheses to be tested for this evaluation. Note: the approved Children´s Design is slightly different than the design proposed in the original application. Slight changes to the wording of goals and research questions have been made as a result of these changes and to reflect information available to the evaluator. For example, children served only in the FFS delivery system were omitted from the approved Demonstration, so goal six in the original application that addressed the health status of FFS HCBS-enrolled children was not incorporated into this evaluation design. Another example is that children meeting level of need criteria will not be enrolled under the Demonstration at this time.

These research questions and hypotheses reflect the complete, Summative Evaluation design. Hypotheses that can be at least partially addressed in the Interim Evaluation Report have been noted and italicized in blue print.

Goal 1: Improve the health outcomes for individuals under 21 receiving HCBS (HCBS Child/Youth) with access to the Medicaid managed care delivery system.

Research Question 1.1 (Access to Care) – What are the consequences of targeting availability of HCBS to a more narrowly-defined population than the criteria in the State Plan?
- Hypothesis 1.1.1: Targeting HCBS availability to a more narrowly-defined population will improve the health outcomes of the population most needing supports to remain in the community, as measured by a reduction in Potentially Preventable Emergency Room Visits (PPVs) and stakeholder observations about the consequences of targeting availability of HCBS to a more narrowly-defined population. [Interim Evaluation Report will include qualitative data only.]
Research Question 1.2 (Costs) – What are the per member per month (PMPM) costs of HCBS for children enrollees who receive services and how have they improved health outcomes?
- Hypothesis 1.2.1: The PMPM costs of HCBS for children enrollees will decrease because more children are eligible to receive former HCBS services under State Plan authority in an integrated managed care setting. [The Summative Evaluation Report will incorporate.]
- Hypothesis 1.2.2: The receipt of services in an integrated managed care setting will improve outcomes among Health Home/HCBS/Fo1-enrolled children, as demonstrated by a stable or decreasing percentage of the Health Home/HCBS/Fo1 population who have had an emergency room visit (AMB-CH). [The Summative Evaluation Report will incorporate.]
Research Question 1.3: To what extent are children with special needs accessing primary care providers who understand the child´s needs?
- Hypothesis 1.3.1: Parents of children with special needs will report being satisfied with primary care providers´ understanding of their children´s special conditions (CPC-CH, questions 44 and 45). [Interim Evaluation Report will include baseline data only.]
- Hypothesis 1.3.2: The number of children enrolled in MMMC/Health Home/HCBS/Fo1 who are receiving child/adolescent well-care visits will increase (W15-CH, W34-CH and AWC-CH) [Interim Evaluation Report will include baseline data for the entire MMMC population only.].

Goal 2: Improved timely access to the additional Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) benefits that address early behavioral health needs and health needs of children will improve health outcomes and long-term financial savings.

Research Question 2.1: To what extent are MMMC enrollees accessing community-based specialty services in a timely manner?
- Hypothesis: 2.1.1: MMMC child enrollees will report being satisfied with their access to community-based specialty services for children with chronic conditions (CPC-CH questions 20, 23, 26). [Interim Evaluation Report will include baseline data only.]
- Hypothesis 2.1.2: MMMC child enrollees will have improved access to behavioral health care, as demonstrated through increased use of first-line psychosocial care for children and adolescents on antipsychotics (APP-CH) [The Summative Evaluation Report will incorporate.]
Research Question 2.2 (Access to Care): To what extent are MMMC enrollees accessing community-based health care or integrated health/behavioral health care in a manner that results in improved health care outcomes?
- Hypothesis 2.2.1: MMMC child enrollees will have better follow up after hospitalizations compared to non-enrollees (FUH-CH) [Interim Evaluation Report will include baseline data only.]
- Hypothesis 2.2.2: MMMC child enrollees will have enhanced integrated health/behavioral health care, as demonstrated through increased follow up for children prescribed ADHD medication (ADD-CH). [Interim Evaluation Report will include baseline data only.]
- Hypothesis 2.2.3: MMMC child enrollees will have enhanced integrated health/behavioral health care, as demonstrated through increased metabolic monitoring for children and adolescents on antipsychotics (APM-CH)). [Interim Evaluation Report will include baseline data only.]
- Hypothesis 2.2.4: Children who have these behavioral health interventions (follow up after hospitalizations, or prescribed ADHD medication, or increased metabolic monitoring) will have lower numbers of emergency department visits and fewer hospital admissions, compared with children who do not. [The summative evaluation will incorporate.]

Goal 3: Increase appropriate access to the uniform HCBS benefit package for children who meet level of care criteria to achieve improved health outcomes while recognizing that children´s needs, including the duration, scope, and frequency of services, change over time.

Research Question 3.1: How has enrollment in HCBS increased over the length of the Demonstration?
- Hypothesis 3.1.1: Enrollment in HCBS will increase over the length of the Demonstration. [Interim Evaluation Report will include baseline data on the number of children enrolled in HCBS only.]
Research Question 3.2: What are the demographic, social, functional, and clinical characteristics of the HCBS population and do they change over time?
- Hypothesis 3.2.1: The relative number of children within each target group in the 1915(c) Children´s Waiver/1115 waiver will remain the same over time. Target groups include HCBS Serious Emotional Disturbance (SED), HCBS Medically Fragile (MF), HCBS Developmentally Disabled (DD) with Foster Care, HCBS Developmentally Disabled and Medically Fragile (DD & MF), children in foster care, children eligible under Family of One. [The Summative Evaluation Report will incorporate.]

Goal 4: Increase access to HCBS under the Demonstration and reduce the number of children being referred and diverted to more costly institutional levels of care. More children will remain in the community and be diverted from institutional services if HCBS are delivered prior to the child meeting an institutional level of care.

Research Question 4.1: To what extent has the Demonstration improved the availability of HCBS for children? What are their health outcomes, and have they been able to remain in the community?
- Hypothesis 4.1.1: Children are being admitted to institutional settings (i.e., psychiatric hospitals, general hospitals, intermediate care facilities for individuals with intellectual disabilities [ICF-ID], nursing facilities, and psychiatric residential treatment facilities [PRTFs]) less frequently and for shorter lengths of stays after the implementation of the Children´s Design. [The Summative Evaluation Report will incorporate.]
Research Question 4.2 (Costs): To what extent are HCBS cost effective? What are the PMPM costs of inpatient psychiatric services, substance use disorder (SUD) ancillary withdrawal, hospital-based detox, and emergency room services for the children´s HCBS population? Are these costs decreasing over time?
- Hypothesis 4.2.1: PMPM costs for inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox, and emergency room services for the children´s HCBS population will decrease during the Demonstration period. [The Summative Evaluation Report will incorporate.]

Goal 5: Improve access to the integrated Health Home model for all children to improve the coordination of care for children and increase access to services.

Research Question 5.1: To what extent are Health Home/HCBS enrollees accessing primary care?
- Hypothesis 5.1.1: Stakeholders will report improved care coordination. [Interim Evaluation Report will include.]
- Hypothesis 5.1.2: The number of child/adolescent immunizations will increase (CIS-CH and IMA-CH). [Interim Evaluation Report will include baseline data only.]
Research Question 5.2 (Access to Care): To the extent there is capacity for HCBS services, to what extent are Health Home/HCBS/Fo1 enrollees accessing community-based health care or integrated health/behavioral health care?
- Hypothesis 5.2.1: Health Home/HCBS/Fo1 child enrollees will have increased utilization of first-line psychosocial care for children and adolescents on antipsychotics (APP-CH). [The Summative Evaluation Report will incorporate.]
- Hypothesis 5.2.2: Rates of follow-up for Health Home/HCBS/Fo1 child enrollees prescribed ADHD medication will increase (ADD-CH). [The Summative Evaluation Report will incorporate.]
- Hypothesis 5.2.3: Metabolic monitoring for Health Home/HCBS/Fo1 child enrollees who are prescribed antipsychotics will increase (APM-CH). [The Summative Evaluation Report will incorporate.]
Research Question 5.3 (Quality of Care): Are Health Home/HCBS enrollees accessing necessary services such as health monitoring and prevention services? Are chronic health and behavioral health conditions being managed appropriately?
- Hypothesis 5.3.1: The receipt of services in an integrated managed care setting will result in an increased asthma medication ratio among Health Home/HCBS/Fo1 enrolled children (AMR-CH). [The Summative Evaluation Report will incorporate.]
- Hypothesis 5.3.2: The receipt of services in an integrated managed care setting will result in increased weight assessment and counseling for nutrition and physical activity for children/adolescents (WCC-CH). [The Summative Evaluation Report will incorporate.]
- Hypothesis 5.3.3: MMMC enrollees with chronic conditions will report that someone helped them coordinate care (CPC-CH questions 21, 24, 27, and 30). [Interim evaluation will have baseline data only.]

Goal 6: Improve continuity of care for youth as they transition into the adult Medicaid services system, specifically to the Health and Recovery Plan from the children´s Medicaid Mainstream Managed Care benefits.

Research Question 6.1: Are chronic health and behavioral health conditions for young adults (e.g., ages 21-25) who transition to adult HCBS and other Medicaid services in the Demonstration being managed appropriately?
- Hypothesis 6.1.1: Young adults transitioning to HCBS and other Medicaid services in the Demonstration have their chronic conditions properly managed, as measured by lower rates of emergency department visits (AMB-CH). [The Summative Evaluation Report will incorporate.]

Study Design

The overall evaluation of the Children´s Design will include a mixed-methods approach, using primarily quantitative performance metrics to test hypotheses. However, it will also include a qualitative component designed to describe the process of implementing the Children´s Design, including any challenges to implementation that may have an impact on expected outcomes.

Interim Evaluation Report Design: The majority of this design relates to a future, potential Summative Evaluation Report of the project. For the Interim Evaluation Report, due in September of 2020, only qualitative measures and select baselines for some high-level quantitative measures will be included.

Overall Design: Because children will not be randomly assigned to receive (or be eligible for) Demonstration services, an experimental evaluation design is not feasible. Any Summative Evaluation Report of the Children´s Design will utilize a quasi-experimental pre-post design that compares trends in performance metrics before implementation of the waiver amendment to the time period directly after. In order to attribute any observed changes over time to the amendment, a comparison group will be matched to the target population, where possible. Comparison groups will be utilized on a measure-by- measure basis when an adequate comparison pool is available. This is discussed for each measure in the Research Design table at the end of this document.

In cases where the evaluation question is either descriptive in nature or a comparison group is not available, the Summative Evaluation Report will use a descriptive time series analysis to illustrate changes over time both before and after implementation of the amendment.

The primary limitation of the design is the short time frame between implementation of the amendment and end of the waiver. Where possible, data will be reported on a rolling-year quarterly basis. However, even rolling-year quarterly data reporting limits the number of data points available for analysis. The table below shows the evaluation time frame, assuming an Interim Evaluation Report is due September 2020 and that a Summative Evaluation Report may be required by future approved STCs.

EVALUATION PERIOD	REPORT	TIME FRAME	NUMBER OF DATA POINTS
Pre-Implementation Period	Limited qualitative and high-level quantitative data for Interim Evaluation Report Summative Evaluation Report	Q1 2018 through Q2 2019 for Interim Evaluation Report Q1 2018 through Q3 2019	6 data points 7 data points
Waiver Amendment Implementation	No data available for Interim Evaluation Report
Period	Summative Evaluation Report	Q4 2019	1 data point
Post-Implementation Period	No data available for Interim Evaluation Report Summative Evaluation Report	Q1 2020 through Q1 2021	5 data points

This limited data availability means that more sophisticated time-based regression analysis, such as that used with Interrupted Time Series analysis (ITS) or difference-in-difference testing is not likely to show significant differences due to a small number of data points. To the extent that HCBS or Fo1 stratification is limited in either the sample size or availability of data pre-implementation, the state may utilize a post-only comparison group design or a descriptive interrupted time series analysis.

The short evaluation window for the project also leaves very little time for significant changes to occur and be observed. As changes to the waiver are implemented through the Children´s Design, it is expected that there will be a time delay between the official "start date" for those changes and when changes in most of the performance measures proposed here are implemented, particularly those related to health outcomes. Therefore, the main value of this design will be to provide a description of changes that happen following implementation. Any observed differences (even if not statistically significant) between Children´s Design participants and any available comparison group or national/regional benchmarks could be an indicator of positive impact. This design, however, is limited in its ability to directly attribute change to the program. This limitation is addressed somewhat through the use of comparison groups, provided that any confounding events (e.g., other health reform efforts or interventions that could also influence outcomes, in addition to the Children´s Design activities) have an equal impact on both the target population and the comparison groups.

The time frame also necessitates the use of existing performance measures to address research questions. There is not enough time between the potential approval of the evaluation design and the end of the waiver to collect new data. Performance measures-drawn from metrics already being reported by the state-that closely match the questions have been selected. However, they may not align completely, limiting the ability of the evaluation to directly address the original research questions.

When it is possible to use comparison groups, the evaluation will do so. In most cases, an attempt will be made to use children in fee for service (FFS) Medicaid to draw a comparison group. For some measures, we will rely on FFS children receiving similar services, which will help to ensure that the groups are similar. For some of the other measures, there are likely to be some differences between the FFS and the evaluation population, based on differences in program eligibility requirements. Because of this, we will use matching techniques (e.g., propensity score matching or coarsened exact matching) to ensure that the comparison group is similar to the target population on various demographic variables (age, race/ethnicity, gender, location) as well as health indicators (diagnoses, health history, level of care etc.). This selection will also consider, as discussed above, ways to ensure the use of a population that is equally likely as the target population to be affected by outside factors.. If an adequate comparison group cannot be derived for any of the measures, we will use data from similar measures in other states for comparison.

Future, summative evaluation designs will also consider the degree to which additional states have data on comparable children´s populations that can be used either in place of, or in addition to FFS comparison groups.

Most performance measures can be stratified by specific population groups (e.g., entire MMMC population versus HCBS, Health Home, and Fo1 populations). This will strengthen the design by establishing exact dates (specific to a calendar quarter) that waiver activities began for that population and examining change from time periods directly before and directly after that change. This is a technique that uses an interrupted time series design when comparison groups are not available. Limited data points or challenges in stratifying data to these populations may restrict a regression analysis of change over time, but these descriptive, observable changes will help to indicate whether changes could be attributed to the Children´s Design. Further, stratifying the populations into these specific subgroups will allow for more precise matching to the FFS population. In cases where stratifications can only be done post-intervention, the analysis will utilize a post-only comparison group design in addition to the descriptive interrupted time series for the treatment population.

Performance Measures

As stated previously, the timeframe for this evaluation is very short due to the 2021 end date of the current 1115 waiver. The State will utilize existing measures in order to address this concern. Because these measures are already routinely collected and reported by the State, there is certainty that measures will be available, even without adequate time for data collection. Specific performance measures for each research question are fully described in the research design table at the end of this document. Because these measures are standardized Medicaid core measures for children´s programs, the evaluation will include a comparison to national/regional trends for each measure.

Data Collection

There will be three main data sources for this evaluation: (1) the Medicaid Data Warehouse, (2) the Consumer Assessment of Healthcare Providers and Systems 5.0H Children with Chronic Conditions questionnaire, and (3) qualitative data collected from key informant interviews and document reviews regarding Children´s Design implementation.

Medicaid Data Warehouse

This robust dataset includes enrollment and eligibility data as well as claims and managed care encounters. Several 3M products are used to evaluate members´ clinical risk (Clinical Risk Groups) and preventable event measures, such as prevention quality indicators. These data will be used to evaluate patterns of care and health outcomes for the sub-populations of interest.

The Office of Quality and Patient Safety will calculate aggregate performance rates from the data warehouse. These rates will be provided to evaluators in February 2020 for analysis for the Interim

Evaluation Report and inclusion in the Summative Evaluation Report. Evaluators will use the data provided to calculate each identified performance measure on an annual rolling-quarter basis from the first quarter (Q1) of 2018 through the fourth quarter (Q4) of 2019 (interim reporting period) and from Q1 2018 through Q1 2021 (Summative Evaluation reporting period).

As previously mentioned, the primary limitation for this data set is the limited timeframe. However, because this is a data set from which the study measures are generally calculated, it is anticipated that there will be no challenges for the state to provide the data to the evaluator and for the measures to be calculated.

CAHPS® 5.0 CCC Questionnaire

The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) 5.0 Children with Chronic Conditions (CCC) questionnaire is a comprehensive tool designed to assess consumers´ experience with health care and health plans. CAHPS® CCC is the questionnaire that asks parents/caretakers of child health plan members about experiences with access to care, health care providers, and health plans. The CCC component of the questionnaire is a supplement to the CAHPS® Child Medicaid questionnaire, which allows health plans to identify children with chronic conditions and evaluate their experience of care. The New York State Department of Health (NYSDOH) sponsored the CAHPS® CCC survey in response to CMS Children´s Health Insurance Program Reauthorization Act requirements. Results will be used to determine variation in parent/caretaker satisfaction among the plans.

CAHPS® baseline data collected in 2018 are currently available for the Interim Evaluation Report. Aggregate data files will be provided to the evaluators for reporting at baseline. This survey will be re- administered in fall 2020.

The primary limitation of using CAHPS® data to analyze research questions related to satisfaction is that the data are not linked to specific Medicaid clients. This means that while some comparisons can be made, generally, between the enrollees of different MMMC plans as well as between Medicaid FFS and MMMC populations, the data cannot be stratified for HCBS or other Children´s Design targeted population. To the extent that the CCC version of the CAHPS® allows respondents to identify if they have special needs or chronic conditions, New York is utilizing those responses for this evaluation. Additional satisfaction data cannot be linked to any outcome measures.

There are no anticipated challenges with providing these data sets to the evaluators.

Qualitative Data

In order to understand the perceived challenges and benefits of activities under the Children´s Design, key informant interviews will be conducted with stakeholders, including advocates, plan administrators, providers, and families for any Summative Evaluation conducted. For the Interim Evaluation report, Department of Health staff will use existing advocate/stakeholder meetings to gather information on the evaluation questions. In addition, key stakeholders will be interviewed for the Interim Evaluation Report regarding the nature of the implementation. Questions will center on specific barriers to implementing the planned activities under the Children´s Design, any challenges or barriers to children accessing needed services, and perceived outcomes associated with delivered services. Both the interim and summative evaluation reports will include a complete list of Key Informant and meeting questions as an appendix. Topics for these questions include:

Eligibility determinations
Service array
Provider qualifications
Accessibility of care
Care Management
Appeals and grievances
Critical incidents, restrictive interventions, seclusion and restraint
Quality of care
Fiscal accountability

In addition, documentation will be reviewed to confirm that the Children´s Design was implemented as intended and to identify any challenges or delays to implementation.

Qualitative data is generally limited in that it only reflects the level of information available to a key informant and can reflect individuals´ biases. These limitations can be minimized by using standardized interview protocols and consistent interview techniques. These data can be used to provide important context for quantitative data analysis findings and may also be able to provide supporting evidence to demonstrate that observed changes in performance metrics could reasonably be inferred to be results from demonstration activities.

Qualitative data will be included in both the Interim Evaluation and Summative Evaluation Report drafts.

Assurances Needed to Obtain Data

This report utilizes measures already being calculated by the state. The state will provide aggregated rates to the independent evaluator for analysis.

Data Analysis

The following table outlines, for each hypothesis, the specific measures to be used, the study and comparison populations, the level of analysis, the measure steward or descriptions of the numerator and denominator, the data sources, and the analytic methods to be used.

Two quantitative analytic methods will be used, depending on the measure and availability of a comparison group. Difference-in-difference (DID) testing will be used to articulate the hypotheses "counterfactual" (what would have happened without implementation of the Children´s Design) and to estimate the effect (difference in the change over time between the target population and control group). "DID is typically used to estimate the effect of a specific intervention or treatment (such as a passage of law, enactment of policy, or large-scale program implementation) by comparing the changes in outcomes over time between a population that is enrolled in a program (the intervention group) and a population that is not (the control group).²

Descriptive time series analysis will be used to describe trends over time for measures when a comparison group is not appropriate or not available. The graphic below shows an example of a descriptive time series analysis, utilizing an interrupted time series analysis approach. As previously mentioned, there will not be enough data points to conduct a regression analysis; the graphical presentation of the data will allow for a general description in changes over time. We also propose that difference-in-difference pre-post regression analysis be conducted for those variables where a comparison group and sufficient data are available. This is indicated for the appropriate measures in the detailed research design (Table 1).

Qualitative data analysis will include a summarization of key informant interviews and any reviewed documents and an analysis of this narrative content to describe the implementation of the Children´s Design.

Figure 1: Sample of Time Series Analysis

Table 1: Detailed Research Design Table

PROCESS OR OUCOME MEASURE(S)	STUDY POPULATION & COMPARISON	LEVEL OF ANALYSIS	MEASURE STEWARD OR NUMERATOR AND DENOMINATOR	DATA SOURCE(S)	ANALYTIC METHODS
Goal 1: Improve the health outcomes for individuals under 21 receiving HCBS (HCBS Child/Youth) with access to the Medicaid managed care delivery system.
Research Question 1 (Access to Care): What are the consequences of targeting availability of HCBS to a more narrowly-defined population than the criteria in the State Plan?
Hypothesis 1.1.1: Targeting HCBS availability to a more narrowly-defined population will improve the health outcomes of the population most needing supports to remain in the community, as measured by Potentially Preventable Emergency Room Visits (PPVs) and stakeholder observations about the consequences of targeting HCBS availability to a more narrowly-defined population.
Potentially Preventable Emergency Room Visits (PPVs) Stakeholders´ (e.g., advocates, plan administrators and providers) views of the consequences of targeting availability of HCBS to a more narrowly-defined population than the criteria in the State Plan	MMMC enrollees with HCBS (children and youth) FFS matched comparison group, or metrics from similar group from another state Plan administrators and services providers	Individual Summative Evaluation Report only Demonstration – Children´s Design Interim Evaluation Report	3M N/A Qualitative data	Medicaid data warehouse Key informant interviews	Pre-post with comparison group Difference-in-difference testing, if available Narrative analysis
Research Question 1.2: (Costs) What are the PMPM costs of HCBS for children enrollees who receive services and how have they improved health outcomes?
Hypothesis 1.2.1: The PMPM costs of HCBS for children enrollees will decrease because more children are eligible to receive former HCBS services under State Plan authority in an integrated managed care setting.
PMPM cost of HCBS for enrolled children and youth	HCBS children and youth Fee-for-service HCBS costs for FFS population (estimate a PMPM) or PMPM from similar group from another state	Individual Summative Evaluation Report only	Numerator: Total HCBS cost of care Denominator: Number of member months	Medicaid data warehouse (372 data)	Pre-post with comparison group Difference-in-difference testing, if available
Hypothesis 1.2.2: The receipt of services in an integrated managed care setting will improve outcomes among HCBS-enrolled children, as demonstrated by stable or decreasing percentage of the HCBS population who have had an emergency (AMB-CH).
Emergency room visits among HCBS-enrolled children and youth	Stratify: Health Home (HH) children and youth HCBS children and youth Fo1 children and youth Comparison Group: FFS population (estimate a	Individual Summative Evaluation Report only PMPM), or PMPM from similar group from another state	AMB-CH	Medicaid data warehouse	Pre-post with comparison group Difference-in-difference testing, if available
Research Question 1.3: To what extent are children with special needs accessing primary care providers who understand the child´s needs?
Hypothesis 1.3.1: Parents of children with special needs will report being satisfied with primary care providers´ understanding of their children´s special conditions (CPC-CH questions 44 and 45).
Parent reports of satisfaction with primary care providers´ understanding of special conditions	Parents/caregivers of children needing chronic care	Individual Interim Evaluation Report—baseline data only	CPC-CH questions 44 and 45	CAHPS®	Pre-post with comparison group Difference-in-difference testing, if available	Hypothesis 1.3.2: Number of children in MMMC/HH/HCBS/Fo1 receiving child/adolescent well-care visits will increase (W15-CH, W34-CH and AWC-CH).
Child/adolescent well-care visits (W15-CH, W34-CH and AWC-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison group: FFS population, or metrics from similar group from another state	Individual Interim Evaluation Report—baseline data only for the entire MMMC population only.	NCQA #1392, NCQA #1516, NCQA	Medicaid data warehouse		Pre-post with comparison group NCQA	Difference-in-difference testing, if available
Goal 2: Improved timely access to the additional EPSDT benefits that address early behavioral health needs and health needs of children will improve health outcomes and long-term financial savings.
Research Question 2.1: To what extent are MMMC enrollees accessing community-based specialty services in a timely manner?
Hypothesis: 2.1.1: MMMC child enrollees will report being satisfied with their access to community-based specialty services for children with chronic conditions (CPC-CH).
Access to community-based specialty services for children with chronic conditions (CPC-CH).	MMMC child population where parent reported that the child received Durable Medical Equipment, therapies, or behavioral health services	Interim Evaluation Report—baseline data only	Questions 20, 23, 26 on CPC-CH	CAHPS®		Pre-post with comparison group Difference-in-difference testing, if available
Hypothesis 2.1.2 MMMC child enrollees will have improved access to behavioral health care, as demonstrated through increased use of first-line psychosocial care for children and adolescents on antipsychotics (APP-CH).
Increased use of first-line psychosocial care for children and adolescents on antipsychotics (APP-CH)	MMMC enrollees Comparison group: FFS children and youth, or metrics from similar group from another state	Individual Summative Evaluation Report only	NCQA 2801	Medicaid data warehouse		Pre-post with comparison group Difference-in-difference testing, if available
Research Question 2.2: To what extent are MMMC enrollees accessing community-based health care or integrated health/behavioral health care in a manner that results in improved health care outcomes?
Hypothesis 2.2.1: MMMC child enrollees will have improved follow up after hospitalizations (FUH-CH) compared to non-enrollees.
Follow-up after hospitalizations (FUH-CH)	MMMC enrollees Comparison group: FFS children and youth, or metrics from similar group from another state	Individual Interim Evaluation Report—baseline data only	NCQA #0576	Medicaid data warehouse		Pre-post with comparison group Difference-in-difference testing, if available
Hypothesis 2.2.2: MMMC child enrollees will have enhanced integrated health/behavioral health care, as demonstrated through increased follow-up for children prescribed ADHD medication (ADD-CH).
Follow-up for children prescribed ADHD medication (ADD-CH)	MMMC enrollees Comparison group: FFS children and youth, or metrics from similar group from another state	Individual Interim Evaluation Report -Baseline data only.	NCQA #0108	Medicaid data warehouse	Pre-post with comparison group Difference in difference testing, if available
Hypothesis 2.2.3: MMMC child enrollees will have enhanced integrated health/behavioral health care, as demonstrated through increased metabolic monitoring for children and adolescents on antipsychotics (APM-CH).
Increased metabolic monitoring for children and adolescents on antipsychotics (APM-CH)	MMMC enrollees Comparison group: FFS children and youth, or metrics from similar group from another state	Individual Interim Evaluation Report—baseline data only	NCQA #2800	Medicaid data warehouse	Pre-post with comparison group Difference-in-difference testing, if available
Hypothesis 2.2.4: Children who have these behavioral health interventions (follow up after hospitalizations, prescribed ADHD medication, increased metabolic monitoring) will have lower numbers of emergency department visits and fewer hospital admissions.
Access to Care - Follow up after hospitalizations - Prescribed ADHD medication - Increased metabolic monitoring	MMMC enrollees FFS comparison group, or metrics from similar group from another state	Individual Interim Evaluation Report—baseline data only	NCQA (various)	Medicaid data warehouse	Pre-post with comparison group Difference-in-difference testing , if available Pearson´s R correlation (with follow-up measures above)
Goal 3: Increase appropriate access to the uniform HCBS benefit package for children who meet level of care criteria to achieve improved health outcomes while recognizing that children´s needs, including the duration, scope and frequency of services, change over time.
Research Question 3.1: How has enrollment in HCBS increased over the length of the Demonstration?
Hypothesis 3.1.1: Enrollment in HCBS will increase over the length of the Demonstration.
Number of children enrolled	Children enrolled with HCBS	Individual Interim Evaluation Report—baseline data only	N/A	Medicaid data warehouse	Descriptive time series analysis
Research Question 3.2: What are the demographic, social, functional, and clinical characteristics of the HCBS population and do they change over time?
Hypothesis 3.2.1: The relative number of children within each target group in the 1915(c) Children´s Waiver/1115 waiver will remain the same over time. Target groups include HCBS Serious Emotional Disturbance (SED), HCBS Medically Fragile (MF), HCBS Developmentally Disabled (DD) with Foster Care, HCBS Developmentally Disabled and Medically Fragile (DD & MF), children in foster care, and children eligible under Family of One.
Number of children by target group	Children enrolled with HCBS	Individual Summative Evaluation Report only	N/A	Medicaid data warehouse	Descriptive time series analysis
Goal 4: Increase access to HCBS under the Demonstration and reduce the number of children being referred and diverted to more costly institutional levels of care. More children will remain in the community and be diverted from institutional services if HCBS are delivered prior to the child meeting an institutional level of care.
Research Question 4.1: To what extent has the Demonstration improved the availability of HCBS for children? What are their health outcomes, and have they been able to remain in the community?
Hypothesis 4.1.1: Children are being admitted to institutional settings (i.e., psychiatric hospitals, general hospitals, ICF-ID, nursing facilities, and PRTFs) less frequently and for shorter lengths of stays after the implementation of the Children´s Design.
Child days in institutions	Children´s HCBS population in MMMC Comparison group: FFS children receiving comparable services	Individual Summative Evaluation Report only	Numerator: total days	Medicaid data warehouse	Pre-post with comparison Difference-in-difference testing, if available
Research Question 4.2: Costs: To what extent are HCBS cost effective? What are the PMPM costs of inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox, and emergency room services for the children´s HCBS population? Are these costs decreasing over time?
Hypothesis 4.2.1: PMPM costs for inpatient psychiatric services, SUD ancillary withdrawal, hospital-based detox, and emergency room
services for the children´s HCBS population will decrease during the Demonstration period.
PMPM costs for all costs for HCBS enrollees Note: PMPM above is just HCBS costs PMPM costs for inpatient psychiatric services PMPM costs for SUD ancillary withdrawal PMPM costs for hospital-based detox PMPM costs for emergency room services	Children´s HCBS population in MMMC Comparison group: FFS children receiving comparable services Children´s HCBS population in MMMC Comparison group: Fee for service children receiving comparable services Children´s HCBS population in MMMC Comparison group: FFS children receiving comparable services Children´s HCBS population in MMMC Comparison group: FFS children receiving comparable services Children´s HCBS population in MMMC Comparison group: FFS children receiving comparable services	Individual Summative Evaluation Report only Individual Summative Evaluation Report only Individual Summative Evaluation Report only Individual Summative Evaluation Report only Individual Summative Evaluation Report only Individual Summative Evaluation Report only	Numerator: total costs for HCBS managed care enrollees Denominator: total number of member months Numerator: total costs for inpatient psychiatric services Denominator: total number of member months Numerator: total costs for SUD ancillary withdrawal Denominator: total number of member months Numerator: total costs for hospital-based detox Denominator: total number of member months Numerator: total costs for emergency room services Denominator: total number of member months	Medicaid data warehouse (372 data) Medicaid data warehouse (372 data) Medicaid data warehouse (372 data) Medicaid data warehouse (372 data) Medicaid data warehouse (372 data) Medicaid data warehouse (372 data)	Pre-post with comparison Difference-in-difference testing, if available Pre-post with comparison Difference-in-difference testing, if available Pre-post with comparison Difference-in-difference testing, if available Pre-post with comparison Difference-in-difference testing, if available Pre-post with comparison Difference-in-difference testing, if available
Goal 5: Improve access to the integrated Health Home model for all children to improve the coordination of care for children and increase access to services.
Research Question 5.1: To what extent are Health Home/HCBS enrollees accessing primary care?
Hypothesis 5.1.1: Stakeholders will report improved care coordination.
Stakeholders (advocates, plan administrators, and providers) view of access to care and care coordination	Plan administrators and services providers	Demonstration—Children´s Design Interim evaluation—qualitative data	N/A Qualitative data	Key informant interviews	Narrative analysis
Hypothesis 5.1.2: The number of child/adolescent immunizations will increase (CIS-CH and IMA-CH).
Child/adolescent immunizations (CIS-CH and IMA-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison group: FFS population, or metrics from similar group from another state	Individual Interim Evaluation— baseline data only	NCQA #0038 NCQA #1407	Medicaid data warehouse	Pre-post with comparison group Difference-in-difference testing, if available Note: To the extent that HCBS or Family of One stratification is limited, the state may utilize a post-only comparison group design or a descriptive time series analysis.
Research Question 5.2 (Access to Care): To the extent there is capacity for HCBS services, to what extent are Health Home/HCBS/Fo1 enrollees accessing community-based health care or integrated health/behavioral health care?
Hypothesis 5.2.1: HH/HCBS/Fo1 child enrollees will have increased utilization of first-line psychosocial care for children and adolescents on antipsychotics (APP-CH).
Increased use of first-line psychosocial care for children and adolescents on antipsychotics (APP-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison group: FFS children, or metrics from similar group from another state	Individual Summative Evaluation Report only	NCQA 2801	Medicaid data warehouse	Pre-post with comparison group Difference-in-difference testing, if available Note: To the extent that HCBS or Family of One stratification is limited, the state may utilize a post-only comparison group design or a descriptive time series analysis.
Hypothesis 5.2.2: Rates of follow-up for HH/HCBS/Fo1 child enrollees who are prescribed ADHD medication will increase (ADD-CH).
Follow-up for children prescribed ADHD medication (ADD-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison Group: FFS population, or metrics from similar group from another state	Individual Summative Evaluation Report only	NCQA #0108	Medicaid data warehouse		Pre-post with comparison group Difference-in-difference testing, if available Note: To the extent that HCBS or Family of One stratification is limited, the state may utilize a post-only comparison group design or a descriptive time series analysis.
Hypothesis 5.2.3: Metabolic monitoring for HH/HCBS/Fo1 child enrollees who are prescribed antipsychotics will increase (APM-CH).
Increased metabolic monitoring for children and adolescents on antipsychotics (APM-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison Group: FFS population, or metrics from similar group from another state	Individual Summative Evaluation Report only	NCQA #2800	Medicaid data warehouse		Pre-post with comparison group Difference-in-difference testing, if available Note: To the extent that HCBS or Family of One stratification is limited, the state may utilize a post-only comparison group design or a descriptive time series analysis.
Research Question 5.3 (Quality of Care): Are Health Home/HCBS enrollees accessing necessary services such as health monitoring and prevention services? Are chronic health and behavioral health conditions being managed appropriately?
Hypothesis 5.3.1: The receipt of services in an integrated managed care setting will result in an increased asthma medication ratio among HH/HCBS/Fo1 enrolled children (AMR-CH).
Asthma medication ratio (AMR-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison Group: FFS population, or metrics from similar group from another state	Individual Summative Evaluation Report only	NCQA #1800	Medicaid data warehouse			Pre-post with comparison group Difference-in-difference testing, if available Note: To the extent that HCBS or Family of One stratification is limited, the state may utilize a post-only comparison group design or a descriptive time series analysis.
Hypothesis 5.3.2: The receipt of services in an integrated managed care setting will result in increased weight assessment and counseling for nutrition and physical activity for children/adolescents (WCC-CH).
Weight assessment and counseling for nutrition and physical activity for children / adolescents (WCC-CH)	Stratify: HH children and youth HCBS children and youth Fo1 children and youth Comparison Group: FFS population, or metrics from similar group from another state	Individual Interim Evaluation - baseline data only.	NCQA #0024	Medicaid data warehouse		Pre-post with comparison group Difference-in-difference testing, if available Note: To the extent that HCBS or Family of One stratification is limited, the state may utilize a post-only comparison group design or a descriptive time series analysis.
Hypothesis 5.3.3: MMMC enrollees with chronic conditions will report that someone helped them coordinate care (CPC-CH questions 21, 24, 27, and 30).
Someone helped coordinate my child´s care (CPC-CH)	MMMC child population where parent reported that the child received DME, therapies, or behavioral health services	Interim Evaluation— baseline data only	Questions 21, 24, 27, and 30 on CPC-CH	CAHPS®			Descriptive time series (pre-post with no comparison group)
Goal 6: Improve continuity of care for youth as they transition into the adult Medicaid services system, specifically to the Health and Recovery Plan from the children´s Medicaid Mainstream Managed Care benefits.
Research Question 6.1: Are chronic health and behavioral health conditions for young adults (e.g., 21–25) who transition to adult HCBS and other Medicaid services in the Demonstration being managed appropriately?
Hypothesis 6.1.1: Young adults transitioning to HCBS and other Medicaid services in the Demonstration have their chronic conditions properly managed, as measured by lower rates of emergency department visits.
Ambulatory Care: emergency department visits (AMB-CH)	Young adults (21-25) transitioning from HCBS Young adults (21-25) in FFS Medicaid	Individual Summative Evaluation Report only	NCQA	Medicaid data warehouse		Pre-post with comparison group Difference-in-difference testing, if available

ATTACHMENT N

Behavioral Health HCBS services offered by HARPs and HIV SNPs and Individual Directed Goods and Services

Non-treatment Goods and Services Eligible for Self-Direction

Non Treament Goods and Services may include the following:

Wellness activities
- Gym/ health club membership
- Wellness coaching
- Smoking cessation tools/ education
- Dental care
- Eyeglasses/care
- Out of network health/BH/specialty services
- Family planning and sexual health education/ services
- Acupuncture/pressure
- Yoga classes/ meditation guidance
- Massage/ reiki/ shiatsu/ tai chi instruction
- Pet adoption funds, including appointments/resources related to pet health and maintenance
- Workout equipment and clothing
- Nutritional supplements and vitamins
Occupational/ skills development o Computer literacy
- Resume development
- Interview preparation
- PC/ communication technology
Personal preparation/ resources to prepare for interviews or to enhance confidence during employment, including purchase of a wardrobe or maintenance of personal hygiene (including but not limited to skin and hair care)
Resources for entrepreneurial development, including business cards, website development
Course Fees and Educational course fees and materials
Transportation
- Public transportation costs
- Car repair/ maintenance
- Bicycle and related costs
In-home/ social/ community supports
- Training and supports for daily living including cooking and nutrition classes, sequencing, time management, etc.
- Housing start-up (down payments), non-recurring housing bills or costs related to home maintenance, including furniture or air conditioner
- Groceries
- Travel to and from family or social functions, including special trips to visit family members or friends
- Meetings in the community with friends or family members at restaurants, coffee houses, or other social venues, that promote the social inclusion of the participant
- Financial contributions at social activities including church services
- Registration fees for conferences, trainings, community activities
- Membership dues in groups, societies, guilds, leagues

|top of section| |top of page|

ATTACHMENT O

Design Evaluation Questions

Specific evaluation questions for the demonstration are noted below. With CMS approval, the state may modify these questions.

Waiver Evaluation

MLTC. The evaluation questions for MLTC should include, but are not limited to:

How has enrollment in MLTC plans increased over the length of the demonstration?
What are the demographic characteristics of the MLTC population? Are they changing over time?
What are the functional and cognitive deficits of the MLTC population? Are they changing over time?
Are the statewide and plan-specific overall functional indices decreasing or staying the same overtime?
Are the average cognitive and plan specific attributes decreasing or staying the same over time?
Access to care: To what extent are enrollees able to receive timely access to personal, home care and other services such as dental care, optometry and audiology?
Quality of care: Are enrollees accessing necessary services such as flu shots and dental care?
Patient Safety: Are enrollees managing their medications? What are the fall rates and how are they changing over time?
Satisfaction: What are the levels of satisfaction with access to, and perceived timeliness and quality of network providers?
Costs: What are the PMPM costs of the population?
What are the demographics of the enrollees who are new to MLTC?

Individuals Moved from Institutional Settings to Community Settings for Long Term Services and Supports. The evaluation should include a focus specifically on the population moved from institutional settings to community settings for Long Term Services and Supports. Evaluation questions should include, but are not limited to:

What are the demographic characteristics of the HCBS Expansion population?
Satisfaction: What are the levels of satisfaction with access to, and perceived timeliness and quality of providers and services?
Costs: What are the PMPM costs of the population? How have costs changed over time?
What is the average time in short term nursing facility?
Patient Safety: Are enrollees managing their medications? What are the fall rates and how are they changing over time?

Temporary Assistance for Needy Families (TANF) Recipients.

What portion of beneficiaries enrolled through express eligibility were later deemed to be ineligible for coverage?

Twelve-Month Continuous Eligibility Period.

How many beneficiaries are receiving Medicaid coverage as a result of continuous eligibility?
What is the total member months covered under continuous eligibility?
What is the total cost of coverage for continuous eligibility?
How many of the beneficiaries covered under continuous eligibility would have been ineligible for coverage if not for the waiver?

HARP Evaluation

To what extent have the provisions of continuous eligibility affected the stability and continuity of coverage and care to adults?
How has the implementation of the Statewide Enrollment Center impacted "churning" by demonstration participants?
A quantitative and qualitative assessment of the effectiveness of the provider and enrollee education and outreach efforts, as well as plan oversight and compliance monitoring, in minimizing the impact of the transition of individuals living with HIV into mandatory Medicaid managed care.
To what extent has the mandatory enrollment of individuals living with HIV into MMC impacted their perceptions of care (fee-for-service vs. Safety Net Population/SNP vs. mainstream)?
Has the required enrollment of individuals living with HIV into Medicaid managed care (either mainstream plans or HIV SNPs) impacted quality outcomes, which in earlier studies showed that these individuals enrolled in managed care on a voluntary basis received better quality care than in fee-for-service?
An assessment of the successes and failures, along with recommendations for improvement, of the HIV SNP program.
How have the results of the Marketplace Subsidy Program for enrollment in a QHP, using childless adults who are not eligible to receive a subsidy as a comparison group, expanded access to health insurance coverage?

APPENDIX A

DSRIP Independent Assessor Firewall Policies and Procedures Summary

The primary function performed by Public Consulting Group (PCG) under the Delivery System Reform Incentive Payment (DSRIP) Program under Funding Availability Solicitation (FAS) #15649, is that of the Independent Assessor, the entity responsible for evaluating the performance of the Performing Provider Systems (PPS) throughout the life of the DSRIP Program. Additional responsibilities of the PCG team under this contract include the provision of technical assistance to the PPS, assistance in the development of the web-based portal to be used for the capture of all PPS reporting and documentation, and the review of Certificate of Public Advantage (COPA) or Accountable Care Organization (ACO) applications submitted by PPS. In fulfilling the requirements of PCG´s contract with DOH, the multiple PCG teams will have interaction, both direct and indirect, with the PPS and as such will have varying levels of access to PPS information.

PCG has developed a "Firewall Policy and Procedure Guide" to establish the policies and procedures that all members of the PCG team will be expected to follow as it relates to functions performed by each team. More specifically, the policies and procedures set forth are intended to provide a clear separation of the work being completed by the Independent Assessor and that of all other teams supporting the DSRIP project. Specifically, the firewall defines the following detail;

PCG Organizational Structure
General Firewall Policies
- Participation in Internal Meetings
- Participation in Client Meetings
- Participation in Events
- Internal Document Retention and Access
- Escalation Processes
- Identification and Notification of Breaches
- Modifications to Firewall Policies
Interaction with the Independent Assessors
- Program Support
- PPS Support
- Performance Management
- MAPP
- Health Homes
- COPA/ACO
- DSRIP Director Support

The PCG Project Leadership will work with DOH to review the content of the firewall document and the effectiveness of the policies on a regular basis as outlined in the General Firewall Policies section. Further, all PCG staff participating on the DSRIP project will be required to participate in training on the firewall policies to ensure all staff understand the content of this document and the application of the policies to the specific tasks they are expected to carry out. PCG will convey to DOH the policies and procedures that will ensure that the Independent Assessor is able to carry out its duties in assessing the performance of the PPS in an independent and unbiased fashion without any influence from outside parties, including those internal to PCG

|top of section| |top of page|

New York State Medicaid Redesign Team Section 1115(a) Medicaid Demonstration
CMS Approved: December 7, 2016 through March 31, 2021
Last Amended on December 19, 2019

_______________________________

^{1. The Children´s Design was approved in August, 2019. Many elements were not implemented until October, 2019, and some elements will not be implemented until July 2020. 1}
^{2. Columbia University Mailman School of Public Health. (n.d.). Population health methods, difference-in-difference estimation. 2}

Draft due to NYSDOH for review	2/15/2019
Draft due to CMS for review	3/30/2019
Final due to NYSDOH for review	5/15/2019
Final due to CMS	6/30/2019

Department of Health

MRT Plan Current STCs - April 16, 2020

CENTERS FOR MEDICARE & MEDICAID SERVICESWAIVER AUTHORITIES

NUMBER: 11-W-00114/2

TITLE: Medicaid Redesign Team

AWARDEE: New York State Department of Health

1. Statewideness Section 1902(a)(1)

2. Comparability Section 1902(a)(10), section 1902(a)(17)

3. Amount, Duration & Scope Section 1902(a)(10)(B)

4. Freedom of Choice Section 1902(a)(23)(A)

5. Reasonable Promptness Section 1902(a)(8)

Title XIX Requirements Not Applicable to Self-Direction Pilot Program(see Expenditure Authority 8, "Self-Direction Pilot")

6. Direct Payment to Providers Section 1902(a)(32)

CENTERS FOR MEDICARE & MEDICAID SERVICESWAIVER AUTHORITIES

NUMBER: 11-W-00114/2

TITLE: Medicaid Redesign Team

AWARDEE: New York State Department of Health

CENTERS FOR MEDICARE & MEDICAID SERVICESWAIVER AUTHORITIES

NUMBER: 11-W-00114/2

TITLE: Medicaid Redesign Team

AWARDEE: New York State Department of Health

I. PREFACE

II. PROGRAM DESCRIPTION AND OBJECTIVES

III. GENERAL PROGRAM REQUIREMENTS

IV. POPULATIONS AFFECTED BY AND ELIGIBILITY UNDER THE DEMONSTRATION

Table 1: State Plan Populations Affected by the Demonstration

V. DEMONSTRATION BENEFITS AND ENROLLMENT

VI. DELIVERY SYSTEMS

VII. DELIVERY SYSTEM REFORM PROGRAM DESCRIPTION AND OBJECTIVES

VIII. GENERAL REPORTING REQUIREMENTS

IX. GENERAL FINANCIAL REQUIREMENTS

X. MONITORING BUDGET NEUTRALITY

XI. EVALUATION OF THE DEMONSTRATION

XII. SCHEDULE OF DELIVERABLES FOR DEMONSTRATION

ATTACHMENT A

Mainstream Medicaid Managed Care (including HIV SNP and HARP) Benefits

ATTACHMENT B

ATTACHMENT C

Home and Community-Based Services Expansion Program Benefits

Settings that are not Home and Community-Based:

ATTACHMENT D

Behavioral Health Home and Community Based Services in HARPS and HIV SNPs

ATTACHMENT E

Quarterly Operational Report Format

NARRATIVE REPORT FORMAT:

Title

Section 1115 Quarterly Report

Enrollment Information:

Enrollment Counts

Total enrollment in each MCO by month

Voluntary Disenrollments:

Involuntary Disenrollments:

Enrollment Information for Specific Sub-populations:

Program Operations

ATTACHMENT F

Self-Directed Care Pilot

Overview

Background

Self-Directed Care

New York State Context

SDC Pilot Program

Pilot Evaluation

Goal 1: Implementation of a viable and effective Self-Directed Care program for HARP enrolled/HCBS eligible individuals throughout New York State

Goal 2: Improvement in recovery, health, behavioral health, social functioning and satisfaction with care for SDC participants

Goal 3: Maintenance of Medicaid cost neutrality overall and reduction of behavioral health inpatient and crisis service utilization and cost for SDC participants

Evaluation Framework

Process Evaluation

Outcome Evaluation

Evaluation Timeframe

Figure 1: SDC Pilot Logic Model

Evaluation Methods

Comparison Group (PSM Group)

Quantitative Methods

Evaluation Tools

Goal 1: Implementation of a viable and effective Self-Directed Care program for HARP enrolled/HCBS eligible individuals throughout New York State

Goal 2: Improvement in recovery, health, behavioral health, social functioning and satisfaction with care for SDC participants

Goal 3: Maintenance of Medicaid cost neutrality overall and reduction of behavioral health inpatient and crisis service utilization and cost for SDC participants

References

Appendix AData Sources

Appendix B

CENTERS FOR MEDICARE & MEDICAID SERVICES
WAIVER AUTHORITIES

Title XIX Requirements Not Applicable to Self-Direction Pilot Program
(see Expenditure Authority 8, "Self-Direction Pilot")

CENTERS FOR MEDICARE & MEDICAID SERVICES
WAIVER AUTHORITIES

CENTERS FOR MEDICARE & MEDICAID SERVICES
WAIVER AUTHORITIES

Appendix A
Data Sources

PART 3: HEALTH, WELLNESS, AND QUALITY OF LIFE
The next questions are about your health.

PART 4: BACKGROUND INFORMATION
The following information is collected to help ensure that services meet the needs of all individuals. Please do not share your name. Please check the boxes and fill in the blanks as applicable.