New York State Uniform Assessment Tool Evaluation
Prepared by Fox Systems Inc.
Wednesday, April 08, 2009
- Fox Report is also available in Portable Document Format (PDF, 143KB)
| Contents |
|---|
| Objectives and Background |
| Scope of Work |
| Evaluation |
| Findings |
| Conclusion |
| Appendix A: Domain Assessment |
Objectives and Background
New York State offers a variety of programs to meet the needs of long term care clients. These include individuals served by:
- Skilled Nursing Facilities
- Home Care
- Hospice
- Assisted Living Program
- Adult Day Health Care
- Adult Homes
- Managed Long Term Care
- Expanded In-Home Service for the Elderly Program
Over time, more than thirteen assessment tools have been developed and used by State agencies to determine the needs and service options of clients served by these programs. Many of these tools were internally developed, and some have been in use for over ten years. The resulting fragmentation of assessments has created lack of consistency across programs, duplication of effort and raises questions regarding the validity of assessments in some cases.
In order to address the complexities created by multiple programs using a variety of assessment instruments, the New York Department of Health created a work group to evaluate the feasibility of adopting a Uniform Assessment Tool (UDT) to be used across programs. The objectives of the tool adoption would include:
- Continue to meet NYS and Federal data set requirements.
- Be applicable to all LTC setting (SNF, HC, ALP, ADHC, EISEP etc).
- Be electronically collected and transmitted.
- Reduce administrative burden and streamline care coordination.
- Collect sufficient health and medical information.
- Assess functional needs and abilities through empirically tested and validated means.
- Assess availability of informal supports.
- Assure the quality, consistency and completeness of assessment and service plans.
- Ensure compliance with CMS protocols and standards
- Provide consistency and objectivity in application of State policies and procedures.
- Incorporate MDS and OASIS quality indicators.
- Assist with care planning and protective oversight.
- Facilitate payor accountability.
- Facilitate data collection and information for policy setting.
- Support personalized services reflecting the preferences and capabilities of consumers.
A mature comprehensive assessment tool is based upon the science of standardization and validation. Within mature instruments, questions and embedded assessment scales are standardized and validated. Standardization and validation will prevent the most common problems and inconsistencies in applying the assessment tool across a variety of clients and potential care settings. Most of the thirteen plus data sets and assessment tools in active use in NYS lack standardization and have not been tested for reliability or validity. Furthermore, the tools are not accompanied by decision-support algorithms that aid in LTC assessment and performance results.
As a result, the development of a care plan upon which LTC referrals can be based is often subjectively based and tied to the inconsistencies of the assessment tool used. The adverse consequences of these decisions range from limited resources being allocated ineffectively, to inappropriate use of services, to administrative duplication, to transition roadblocks across settings or regions. Further, because none of the tools are MDS based, policy makers are unable to compare characteristics of home and community based populations with the nursing home populations. Standardization of assessment is particularly important for programs directed at nursing home diversion.
An extensive review of the research literature was undertaken by the Office of Long Term Care (OLTC) to identify UDS models and templates developed by other states and countries. The objective of these studies was to identify potential tools that could bring uniformity and validity to information used in the New York LTC service system. That effort identified several potential tools available for use as Uniform Assessment collection instruments, including the InterRAI, the CARE tool being developed and piloted by CMS, the and the SAAM tool. OLTC completed an internal evaluation of these tools, and contracted with FOX systems to review and validate the internal analysis completed to date and to note any additional issues to be considered. In addition, FOX was to identify any additional tools that could enhance assessments and be considered alternatives to the tools already identified. Finally, FOX was to compare the UDT candidates with assessment instruments currently in use in the State. Following the review of current work, FOX conducted a goodness of fit and gap analysis of the New York State tools currently in use and the candidates for the UDT replacement.
|top of section| |top of page|Scope of Work
A complete and reliable home assessment is required to determine a person´s ability to remain at home and to identify the necessary supportive services that would be required to provide a safe home environment. Consistency in assessment information, processes, and professional judgment is essential to the efficient and timely documentation of the data.
To begin the process, an "Ideal" domain assessment was developed. This domain set is intended to capture the general medical, mental health, social, and functional, and personal needs of a person. In developing these Ideal domains, special consideration was given to the multiple settings in which the assessment would be utilized. FOX Systems staff worked with OLTC staff to agree upon a set of domains and high level assessment topics that would represent the full range of client needs.
Development of the ideal domain set was initiated using several sources of information:
- A literature search to capture the most current comprehensive assessment information.
- the information found in the New York State tools,
- the Centers for Medicare & Medicaid Services (CMS) tools,
- Current national and international assessment tools.
As a product of our analysis, twelve distinct domain sets were identified and a high level summary of the data to be captured was specified. The resulting domain set is presented in Appendix A.
Comparison of the ideal domain with the assessment categories used by New York staff are not straightforward because differences in the organization of tools creates apparent differences when none exist. For example, the ideal set includes a domain called Environmental Conditions that includes many of the fields included in the New York Domain called Social History. After taking into account naming differences, two domains were added to the New York Set:
- Prevention
- Personal Preferences
An important consideration in assessing a sophisticated comprehensive assessment tool is the set of uniform and validated questions measuring function, cognition, behavior, and mood that standardize results over time and across settings. Both interRAI and CARE are largely built upon validated embedded assessment scales. In evaluating the fourteen assessment tools against our ideal domain set, the existence of validated embedded assessment scales in each tool was a key consideration. Our results indicate the tools where a domain was addressed, but an embedded assessment scale was not used.
Process
Project staff identified and defined the essential domains for a person- centered home care assessment. Staff analyzed each domain to enumerate the required elements for each. As completed, the domain assessment summarizes information required to complete a home care screening and initiation of a safe and practical plan of care. The Domain summary documents twelve major domains, which are then used to conduct the detailed review of the identified assessment tools. The domains and their components are presented in Appendix A.
|top of section| |top of page|Evaluation
Project staff evaluated four assessment systems that are used nationally or internationally. Each tool was reviewed to determine the extent to which it met the essential assessment domains. The results of these evaluations are presented in Table 1: Domain Evaluation Summary. In addition, fourteen Long Term Care Assessment tools currently used by New York State were evaluated against the ideal domain set. The results of this comparison are presented in TABLE 1: DOMAIN EVALUATION SUMMARY
FOX Systems performed a gap analysis to identify weaknesses in the assessment tools when compared with our ideal set domain set. Fourteen New York State tools, three national tools, and one international tool were compared to the domain set. The domain was considered validated if the tool met the essential components of the domain. If the essential components were not met or not present, the tool was evaluated as not meeting the domain. In some cases, a domain is addressed, but the tool has significant shortcomings, the domain is considered to be partially met and a comment was logged. The strengths and barriers of the tool were also documented in the comment section.
FOX Systems did not identify any additional assessment tools that would improve the capabilities of the tools already being considered. All potential candidates that could serve as a basis for a UDT had been identified by OLTC staff and have been included in this analysis.
|top of section| |top of page|Findings
The results of the assessment tool analysis are presented in Table 1. In that table, the tools that were evaluated are listed in the first column. General use tools that are used in multiple states are listed first, and the tools currently used in New York State are listed in the second section. The domains against which each tool was evaluated are presented in the first row of each section. A more detailed description of the specific categories used to evaluate each domain is presented in Appendix A.
Each tool is scored for each domain. In Table 1, a filled in circle indicates that the tool adequately addresses our requirements for that domain. An empty circle indicates that the tool did not adequately address our requirements for the domain. If the tool partially met requirements for the domain, our comments are indicated in the table with a reference number and our comments and concerns are listed in the comments column at far right.
Each of the domains that is not met or partially met with qualifications represents a gap between the tool´s assessment capabilities and the complete set of capabilities required by New York State. The UDT selected should minimize or eliminate these gaps. However, as Table 1 indicates, no single tool will completely meet the requirements for all domains without additional data collection elements.
The highlights of the findings are as follows:
New York State Tools
Ten tools that are currently in use by New York State were evaluated. Overall, assessment tools currently in use by New York State do not cover more than seven of the twelve domains in our essential domains listing. The lack of coverage of all domains is indicated by the large number of empty circles and commented cells in the matrix. In addition, no tool reviewed would meet the definition of a mature standardized, comprehensive evaluation tool because all lack validated, embedded scales in one or more domains. Problems include:
- More than one tool is often used to evaluate the same client and certain information is collected multiple times.
- Some tools are used for eligibility and evaluation of social needs.
- Most tools did not have standardized tools or scales to use for assessment and therefore, lacked consistent data collection.
- Some tools included yes or no questions with little ability to document the underlying needs of the person.
- No tools incorporate the MDS 2, so comparisons of populations to the nursing home population are not possible.
- Some tools included long narrative sections that allow for bias or incompleteness in the evaluation. Narrative evaluations do not support automated comparisons of clients or across client groups.
General Use Tools
Four tools reviewed are intended for general use, though the CARE tool is currently in pilot status.
MDS 3.0
The Minimum Data Set (MDS 3.0) The MDS tool is mandated for use in Skilled Nursing Facilities throughout the U.S. by CMS. The 3.0 version to be released for use in 2010 was evaluated. The MDS 3 includes assessment of personal preferences, and has enhanced sensitivity of the assessments in some domains. However, even with the enhancements to version 3.0, this tool Lacked details on demographics, medication management, and home living situation information. It did Incorporate tools such as the Brief Interview for Mental Status-BIMS, Confusion Assessment Method (CAM), and Patient Health Questionnaire-9 (PHQ-9 for the mental health evaluation. These components have greater sensitivity than the MDS 2.0 tool. The 3.0 version is also is more closely aligned with selected OASIS measures than the previous version.
OASIS
OASIS (Outcome and Assessment Information Set), which is mandated by CMS for use for home health care throughout the U.S., was assessed against our domain set. The OASIS C Tool Was evaluated using the same domains presented in Table I. The OASIS C covers most of the domains adequately, and met all of the domains with the exception of preventative and personal preferences sections. It offered a complete behavioral and functional status section. However, The OASIS tool does not contain validated embedded scales, therefore is not aligned with MDS 3.0 scales. Accordingly, OASIS is not considered to be a comprehensive assessment tool that can be used on a stand-alone basis, so it cannot be considered as a UDT candidate.
CARE
CARE (Continuity Assessment Record and Evaluation) Tool-Home Care version was assessed against our domain set. This tool was developed as part of the Post Acute Care Payment Reform Demonstration under the direction of CMS. CARE is being piloted for a PAC and transition of care projects in 10 states. The tool is still in its demonstration phase and will not be implemented until at least 2010. It met all the domains with the exception of preventative and personal preferences sections. The Care tool aligns well with MDS 3.0 and contains some OASIS C items. It also incorporates standard assessment tools such as the. Brief Interview for Mental Status-BIMS and, Patient Health Questionnaire (PHQ-9) for the mental health evaluation. Because the Care tool is still in development, revisions prior to its finalization could address some of the domains that are weak or missing in the current version. CMS has expressed some interest in including domains more focused on home care populations in the final version.
InterRAI
The International Resident Assessment Instrument (InterRAI) Tool-Home Care version is administered in several US states for waiver populations, including New Jersey, Michigan, Massachusetts, and Louisiana, and is used in other countries including Canada, Europe and Japan. This tool was developed as an international collaborative to evaluate the outcomes and needs of persons served across the continuum of care settings. The Inter-RAI tool is founded on the MDS 2.0 assessment instrument, and at this point there do not appear to be plans to modify it to align with MDS 3.0. The Inter-RAI assessment tool met all of the domains with the exception of the personal preference section.
The InterRAI tool Incorporates scored measurements throughout all domains. The Cognitive Performance Scale (CPS) and Depression Rating Scale (DRS) questions are incorporated to assess the person´s mental health status. The instrumental activities of daily living (IADL) function section assess both performance and capacity of the task with a numeric scale.
The Inter-RAI tool has been extensively tested and scientifically validated, and is in use in many states in several other countries. It represents a stable and mature assessment instrument.
|top of section| |top of page|Conclusion
Upon final analysis, no tool met all of the established domain criteria. However, the national and international tools met a much higher proportion of the domain criteria, with the InterRAI tool meeting the highest number of domains and the CARE scoring as well in all domains except prevention. The MDS 3 tool was the only national and international tool to meet the domain requirements related to personal preferences, which is an important criterion for NYS UDT candidates.
None of the assessment tools used in New York State satisfactorily measured as many of the domains as either the Inter-RAI or the CARE tool. Many of the New York tools do not address all domains, because they appear to be primarily oriented toward specific data elements, and most are focused on a subset of care settings. None of the tools currently in use in New York met more than seven of the thirteen domains evaluated. Finally, no tools met the definition of a mature comprehensive assessment instrument with validated scales to standardize assessment results?
The conclusion to be drawn from this analysis is that New York State can significantly improve the scope of their assessment processes by adopting one of the national or international tools identified as a potential UDS candidate. The InterRAI and CARE tools are the strongest candidates among the assessment tools that were evaluated. The CARE tool will not be complete until 2010, which is both an advantage and a disadvantage. The disadvantage is that the tool will not be available for implementation during 2009. However, the advantage is that the tool can be modified to more closely reflect the needs of the in-home population. CMS has indicated a willingness to work with states to develop this additional capability. It may be possible to join the demonstration by 2010 to contribute an in-home care perspective to the tool.
If New York State is to adopt a uniform assessment process and integration of their long term care programs, a valid UDT will be essential. As the analysis indicates, current tools cannot include the breadth of data required for a comprehensive assessment of clients across programs. Either the Inter-RAI or the CMS CARE tool will provide a complete assessment tool set, as well as validation and reliability that are missing in current New York Tools. The choice between these leading candidates will be evaluated based on additional criteria.
|top of section| |top of page|Appendix A: Domain Assessment
Essential Components of a Comprehensive Needs Assessment, 03/24/2009
1. Demographics
- Multiple examples available
- Basics (such as address, date of birth)
Consider
- Primary language/need for interpreter
- Literacy level
Children´s assessment includes school history
- Home schooled
- Mainstream schooling history
- Special program needs
2. Disease Process / Risk Factors
Active/Current Disease Process
- Complete list including treating physician
- Does person have a neurological diagnosis?
- Does person have a history of seizures? Count of incidents from the previous year
Chronic Disease Processes with date of onset
- Diabetes
- COPD
- CHF
- Heart Disease
- Congenital Disorder
Treatments and therapies for identified conditions
Allergies with cause and reaction
Risk Factors
- Smoking
- Weight issues
- Alcohol use
- Drug use
- Shortness of breath with activity
Fall Risk Assessment
- Verify if person has had two or more falls in last year
- Verify if person has had any fall with injury in the last year
3. Mental Health
- Cognition
- Memory
- Long-term Y Short-term Y Orientation Y Person
- Place
- Time
- Comprehension/Skill with problem solving
- Complex ideas
- Abstract ideas
- Basic daily needs
- Social skills
- Mood
- Depression
- Anxiety
- Behavior
- Wandering behavior
- Verbal disruptions
- Physical aggression
- Inappropriate demonstrations
Observe for triggers for abuse or intentional injury
4. Function
- Basic activities of daily living (ADLs)
- Bathing
- Shower
- Tub bath
- Sponge bath
- Grooming
- Oral care
- Hair grooming
- Shaving
- Make-up application
- Dressing and undressing (include use of prosthesis or orthosis)
- Toileting
- Transferring
- Continence of bowel and bladder
- Feeding (eating/swallowing issues/ nutrition)
- Regular diet
- Modified consistency
- Tube/Parenteral feeding
- Hydration concerns
(Based on Katz Basic ADL Scale)
- Instrumental activities of daily living (I ADLs)
- Ability to use standard telephone or cell phone
- Shopping for groceries and clothing
- Food preparation
- Cooks for self
- Caregiver prepares meals
- Meals on Wheels
- Housework
- Laundry
- Mode of transportation
- Drives self
- Driver required
- Private car or van
- Adaptive equipment required
- Taxi
- Public transportation
- Management of medications
- Ability to handle own finances
- Sleep habits
- Number of hours per night
- Napping
- Location
- Standard bed
- Special medical bed
- Recliner
- Other
- Use of Bi-Pap or C-Pap at night
(Based on Lawton-Brody IADL scale)
5. Communication (Sensory Status)
- Evaluation of hearing, vision, and speech
- Adaptive devices and date last evaluated
- Glasses/contacts
- Hearing aids
- Dentures
- Verify the following
- Ability to hear normal conversation and electronics
- Ability to see in normal lighting
- Ability to clearly express ideas and needs
- Ability to use standard telephone or cell phone
6. Mobility
- Ambulation (include use of assistive devices)
- Cane
- Walker
- Wheelchair
- Prosthetic device
- Locomotion
- Walking
- Wheelchair use (if needed)
- Stair climbing ability (1 flight is 12-14 steps)
7. Environmental (Living Conditions)
- Status
- Lives alone
- Lives with Caregiver
- Spouse
- Parent
- Child
- Relative
- Paid caregiver
- Other
- Hours of caregiver time required
- 24/7 supervision required
- Several times a day (am and pm)
- One visit daily
- Structural Concerns
- Uneven surfaces in travel path
- Stairs
- Hallways
- Doorways
- Space limits
- No locks on exterior doors
- Poor temperature control
- Risks Observed
- No working toileting facilities
- No safety devices in specialized areas
- No cooking facilities
- No refrigeration facilities
- Inadequate lighting
- No running water
- Working communication device
- Land line telephone
- Cell phone
- Medical alert system
- Web based monitoring system
8. Medication Management
- Oral medications (including all over the counter medications)
- Name
- Dose
- Route
- Reason
- Prescriber
- Self Management Issues
- Pharmacy pick-up
- Delivered
- Mail order
- Requires weekly set-up
- Injected
- Self-administered
- Assistance required (lay or professional)
- Inhalants (Hand held inhaler or nebulizer)
- Self administered
- Assistance required (lay or professional)
9. Skin Integrity
- Is skin currently intact? (Note any skin concerns)
- Verify Braden and PUSH Tools are used
- Document history of pressure ulcers
- If a wound or ulcer is present, list the location and character of each area (grid)
- Wounds/Ulcers verification
- Number of pressure ulcers
- Number of stasis ulcer
- Number of non-healed surgical wounds
- Number of other wounds or injuries
- Verify feet condition and treatment (if applicable)
10. Pain Management
- Pain presence frequency
- None
- Rarely
- Daily
- Weekly
- Pain severity (scale 0 to 10)
- Pain effect on
- ADLs
- Sleep
- Mood
- What provides relief?
- Medication schedule
- Alternative methods tried
11. Prevention Services
- Verify dates of
- Last complete physical
- Flu immunization
- Pneumococcal immunization
- Tetanus immunization
- Mammogram
- Colonoscopy
12. Personal Preferences Section
- Measurement
Verify a section is available for preferences such as- Chooses to work, if able
- Choosing clothes to wear
- Caring for personal belongings
- Reading books, newspapers, or magazines
- Bathing preference (tub, shower, or sponge)
- Listening to music
- Receiving shower
- Being around animals such as pets
- Keeping up with the news
- Doing things with groups of people
- Snacks between meals
- Participating in favorite activities
- Staying up past 8:00 p.m.
- Spending time away from the house
- Spending time with family or significant other
- Spending time outdoors
- Involvement in care discussions
- Participating in religious activities or practices
- Place to lock personal belongings
- Use of telephone in private
Follow Us