Gain unique access to behavior change leaders with our webcast series: ASAM Criteria Unplugged and Other Tales from the Field. Hear from leading experts in the field of substance use treatment, including Dr. David Mee-Lee, Dr. William R. Miller, and many more.
Register here for our next ASAM Criteria Unplugged session and watch past episodes on-demand on the Fidelity Platform, an online training hub designed for facilitators. Be the first to learn about upcoming webinars and open-registration events by signing up for our newsletter.
In this recent webinar, Instrument Confusion: Expecting the Wrong Screen or Assessment to Achieve the Correct Objective, Dr. Norman Hoffmann delves into the critical distinction between screening and diagnostic tools in behavioral health assessments. Check out the highlight video below:
About special guest, Dr. Norman Hoffmann
Norman G. Hoffmann, PhD is a clinical psychologist who is a nationally and internationally recognized expert in the clinical assessment of mental health and substance use disorders and in the evaluation of behavioral health programs. He has worked with private organizations and governmental agencies in a variety of states and countries. Dr. Hoffmann developed assessment instruments used throughout the United States and Canada as well as in several European countries. Instruments include brief screens, detailed diagnostic interviews, and treatment planning aids. He was the founder and science officer for the largest independent outcome evaluation system in the US and has authored or co-authored more than 220 publications.
Sitting down with The Change Companies Chief Program Officer Valerie Bagley and Train for Change Senior National Director Scott Boyles, Dr. Hoffmann addressing common challenges faced by clinicians and organizations in using the right tools for accurate assessments.
Jump ahead to questions covered in this blog:
- What is the key difference between screening and diagnostic assessments?
- Why is it problematic to use screening tools in place of full diagnostic assessments?
- What do sensitivity and specificity mean in the context of screening accuracy? How do sensitivity and specificity relate to each other?
- What factors can influence the accuracy of a screening?
- How can clinicians avoid becoming overly reliant on assessment tools?
- Can you provide more information on the UNCOPE as an effective screening tool for substance use disorders? How can people access the UNCOPE screening tool?
- What are the most common mental health conditions identified in screenings?
- What are the options for creating or using assessment tools?
- How can behavioral health assessments be leveraged within the criminal justice system to coordinate care between correctional facilities and community providers?
Q: What is the key difference between screening and diagnostic assessments?
Dr. Hoffmann: Screening is a preliminary process used to determine if a more detailed assessment is required for a specific condition. It’s often quick, can be administered by non-clinicians, and is designed to flag potential issues. Screens can be oral or written. A diagnostic assessment, on the other hand, is more comprehensive and definitive. It involves deeper evaluation, usually performed by clinicians, and is necessary for treatment planning. It’s important to note that using a screen where a diagnostic assessment is required can lead to incomplete or inaccurate treatment decisions.
Note: No single tool can do everything. There is no such thing as a universal assessment tool. Screening and diagnostic assessments are essentially mutually exclusive tasks. Screening is short, fast, and diagnostic assessments are definitive and comprehensive.
Q: Why is it problematic to use screening tools in place of full diagnostic assessments?
Dr. Hoffmann: Using a screen for conditions that have already been flagged, such as using a substance use disorder screen for someone already in a substance use treatment program, is redundant and inappropriate. A screen is not meant to replace a diagnostic evaluation. It only identifies risks, whereas an assessment confirms the presence and severity of a disorder and informs the treatment plan. For example, you wouldn’t make a life-altering decision based solely on a mammogram; you’d need a biopsy to confirm any diagnosis.
Q: What do sensitivity and specificity mean in the context of screening accuracy? How do sensitivity and specificity relate to each other?
Dr. Hoffmann: Sensitivity and specificity are two key measures to evaluate the accuracy of a screening tool. Sensitivity refers to the proportion of people with a condition that the screen correctly identifies as having the condition. The main error associated with sensitivity is a false negative—this happens when someone with the condition is missed by the screen.
Specificity, on the other hand, is the proportion of people without the condition who are correctly identified as not having it. The error here is a false positive, which occurs when a screen incorrectly indicates that someone has a condition when, in fact, they do not. So, the two types of errors we encounter are either false negatives (missing someone with the condition) or false positives (incorrectly identifying someone as having the condition).
There’s an inverse relationship between sensitivity and specificity. If you aim to reduce false positives, you’ll end up tolerating more false negatives. Conversely, if you want to eliminate false negatives, you'll have to accept more false positives. In most cases, the goal is to strike a balance between the two. When working with screens, you can often adjust the "cut score" to control the level of sensitivity and specificity, depending on the relative cost of making errors in either direction.
Q: What factors can influence the accuracy of a screening?
Dr. Hoffmann: Several factors can influence the accuracy of screening, including:
- Who is asking the questions? In correctional settings, for example, having a sworn officer in uniform conduct the screening is not ideal, as arrestees often lie to law enforcement.
- Timing: The timing of the screening matters. Doing it too early in a mental health clinic might distract from the primary complaint. Conversely, conducting it too late, when the person is leaving, limits your ability to intervene.
- Competing activities: Screening during busy periods, such as the booking process in a correctional setting, is not recommended. It’s better to wait until the individual has settled in. Similarly, in residential treatment settings, it may be better to wait until later in the assessment process to screen for conditions.
- Setting: The environment is crucial. The individual needs to feel safe and able to be honest. Privacy is essential for the screening to be accurate.
Q: How can clinicians avoid becoming overly reliant on assessment tools?
Dr. Hoffmann: Instruments are just tools – they should serve the clinician, not the other way around. Ultimately, it’s up to human beings to make informed decisions based on the data provided by these tools. Tools cannot replace clinical judgment, and clinicians must be cautious about becoming servants to the instruments they use. This applies to both manual assessments and digital tools, including those using artificial intelligence.
Q: Can you provide more information on the UNCOPE as an effective screening tool for substance use disorders? How can people access the UNCOPE screening tool?
Dr. Hoffmann: The UNCOPE is a six-item universal screen that has been validated for substance use disorders. It covers key diagnostic criteria for various substances and has been proven to be highly sensitive and specific, particularly when the cut score is set at three or more to indicate a serious substance use disorder. This score corresponds to two or more positive criteria from the DSM, which makes the specificity very high. If a person has three or more positive indications on the UNCOPE, they are likely to have even more positive criteria, such as five, six, or more. This tool works well for both adolescents and adults and can quickly flag individuals who may need further diagnostic evaluation for substance use.
There are fewer false positives with substances like cocaine, opiates, and methamphetamines. Interestingly, alcohol and cannabis tend to show more individuals in the mild to moderate range of substance use disorders. In these cases, the DSM findings are often lower than with the more addictive substances.
The UNCOPE is readily available with a quick Google search, or you can access it through resources like Children and Family Futures. They have an excellent video that demonstrates how to use the UNCOPE in a mock interview setting, focusing on child neglect cases. In the video, a freeform discussion takes place, and the interviewer checks off the UNCOPE items as they come up in conversation.
Q: What are the most common mental health conditions identified in screenings?
Dr. Hoffmann: In correctional populations, we frequently encounter major depression and PTSD as the most prevalent conditions. There are effective screening tools for these disorders, including a simple two-item PTSD screen and a two-item depression screen. These brief screens provide a quick way to flag individuals who may need further evaluation for mental health conditions.
Q: What are the options for creating or using assessment tools?
Dr. Hoffmann: You have two main options: you can either build your own assessment tool or buy a pre-existing commercial system. The advantage of building your own is that you can tailor it specifically to your program’s population and unique needs. However, the downside is that you might not use the best wording or achieve the same level of accuracy as a professionally developed tool. Over time, you learn what works and what doesn’t.
On the other hand, if you choose to buy a commercial assessment tool, make sure it’s well-constructed and researched. You should be able to review all the questions to understand what’s being asked and how the tool operates. You don’t want a system that processes answers in a “black box” without transparency. Be cautious about how tests are scored and whether the feedback provided is meaningful and practical for your needs.
For an assessment to be effective, the patient needs a validating experience. First, they need to feel that their condition warrants attention and treatment. Then, they need assurance that their condition is treatable. Finally, they need hope—a sense that a positive outcome is not only possible but likely. An action plan that is realistic and logical must be presented to them.
If the patient doesn’t receive validation, assurance, hope, and a clear action plan, they may disengage. This is why, in mental health clinics, we often see patients coming in for a single visit and never returning. Similarly, in treatment programs, if the intake process is overly taxing, the patient might lose motivation to continue.
Q: How can behavioral health assessments be leveraged within the criminal justice system to coordinate care between correctional facilities and community providers?
Dr. Hoffmann: It really depends on the system and the specific situation. In rural jails, for example, there’s often a very quick turnaround, and sometimes the departure time for individuals is uncertain. In these cases, the jail is usually limited to identifying problems and making referrals to community providers. Establishing strong relationships with these providers is key, and we’re seeing more progressive sheriffs recognizing this. They’re starting to understand that they can’t "arrest their way out" of the issue and need to collaborate with external services.
In state prison systems, because the individuals are there for a designated period of time, prisons have more options. They can either provide addiction services within the prison or coordinate with external providers to come in or pick up the individual once they are released. The same approach applies to the federal system.
We’ve found that local jails tend to have the highest prevalence of behavioral health issues. When you move up to state facilities, the population tends to be a bit more refined because some of the individuals have already been referred out to community providers. The federal system tends to have an even more selective population in terms of behavioral health conditions.
One of the challenges we’ve identified, both through research and clinical consultation, is that there’s often a silo effect between correctional researchers and behavioral health professionals. They don’t always collaborate as effectively as they could, and sometimes they overlook the realities of how these conditions contribute to offending behavior.
|
Interested in learning more about trauma and addiction treatment resources?
|
