Measuring the impact of digital tools: A guide for CCBHCs

Certified Community Behavioral Health Clinics (CCBHCs) often benefit from adopting digital solutions that can help them extend their reach and increase efficiency. But how do you ensure that a digital innovation is “measuring up” to its promises? The following strategies can support adoption, measurement and ongoing improvement of digital tools for CCBHCs:
1. Track Engagement Metrics:
Since patient engagement is a primary outcome of interest, define clear metrics for it. This could include portal log-in rates, percentage of clients completing at least one Atlas module, telehealth visit attendance rates, response rates to digital screenings, etc. Many tools have built-in analytics. For example, Atlas provides engagement reports at both the participant and organization level (showing how often clients are interacting with the Journals or modules). Monitoring these metrics helps identify what’s working. If certain modules have low completion, that might indicate the need to try a different assignment or receive additional encouragement from staff.
2. Clinical Outcomes and Quality Measures:
CCBHCs report on a range of quality measures (e.g., reduction in PHQ-9 scores for depression, initiation and engagement in SUD treatment, follow-up after hospitalization). Digital tools can aid in both achieving and tracking these outcomes. An analytics dashboard can pull data from the EHR to show, for instance, that clients who use Atlas had greater symptom reduction than those who didn’t, or that adding telehealth increased seven-day post-discharge follow-up rates. If data show certain groups (e.g., older adults, specific ethnic communities) are less engaged with a digital tool, the CCBHC can take action (perhaps providing additional training or choosing a different approach better suited to that group).
3. Retention and Service Utilization:
Key outcomes like retention in care (keeping people engaged over time) and service utilization patterns can be measured via the digital systems. For example, look at whether no-show rates decreased after introducing telehealth. Check if the average number of services used per client increased (a possible sign that clients are taking advantage of more wraparound offerings like peer support via online means). In one case, community health workers equipped with mobile tablets were able to extend services to hard-to-reach homeless individuals, leading to improved retention of those individuals in care. Such qualitative outcomes (extending reach) can be quantified by counting new engagements in previously unreached areas or populations.
4. Patient Satisfaction and Feedback:
Use brief surveys (which can be delivered digitally via text or email, or which are native in platforms like Atlas) to assess client and family satisfaction with the digital tools. Questions could include whether they felt more connected to their care, whether the technology was easy to use and suggestions for improvement. High satisfaction scores paired with high utilization rates strengthen the case that digital integration is benefiting clients. If satisfaction is low in any area, that’s an outcome to address — maybe through more user-friendly design or additional orientation for clients.
5. Health Equity Indicators:
A core mission of CCBHCs is to reduce disparities. Digital tools should be examined for their effect on equity. CCBHCs can stratify outcome data by demographic factors (race, age, language, socioeconomic status) to see if digital interventions are helping close gaps. For example, does the introduction of an engagement app improve engagement for younger clients but not older ones? Does a non-English version need to be deployed for certain communities? By measuring outcomes for subpopulations, CCBHCs ensure that technology uplifts all groups and doesn’t inadvertently widen gaps (for instance, if some populations have less access to the required technology).
6. Continuous Quality Improvement (CQI)
By measuring engagement, CCBHCs can demonstrate the value of digital tools to funders and stakeholders. More importantly, they can ensure that these tools are truly making a difference in clients’ lives — helping more people access care, stay in care and achieve positive health outcomes.
Evidence-based, behavioral health Interactive Journaling® curricula, including My Life in Recovery, are now available digitally on Atlas.
Digital interventions offer a number of benefits that can enhance the quality and accessibility of care. By accommodating staff shortages, increasing engagement and providing real-time data, such tools have the potential to transform the landscape of behavioral healthcare.