Berkeley DBT

Pioneering the future of mental health virtual assistants

Components

DeeBoT: an LLM skills trainer

DeeBoT is an innovative AI chatbot built on top of OpenAI's GPT-4 architecture that incorporates the fundamental skills of dialectical behavior therapy (DBT). Designed to serve as your personal DBT therapist, DeeBoT delivers customized guidance tailored by your own individual story and emotions, fostering a more personalized and deep therapy experience.

Conversation Simulation

To create synthetic conversations, we utilized GPT-4 and customized system prompts to generate diverse personas to interact with DeeBoT. By customizing prompts to various personality traits, we crafted a set of realistic conversations without relying on the personal information of users.

Evaluation

Two evaluation methods are implemented to validate DeeBoT's effectiveness: DBT Adherence and DBT Skill Extraction. These evaluation metrics work together to respectively uphold fundamental DBT standards and verify DeeBoT's ability to recommend the appropriate skills for specific issues.

Objectives

The project's objectives

This project aims to demonstrate how new developments in NLP can be utilized in mental health education, particularly in making DBT more accessible. We hope to contribute substantially to the field of mental health services, offering practical solutions that enhance user experience and accessibility.

Cutting-Edge Technology

  • Construct a learning-based machine that can intelligently discuss DBT concepts and guide users through the fundamental DBT skills such as mindfulness, distress tolerance, emotional regulation, and interpersonal effectiveness.

  • Ensure that the chatbot simulates a thoughtful and empathetic conversation style, reflective of therapeutic interactions but clear in its role as a support tool rather than a therapist replacement.

Simulation & Evaluation

  • Develop tools that simulate conversations to train and fine-tune the chatbot, ensuring that the guidance it provides aligns accurately with established DBT protocols.

  • Implement rigorous evaluation metrics to measure the chatbot’s accuracy in recommending DBT skills and its consistency in adhering to the precise standards of DBT, safeguarding both efficacy and safety in user interactions.

Accessibility

  • Address the prevalent challenge in the accessibility of mental health resources by providing a supplementary tool that users can access discreetly and conveniently from their digital devices.

  • Assist both individuals in need of mental health support and providers by offering a readily available resource that helps in understanding and practicing DBT skills, thereby acting as a bridging tool until direct therapist interaction is possible or alongside ongoing therapy.

Ethics & Safety

  • Clearly communicate the chatbot’s intended use as a non-therapeutic tool that provides educational content and guidance on DBT, and not as a replacement for professional mental health treatment.

  • Emphasize on safeguarding user data and ensuring the ethical use of conversational AI in a sensitive domain like mental health education.

DeeBoT: an DBT Skills Training expert

DeeBoT is your virtual companion on the path to emotional well-being. As your personal dialectical behavior therapy (DBT) therapist, DeeBoT listens attentively to your prompts, creating a safe space for you to freely express your thoughts and feelings. With its compassionate approach, DeeBoT offers insightful recommendations grounded in the principles of DBT, tailored specifically to your unique needs and challenges.

Through engaging and interactive conversations, DeeBoT introduces you to the rich toolkit of DBT skills, equipping you with practical skills to navigate life's complexities with confidence and clarity. Try DeeBoT below and experience your own personal DBT journey.

Your data is safe. DeeBoT does not store or collect user data and we do not train any models based on user data.

Image contains a screenshot of DeeBot, a chatbot specialised in DBT Skills Training

Why DBT?

Image contains an overview of Dialectical Behavioral Therapy

Proven Effectiveness

Dialectical Behavioral Therapy (DBT) is a highly researched and effective form of psychotherapy pioneered at the University of Washington. It is routinely used to treat patients dealing with anxiety, depression, suicidal ideation, and personality disorders with very high rates of success. It has 3 main components: individual therapy, skills training, and phone coaching. Notably, its skills training component is unique to DBT, and can be effectively implemented independently of other therapy elements.

Chatbot adaptable

dbt skills training is particularly well-suited to be delivered via a chatbot-like experience due to its structured nature and focus on skill-building through repetitive practice. these skills, categorized into mindfulness, distress tolerance, emotion regulation, and interpersonal effectiveness, lend themselves well to the algorithmic format of a chatbot, which can provide reminders, guided activities, and simulations at scale. a chatbot can deliver interactive exercises and provide consistency in practice without the immediate need for a trained clinician.

On-Demand Mastery

This approach allows users to access therapeutic tools at any time, making it a flexible solution that can complement individual therapy sessions or serve as a preliminary step toward formal care. Furthermore, the personalized feedback and privacy afforded by a chatbot can foster a safe environment for users to practice new skills without fear of judgment.

Technology & Architecture

Let's explore how we developed Berkeley DBT into a comprehensive platform for end-to-end DBT chatbot performance testing.

Conversation Generation

At its core, DBT skills training is all about the dynamic between a skills trainer, who imparts essential coping techniques, and a skills training client, who actively engages in learning and applying these methods to their own emotional and interpersonal challenges. We replicate this duo with two AI constructs: a DBT Skills Trainer Bot and a Patient Bot.

Our DBT Skills Trainer Bot, DeeBoT, utilizes a custom-tailored system prompt taking GPT-4 from "general-purpose AI" to "DBT-focused guide". This customization enables DeeBoT to initiate interactions with a friendly greeting, assess the user's emotional state, and choose the appropriate skill to teach based on the user’s needs. Playing the part of a DeeBoT user is our Patient Bot. Given one of 81 pre-written starting messages capturing various challenges a skills training client may face, Patient Bot is directed via system prompt to continue the conversation from their perspective. Together, these two bots engage in a simulated multi-turn conversation before the resulting synthetic conversation is passed through our evaluation process.

Evaluation

To ensure that DeeBoT adheres to established quality-of-care standards, we employed the DBT Adherence Checklist for Individual Therapy (DBT AC-I). This tool assesses a therapy session's structure, strategy use, stylistic approach, and skill teaching effectiveness to verify compliance with the DBT model. Out of the 26 individual standards outlined in the DBT AC-I, we selected 23 that were relevant in a text-only context. We then converted each of these into a system prompt that enables GPT-4 to analyze a conversation and assign a binary rating: "0" for non-adherence and "1" for adherence to each standard.

To assess the relevance and accuracy of the skills DeeBoT recommends during interactions, we created a database using regular expressions to identify which of the 46 DBT skills listed in the DBT Skills Training Manual appear in a conversation. After identifying the skills present, we compare them to the set we aimed to target for each specific user. Ideally, DeeBoT's recommendations should directly match the targeted skills, but if there is alignment with a closely related skill, it's still a good sign that DeeBoT was on the right track.

                    graph TD
                        A["DBT Skills Trainer System Prompt"] --> B["DBT Skills Trainer"]
                        B --> C["OpenAI GPT-4"]
                        C -->|Conversation generation|D["Synthetic Conversation"]
                        D --> I{Evaluation}
                        I --> E["DBT Adherence Checklist"]
                        I --> F["DBT Skill Extraction"]
                        G["Patient Bot System Prompt"] --> H["Patient Bot"]
                        H --> C
                        %% linkStyle 3 stroke:#ff3,stroke-width:4px,color:red;
                
Team

Meet the team of MIDS capstone students who made Berkeley DBT a reality!

shape1
Rob Mueller

Robert Mueller

Project manager
shape1
#

Hector Rincon

Project lead
shape1
#

Andrew Loeber

Technology Lead
shape1
#

John Van

Performance Lead

Acknowledgements

We extend our deepest gratitude to the individuals who supported us during the ideation, investigation, and development stages of our project:

  • Joyce Shen: MIDS Capstone Instructor
  • Todd Holloway: MIDS Capstone Instructor
  • Inna Lin: PhD candidate at the University of Washington's Behavioral Research Lab, consulted for her expertise on behavioural LLM implementation in the mental health/DBT space.
  • Dr. Chris Harrison: founder of the Wise Mind Institute, consulted for his expertise on clinical delivery and Dialectical Behavioral Therapy