Chat-GPT Therapists

This mental health company connects individuals to certified mental health professionals to prescribe ESA Letters. NineTwoThree developed a Proof of Concept conversational AI that would help human therapists get to as many pet parents as possible.

Using AI to keep pets with owners.

CONCEPT

A mental health company exists that links people to accredited mental health specialists in their respective states, these professionals are competent to issue ESA Letters for accommodation and travel. The organization was established to fill a void in the market for reliable, legitimate, and credible ESA letter services. The founders identified a remarkable potential to provide substantial value in keeping pets with their caregivers.

They came to NineTwoThree looking to assess the feasibility of using state-of-the-art AI models to aid human professionals in therapy conversations. We were the best fit for this project due to our long track record of releasing successful and groundbreaking NLP projects in the AI and machine learning spaces. We developed a Proof of Concept conversational AI that would help human therapists get to as many pet parents as possible.

Challenge

The challenges faced by NineTwoThree Venture Studio in training therapy AI included selecting suitable algorithms, focusing on relevant themes, and establishing baseline performance. GPT, used as a comparison, revealed issues such as requiring large training data, significant computational resources, and fine-tuning for specific tasks, which can be challenging and time-consuming. Generalization is another concern, as GPT might struggle with rare or specific tasks, potentially failing to provide appropriate answers for sensitive topics like self-harm. A therapy bot needs conversational understanding and contextual awareness, making fine-tuned machine-learning models essential for accurate responses in mental health conversations.

To achieve this goal, here are the objectives:

Select

leading algorithms most suitable to the use case.

Pick

one theme most represented in available conversations.

Ingest

the selected conversations and prepare training / test sets.

Establish

the baseline performance of selected algorithms on our test set.

train

the models using the training set conversations.

Measure

the performance of trained models.

Compare

the performance of baseline and trained models using automated tools.

Evaluate

the performance with the help of domain experts (therapists).

solution

Utilizing the client's dataset of therapy conversations from the 90s, NineTwoThree trained two specialized models for anxiety and depression, based on real conversations. We measured their performance, compared that performance to the baseline, and had that baseline evaluated by expert human therapists. Due to time constraints on the project, we did not pursue too much time on data preparation around small talk but focused instead on the bigger picture.

Our fine-tuned models were compared to a regular GPT baseline, which was evaluated by expert therapists. The models excelled in acknowledging users' feelings, asking relevant questions, guiding them to solutions, and helping articulate emotions. During testing, an experienced therapist rated most responses at 4.7 out of 5 for efficacy. These promising results could potentially be improved further with additional fine-tuning.

impact

Integrating machine learning models into older applications remains a challenging proposition. However, making this effort ensures these organizations still have the ability to compete with businesses already embracing AI. In order to get ahead of the competition, you need a partner that has the experience to work with this technology and much more. After working on dozens of machine learning projects (and having a dedicated team), our ML scientists know how to train the models to uncover your problem.