NineTwoThree Case Study: Mental Health, ChatGPT, And NineTwoThree

Companies have long realized the potential of having some online offerings for mental health - just look at Better Help, Talkspace, and ReGain to name a few.

This industry began to boom during the COVID years and launched a new aspect of an industry that had long remained traditionally in-person only. 

But these services are expensive, dependent on therapist availability, and don’t always fulfill the user's needs for support around the clock. Automated chatbots came with similar limitations, being unable to understand the nuances of the user’s emotions, and were much clunkier than seeing a human therapist online. 

Traditional chatbots are not advantageous for therapy because they are typically based on simple rule-based systems or pattern matching, which means they do not have the ability to understand or respond to complex human emotions or mental health issues. Additionally, they do not have the ability to provide personalized or human-like support, which is important in therapy.

That was until relatively recently when OpenAI released ChatGPT - ChatGPT a language generation model developed by OpenAI, is capable of generating human-like text given a prompt. It is the third iteration of the GPT-n series, which stands for "Generative Pre-trained Transformer". GPT is trained on a massive amount of data and can perform a wide range of natural language tasks such as translation, summarization, question answering, and text completion. 

But as this new technology takes off in the business world, teams that have experience in AI and machine learning are now in high demand - including NineTwoThree Venture Studio. And this interest in OpenAI’s latest offering has now extended to the mental health app market.

The Potential Of ChatGPT In Emotional Support 

The ChatGPT  model is pre-trained on a large dataset of text, allowing it to learn patterns and relationships within the data. When given a prompt or a starting text, the model uses this pre-trained knowledge to generate new text that is similar in style and content to the input it was trained on.

The model uses a deep learning algorithm called a transformer, which is designed to process sequential data such as text. The transformer's architecture allows the model to understand the context of the input, and generate text that is coherent and appropriate in the context.

This put forward an interesting question what if ChatGPT  could be trained to be a replacement for a therapist? What kind of results would they find?

The key for this client was to approach a machine learning venture studio that had the capabilities to deal with this ‘new’ approach to AI while simultaneously understanding the nuances of mental health, therapy as an industry, and the need for more mental health support online. This was when they partnered with NineTwoThree Venture Studio.

And it was a partnership made in heaven- a collaboration between businesses with the right skill and an entrepreneurial spirit that made perfect sense.

The Concept Of A Therapy-Based ChatGPT

This client was looking to assess the feasibility of using state-of-the-art AI models to replace human professionals in therapy conversations. As a company, they had access to hundreds to thousands of real therapy conversation logs that happened in the 90s. The sentences of those conversations were labeled as “patient”/”therapist” so as to maintain therapist/patient confidentiality.

 In order to create a proof of concept, this client was willing to provide the NineTwoThree with access to those conversations to be used in training this large language model. This client was ultimately looking to develop a Proof of Concept conversational AI to see if it’s possible to replace a human therapist with AI.

NineTwoThree was the best fit for this project due to our long track record of releasing successful and groundbreaking NLP projects in the AI and machine learning spaces. 

NineTwoThree uses the principles of predictive analytics, market research, and computer vision to deliver world-class solutions used by the globe’s leading companies. 

To achieve this goal, there were several objectives and challenges that had to be addressed as part of the development process.

The Challenges of Training Therapy AI

The challenges NineTwoThree Venture Studio came across during the training of this model started with having to select the best algorithms that would be most suitable to this use case, dialing in on the themes most represented by the available dataset, and establishing a baseline performance of selected algorithms on our test set.

Using GPT as a baseline to compare made apparent several disadvantages of using it as is. Not only does GPT require a large amount of training data in order to learn the nuances of natural language and generate human-like responses, and even if you have access to such data training such a large model requires significant computational resources, including powerful GPUs and a large amount of memory.

And that’s only the backend. Fine-tuning the model for specific tasks or domains can be challenging and time-consuming, as it requires a good understanding of the task and the ability to create and use appropriate training data. 

There is also the issue of generalization - GPT is trained on a diverse set of internet text and can generate coherent and fluent text on any topic, but still might struggle to generalize to a specific task or domain, especially when the task is rare or specific. 

So if a user was to pose a question about a difficult topic like self-harm, the baseline model does not provide a suitable answer.

What you need from a therapy bot is not just conversational understanding, but also the depth of understanding when it comes to context, synonyms, and environmental aspects. You need a finetuned machine-learning model that can identify the right response based on matching criteria, and nowhere is this more crucial than in mental health conversations. 

The Solution To AI Chat Therapy That Really Works

Using the client’s dataset of therapy conversations from the 90s, NineTwoThree was able to train two models, measure their performance, compare that performance to the baseline, and have that baseline evaluated by expert human therapists.

Our combined trained therapy GPT models are trained to specialize in conversations surrounding anxiety and depression based on thousands of real conversations around these topics. Due to time constraints on the project, we did not pursue too much time on data preparation around small talk but focused instead on the bigger picture.

Compared to the baseline of the regular GPT model, our finetuned therapy model was able to:

1. Acknowledge how a user was feeling, asking the right questions about it and helping them to a solution.
2. It was also able to put words to feelings that the user might not have been able to connect themselves, such as asking “Do you think you’re grieving?” in response to a statement about sadness surrounding the end of a relationship. 

The questions and responses received by the models during the testing phase were rated by an experienced therapist on a scale of 1-5 with most of the answers being rated 4.7 in terms of efficacy by a human expert. These results are promising, and can likely be pushed even further given more time to fine-tune the model’s approach. 
‍

Why Work With NineTwoThree On Your ChatGPT Project

While many companies develop new machine learning applications from scratch, a massive amount of legacy software exists throughout the business world. Integrating machine learning models into these older applications remains a challenging proposition. However, making this effort ensures these organizations still have the ability to compete with businesses already embracing AI. 

In order to get ahead of the competition, you need a partner that has the experience to work with this technology and much more. After working on dozens of machine learning projects (and having a dedicated team), our ML scientists know how to train the models to uncover your problem. As a venture studio, we’ve launched over sixty apps and fourteen startups and have the business and technical acumen to ensure your project is a success out of the gate.

Our team is constantly testing "what's possible" and is willing to go the extra 1% in accuracy. Our data scientists can handle all of your data-related requirements including software development, labeling, and modeling. After the algorithm is tested, we wrap the solution in a cloud infrastructure to deliver a fully functioning ML solution.

If your business needs a partner for an upcoming project, reach out to the team at NineTwoThree. A venture studio with significant experience helping businesses from startups to enterprises, we boast the technical chops and business acumen to help ensure a successful project. Connect with us to discuss the possibilities of a partnership.