Founded in 2010, Protect Line has helped over 500,000 UK families find life insurance through a fee-free comparison service. But the broker was losing sales time on unqualified leads, and prospective customers were dropping off before long sales conversations could finish.
Protect Line wanted to explore an AI chatbot that could improve the experience for both its team and prospective customers, so it brought in NineTwoThree to build a proof-of-concept chatbot capable of qualifying leads through natural, human-like conversation.
Conversational chatbot guiding customers through 17 key insurance questions with human-like interaction.
Retrieval augmented generation backed by vector databases using FAQ and help articles for accuracy.
Automated lead quality and engagement scoring with a custom evaluation suite for ongoing improvement.
Custom combination of LLMs with fine-tuned prompts for conversation, qualification, and data validation.
Building an AI chatbot sophisticated enough for production use in a regulated insurance environment required solving for conversational quality, compliance, and lead quality.
The chatbot had 17 questions to ask, but firing them all at once would overwhelm a prospective customer. NineTwoThree needed a conversational way to ask them, clarify vague answers without losing data quality, and still drive the customer toward wanting to talk to a salesperson.
Protect Line operates in a regulated industry where the chatbot had to use non-advisory language and could not recommend specific products. It also needed to handle distressed customers who may have mental health challenges such as depression, recognize when to cut the conversation short, and pass the person to a trained human agent.
NineTwoThree explored whether leads could be scored based on how a potential customer's questions matched Protect Line's best leads. For scoring to work, the chatbot had to extract enough information from the prospective customer during the conversation to generate a reliable quality assessment.
The team combined multiple LLMs, retrieval augmented generation, and chain of thought reasoning to create a conversational chatbot that could qualify leads while staying compliant with insurance regulations.
The engineering team created a question/answer mode where the prospective customer can learn more about what the chatbot is asking and then return to correctly answer the question.
17
key insurance questions asked conversationally
Pre-trained LLMs lack domain-specific knowledge and can hallucinate. The team implemented RAG backed by vector databases using Protect Line's FAQ and help articles as the source of truth, combining retrieval with LLM generation for more accurate responses.
Single LLMs have context window and attention span limitations. The team created a unique combination of LLMs with custom prompts for holding conversations, concluding conversations, qualifying calls, analyzing calls, RAG-based question answering, evaluating customer relevancy, validating user data, and summarizing conversations.
To answer all 17 questions, the team used a ReACT (Reason + Act) bot with Langchain. The bot takes action, gets an observation back to the LLM, refines its reasoning, and continues this multi-step process until all questions are answered.
A list of keywords was created to trigger immediate handoff to a human sales representative. This ensures that anyone suffering from mental health issues or having other difficulties is handled by a trained person rather than the chatbot.
After each conversation, the system generates a report scoring lead engagement (high, medium, low) and quality (hot, warm, cold). The team built a custom evaluation suite measuring conciseness, emotional intelligence, coherence, latency, token usage, and fluency, with a feedback mechanism for the Protect Line team to rate conversations.
The proof of concept validated that a GenAI chatbot could drive revenue for Protect Line. After implementation, the chatbot contributed to increased revenue by reducing time the sales team spent on prospects unlikely to convert and surfacing higher-intent conversations to close.
Instead of a rigid Q&A, the chatbot guides customers through personalized conversations covering 17 key insurance questions, replacing lengthy sales calls with a self-paced experience that prospective customers can navigate on their own terms.
After each conversation, the system scores lead engagement (high, medium, low) and quality (hot, warm, cold) using a custom evaluation suite. Sales representatives receive leads ranked by likelihood to convert, so they can prioritize the strongest opportunities.
The chatbot uses non-advisory language, cannot recommend products, and hands off to a human when it detects distress or mental health concerns using 50 trigger keywords, keeping Protect Line compliant with FCA regulations.
Build a new platform, or add AI capabilities to existing software, we deliver solutions that work in real operational environments.
Meet with founders Andrew Amann & Pavel Kirillov
Custom AI implementation roadmap
No sales pressure
