Adam Parks (00:06)
Hello everybody and welcome to Applying AI, the show where we cut through the hype and get real about using artificial intelligence in regulated industries. Season one of Applying AI is sponsored by EXL. Let's dive in. So today I've got two great guests with me. I've got Sara Burton (Woggerman) and my cohost, Mr. Mike Walsh, joining us from EXL. And we talk a lot about artificial intelligence and the practical application of it to debt collection organizations.
And I thought as we kicked off today's discussion, I wanted to bring something up that had come up in a call recently, which was organizations that are using Agentic agents for outbound calls may not be using the same compliance listening tools that they would be using for their traditional collection methods.
And I thought that was interesting whether they were viewing the AI agents as a judge for LLM opportunities, so the models judging the models versus running it through those standard channels. So I thought I would kick this off talking a little about how we're looking at AI compliance. But Sara, is this something that you've ever seen in the wild?
Sara Woggerman (01:21)
Well, I mean, I've definitely seen that. ⁓ What's interesting is that you have to be able to model those AI tools that are, let's say, judging or reviewing the calls. They need to sort of understand what they're listening for, right? So are they, let's say you have speech analytics or a traditional tool that we've been using for a long period of time, or human listeners, there's very specific things you're listening for, right? And that is not always a black and white thing that's sometimes subjective, right?
Given the nature of the call, what unfolds during the call, so what makes me nervous about AI tools sort of judging other AI tools is I think you could do that for certain elements of a call, like your call opening, maybe even your call closing, maybe you're taking a payment. But everything that happens in the middle, I don't know that it's ready to judge it, right? So I'm not sure why we wouldn't maybe test both of those tools because those LLMs need to be as good as a human listener, ultimately. Mike, ⁓ I'm curious what your thoughts are on this from your perspective.
Mike Walsh (02:43)
Yeah, so I'll kind of go into our process for this, Like, so compliance is built into our tool, but we still, so when you deploy AI voice, especially, ⁓ you're going to get the recording, the transcript, any, a lot of the exceptions, especially early on as you build more journeys.
So if you look at this as the first thing I'm going to deploy and test isn't going to be broken promise, right? Cause you can't get to that to get to the promise, right? Like, so it's like, I always tell people it's like baby steps from, ⁓ what about Bob onto the bus, baby steps onto the boat. and then, so you're gonna, and before you deploy, should test this a thousand times. Like we usually do a lot of testing prior with fake calls. So you're going to do it.
Now the calls recorded that's available to you. The transcript is available to you. We do have guardrails like guardrails monitoring and not, it's almost restricting bad speech for. So that's all there, but you're right Sara. Like, so how do, how do I, as a someone using our tool, for example, audit it, you still can audit it. It still can be on that QA call sheet to listen to. In fact, all those recordings would be lined up for Mike Walsh and like you just pull them up and go. So you can do it manually. ⁓ But I don't know if I would get a different tool because there's really a Agentic AI is not one LLM watching.
You're like multiple not just not just the LLM you're using to understand the speech and there are all these different things working there. there's, we have an orchestrator. you, like, if you look at all those different parts you just mentioned, Sara, the orchestrator controls them and says, wait a minute, they just asked for their statement balance, not in the middle of negotiate, pull out of that and go to that because that's the most important thing based on the call.
So you can definitely audit them. I don't know, I don't think we're using a separate tool to audit them. I mean, I guess he could, why not?
Sara Woggerman (05:06)
Well, here, let me give you this scenario, that might kind of just help us sort of paint the picture for our audience here is one of the things I talk about a lot when I'm talking about AI is kind of like what the myths are versus reality. And there was a lot of compliance concern and just fear around hallucinations, right?
And I've said to a lot of people, I have not experienced hallucinations once with any of the clients that I'm working with with AI. And I have not heard a single horror story about hallucinations in a conversation with a consumer. That hasn't happened to my knowledge, right? And it's because of what you just talked about. It's multiple bots doing their job and then moving to the next phase of the call.
But what I have seen, which isn't it going off the rails or hallucinating, is it learns slang from the consumer base, right? And so if a bot is auditing the bot, it's going to just learn the same words and maybe not say, hey, we don't want it to say bro to our consumer base. We don't want that word to be said. Or we don't want it to say amazeballs, right?
We don't want it to say those things. ⁓ And so that's where the oversight piece still needs to sort of be independent from the whole world of AI, right? Is I'm not worried ⁓ or my fear of it going completely off the rails and doing something crazy ⁓ risky. It's kind of like those little nuances that it can learn ⁓ because it is generative, right?
Mike Walsh (06:57)
Yeah, and there's a solve for that, right? So we call those utterances that it will understand, but it will never respond with, right? Even if you think of text, it's the easiest example is I hit S, or STP and no O. We would give that to a client and say, we think this means stop. Is this stop for you? And they say, yes, that's stop, right? Or, you know, because people don't talk like even our AI we make it sound more human by putting ums in there and things like that some Slang it's not proper English But if it gets too robotic, no one's gonna talk to it. So We don't let it you utilize slang in the outbound.
So there is a place for that and there is a model that is you know ⁓ It's like a sentiment model almost where it's moving the pitch of the speech. It's like, I can't work because someone in my family died. It's not going to say, great, send your payment in two months. It's got to match. So it's getting very, very complex. But it's interesting talking about these little details and then the devil's in the details. So it's a great thing to test. You should.
I love when people like at RMAI someone grabbed my phone testing my, my AI and they said, I got to get my wallet, hold on a second and waited. And it said, take your time. Like a human would, but he said that's broken like three others. So those little details out humans really speech, like do it, do you want to call outside, do a call, uh, in a car, do a call where people talk with kids in the background, right? Like dogs.
⁓ And then you'll see, you see what you're getting. But it is interesting. It's an interesting concept, but you should always have some human oversight. Never trust machines to govern machines totally, not yet.
Adam Parks (09:06)
We've spent millions of dollars and thousands of hours developing our call analytics tools for the live person. I don't understand why we wouldn't use that too. I think there's value in using some of these LLMs to watch LLMs and to start to refine that. But to Sara's original point, I don't think we're there yet to where we have that level of trust in these tools.
But if we've spent all of this money and time and energy, to be able to filter up those specific calls or instances within a call that need human ears and human evaluation, why not continue to use those same things and develop that comfort level? I think from a regulatory standpoint, it also demonstrates how we're treating these tools because as Mike and I have said a few times now in our conversations, we're all responsible for the output of the AI.
And we're just like, we're responsible for the output of the people that work for us. And if we're able to start looking at it that way and leveraging those tools in new ways, I think that's where we're going to find our stride.
Sara Woggerman (10:15)
No, I completely agree with everything you just said there, Adam. And again, it's important to think there has to be checkpoints throughout all this automatic decisioning, right? So compliance doesn't go away just because we built something and we said it should never happen, but exceptions happen, right? So when I think about what Mike's talking about, there's your proactive and your reactive controls. Those still exist in the world of AI.
It's not all 100 % proactive, perfect. We would love to think it is. ⁓ But things still, there's still exceptions to everything that we do. And so I look at the LLMs that are testing the LLMs as another tool in your toolkit to get to your potential issues faster, right? This should make us more compliant, more operationally sound. We should be able to turn over things quicker.
We should be able to be maximum efficiency because we're finding anything that a potential breakdown faster, which we have, we've never been able to get to these things fast. We've been needle in a haystack finding these things.
And so that is nothing but positive in my opinion for the industry from a regulatory testing standpoint, I think it's, those are all positive things. And that's, I think we need to make sure that we're thinking about all these things. How do they all work together for you to find those things quicker?
Adam Parks (11:51)
Have you seen, have either of you seen clients come into the table with measurable compliance, KPI expectations, or is it still an evolving art form as we start to deploy this type of technology?
Mike Walsh (12:06)
I will tell you this, it's changing fast, right? As Agentic's taken off, ⁓ you see the questions now around like latency, disconnections, hallucination percentage, things like that are gonna be key KPIs and there's more, there's a ton more.
Imagine like just think a call in the airline At least everybody's called the airline like think of those frustrated like the death loop like how long and Those are like the old kind the new that you would deploy to collect money in this type of environment They have to be spot-on right like they have to be primo.
So everything matters speed under the ability to understand the misunderstandings the repetitions that's all getting measured and when you think about deploying these solutions, think about measuring those, right? Like think about a cheat sheet of what you've listened to and hated, what went wrong, and then apply that and make sure your customers, you know, and your clients are getting a tool that is satisfactory.
And then when it can't do the job, because of, you know, you're somebody has a utility debt and you ran over their flowers, utility company, and they're not gonna pay until they come back and fix it. Like that's not gonna be an AI call, right? Like that's the exception it should get rid of. It should recognize it can't handle this call and get that to a person and the right person.
You know, if someone says bankruptcy, and you haven't programmed that journey yet, you want to get them to your bankruptcy team. So measuring those type of KPIs. So I think KPIs, I mean, you think about as scoring, it's changing. It's definitely changing. The warm transfers like now on KPI from, how did that go from?
AI to human or human to AI. Like if you're going to take a payment, you don't want your people taking payments anymore over the phone, send it to the AI and make sure it's capturing and it's going quickly and everything goes smoothly. So yeah, I think Sara, I don't know what compliance KPIs you're seeing, but I think they all are really like most all KPIs are compliant.
Adam Parks (14:24)
Well, I think from a Sara perspective, like, have you seen any evolution of the scorecard, for example, as we've started to deploy this and as we're listening to those calls in real time?
Sara Woggerman (14:31)
You know. I'm surprised I haven't seen the evolution of the scorecard. And I think the reason why is probably because my client base is oftentimes debt buyers in the third party collection agencies. And the AI folks I'm working with, some are just tech vendors, and some do third party and have a SaaS thing, right? So I'm seeing it from a little bit different perspective.
So I'm actually glad to hear Mike saying, at least he's starting to get those KPI questions coming in. I think it's not as widespread being used or it's being used for pockets of accounts or you know very well defined we're going to try this low balance portfolio or we're going to try this because we you know we feel confident and the you know to pass-through consent or they're still not deploying it at a large enough scale to compare it to other performance, I think yet.
But I think we're close. I think we're getting close to seeing that evolution. ⁓ I think when I start seeing it on debt buyer scorecards is where I'll say, okay, we've sort of made that change, right? Because they use, oftentimes they use large networks ⁓ and you've got an array of different strategies being deployed within those networks. And I look forward to sort of seeing that evolution because I think, especially some of the bigger players out there, I would suspect that those will start to evolve pretty soon.
Mike Walsh (16:11)
They're there.
Adam Parks (16:12)
Do you think it will start to evolve as they deploy that technology internally? So as debt buyer A or B or C starts to expand in that capability internally?
Sara Woggerman (16:16)
Yes. Of course. They'll start champion challenger. Yeah, they're going to start champion challenging, how people are deploying it, how aggressive they're being. The vendors are, you know, is this vendor working better than this vendor? I mean, all that's going to start coming into play. That's going to be really exciting time. ⁓ And ⁓ I do see that. Yes.
Mike Walsh (16:43)
Yeah, absolutely. I agree. It's, I think it is happening on the creditor side. There's no doubt. ⁓ and it's, it's gonna, it's a lot of reporting, right? If you think of agency ABC out there, you're going to need some more fields. Like there are, or you need a vendor who's going to report all these KPIs back to you. Right? Like that is a big part of not, you know, not all AI is built to sing. Right. Like, so, think of what your clients are gonna look at, especially with phone calls. You know, email, two-way text, it's easy.
It's standard, right? Delivery rates, all that. When you get into phone calls and you're looking at sentiment analysis versus the actual call. Like I think that's what people are going be listening for. Did it match? Did it not? There's so much information that's going to be utilized and can be measured. And, you know, the tools are learning from them, but it can also be, you know, in Sara's original example, you can pull the bad stuff or stuff that doesn't make sense out, right? Like you can say, okay, kill this.
So it will be used to improve and then it's gonna be part of your scorecard. That your AI...this is like an arms race to me. like, you're, all going to use AI eventually. 90 % of agencies are going to use them. So then you're going to be measured on what you're using. And that's going to be a big part of how you're scored.
Sara Woggerman (18:23)
So I really like what you pointed out there, Mike, about the reporting specifically. that is, it kind of pivots into another thought that I think about a lot and I haven't quite seen sort of, a great sort of..I haven't seen any documentation that has quite satisfied my desire to see the decision chain. so when so for example, I recently had a conversation with somebody and said, I love that what I just told you, you went and you fixed and then we listened to a call and we fixed the problem. I love that.
But in five years or how do you know that Sara told you to do this, right? Or there was a compliance review that was done and that you fixed something as a result of actual oversight, right? How are you documenting that? Because, you know, it's great that you can go in and go, that's technical term everyone, ⁓ and that you fix the problem.
Right? So you do that. And that's great that it and now it's fixed. But like, how do we explain this to a regulator in five years? And those are the things that there's so much decisioning and the technology is moving so fast. ⁓ And you might have some really good feedback on this, Mike. You might be working on this or you might have something already developed, but like, you know, there's a, there's a Colorado state law that has a lot of reporting requirements and I'm not exactly sure what those reports are supposed to look like yet. And I don't think they know, which is why they keep pushing out the effective date.
But, that is something like that change management, that decision tree, how those things get reviewed, I think is really critical. I think your clients are going to expect to see that. And I know regulators are going to expect to see that. So I'd love to hear your thoughts around that.
Mike Walsh (20:31)
I think that's a great point.
Adam Parks (20:34)
Documentation of the learning process is what I'm hearing, right? How is it learning and how are we documenting the learning process? Go ahead, Mike.
Mike Walsh (20:42)
Yeah. And, and how is this account auditable? Right? Like how do you audit? Like where is the audit trail? And then it gets slippery, right? Like, because, you know, I don't want to show you how my tool works because it's patenting and everything else. Right. Like, but, but I know that every client of mine needs to be, is going to be audited and needs to show how this decision was made.
Part of this is the initial, and Adam, you and I have talked to a thousand times, is going to the vendor and saying, okay, how is this model trained? Who’s doing it? How did it learn, how does it prevent bias? How is all this stuff in there? Right? Like, so that's step one before you even deploy anything is make sure you understand that. Is it all yours? Is it fourth party? Are you reselling something else?
Like you have to understand that. and then part of what I I'm doing, I'm going to CBA live and talking about what questions to ask. And one of them is how do I audit this? Right? where's, how do I see how you came to this settlement? Right? Or this payment plan. And if you can't show that, man, I don't know what you're going to do because that, I mean, this were highly regulated. It's got to be built. And I think we saw some like companies come over two years ago at RMAI wanted to see the numbers of accounts that are in this business and say, we want a piece of that, but they're marketing companies. had none of this built in. They had no reporting, no audit trail and they left.
Sara Woggerman (22:01)
No, it's going to be a problem, right? Yeah.
Mike Walsh (22:22)
Because I had one say, man, how did you guys pass the data security these banks are out of their mind requiring? I'm like, because we built it for them. Like this is a collection tool. What are you talking about? How did you not know this? But it's great.
Adam Parks (22:37)
Well, it's entry from third party countries to organizations coming in from other countries, not understanding the level of complexity. So the first two questions that I always ask of an organization that says they are bringing in an AI voice bot to the marketplaces, who's your attorney and who's running compliance is the two most important questions that I can ask. And, but I want to go back for a second. We were talking about kind of how you're going through that auditing and how you're documenting that decisioning, but we're gonna have to measure this at some point.
And I think this crossover between the idea of the KPIs and how we're going to put some numbers to being able to measure this output and what these scores are gonna look like over time. Because the documentation of how we change it I think is important, but we're gonna also have to show a positive trending results line that says, look we are improving this over time because if we can't or if it plateaus, I think the regulators are going to have something to say about.
Sara Woggerman (23:39)
I agree with you. ⁓ And I think that, so I think these two things go hand in hand, but I don't think the operational effectiveness and the compliance ⁓ sort of goals are at odds here. I think they are very much aligned. And I think that the metrics and the change management, decisioning and all of those things, I think they all play really nicely together, if we can take that information and then put it in understandable terms for the regulators. That's the other key to it, right?
So if we've got a bunch of data scientists saying, well, I don't know why you don't understand all this beautiful stuff here. ⁓ Like Mike and I have to understand this, right? Like dummy it down for us. Because we've got to be able to explain that to our regulator. and because they're not going to be, they're not, they're going to need it in the most simplest form, but ⁓ we're going to have to show our work, right? That's ultimately what it is.
Adam Parks (24:50)
You're going to have to show technical capacity in a traditional format because the regulators aren't going to understand the technical capacity. They're not hiring data scientists. They're used to auditing a live collection agency and we're going to have to be able to communicate PhD level complicated things to someone who has a middle school understanding of a concept.
And I think that gap is going to be a significant challenge, which is why we focused this episode on that measurement and understanding of what the compliance guardrails are ultimately going to look like as this evolves over the coming years. Mike, I can see you have an opinion. I want to hear it.
Mike Walsh (25:32)
But I would say that right. I think you made a great point. It's got to give the output of traditional collections, right? Like that has to be there. The conversation, the notes, the Spanish translation or French Canadian translation, like that all has to be there. They have to go through the audit. Same thing with like payment processing. That's not hard.
But I do think there's going to be more information that if they want to dive into. ⁓ And I think that's one of the questions you ask your vendors, you know, what responsible AI guidelines have you utilized to make this like what, you know,
Europe's been ahead of us on this. We've talked about that. I think a lot of the state laws are based on the European Union's AI laws, which I think make a lot of sense, right? Like, EXL follows them. We follow them globally, not just over there. We follow them everywhere, every market, because they're logical, right? Like, so I think it's...
You're right. Whoever you're using has to send you everything you need for traditional audits plus more. And that's where like, I always say, be careful what your AI vendor gives you. ask for, they're asking for age. Get a good attorney, right? Like call Adam and find a good attorney or call Sara and find a good attorney. Because right now you are dead.
You are dead. Like that is biased and that is ridiculously dangerous. Like so if you hear something in that initial talk and they're asking for tons of things, 30,000 hours of tapes or a relationship they have nothing to do with, Reg layer might say, how was this train? the train on your information, you know.
They're not credit or ABC. Why did they listen to these private phone calls? Explain that. don't ask you like, I think you have to think of like, as if I'm handing these to a human being and they're gonna listen to them. Is this just, is this gonna get me in trouble? So there's a lot, you're right. It's, you gotta do the traditional and then you gotta be ready for what they're gonna ask for bias and things like that where these models.
You got a responsible AI. I would read, look into that and then model development and training. How does that happen and what's just being developed and trained on.
Adam Parks (28:09)
What are we going to measure? What are these numbers going to look like? hallucination percentage, as Sara mentioned, she hasn't come across one yet. I think hallucination became a major topic with ChatGPT 4.0. And I think it became a major topic because you had attorneys that were submitting documentation that had zero research to it. And it was citing cases that didn't exist and those kinds of things.
So it was like irresponsible use of the tool set, I think is what prompted a lot of that, your set, but I don't think that that's the big challenge that we have now. Cause Mike, as you've talked about stacking those LLMs and looking at it from that perspective and everything, you know, it's, ⁓ it reminds me of the movie casino, right?
Like everybody's watching everybody and the eye in the sky is watching us all. And it feels like that's the way that we're going to go, but we are going to have to slim some of these things down in a measurable way to be able to communicate it to someone who has zero technical expertise. Cause they are not sending them coders out to conduct these audits that where you're going to be able to, I must say, talk above or around them. You're going to have to bring it to their level for their understanding and their comfort level, or we're going to pay the consequences as an industry and as individual organizations.
Mike Walsh (29:25)
I mean, I think it's easier than you think, right? Like, because you know me, Adam, I'm the most technical guy on earth, right? Like, working at an AI company. ⁓ Like, I'm the worst.
But I look at our reporting and I'm on client calls and we go through the measurables that is on like it's a AWS QuickSight usually, but it could be a Power BI tool or Tableau, right? Like you have everything you could possibly dream of measuring right there by segment, by date, can go on account level, you can go on a portfolio level.
The great thing about AI is the best thing about it is it's great at measuring insane amounts of data, even unstructured data we structure and say, why did people ask for assistance? Right? Like the reason they put, they said, right? Like that's all measured now. Like you can have a category of how many people lost their job, right? I think, I think that's going to help regulators not.
Like I think it's going to be more convenient to audit. Like that's how I think of it. where, if you think of pulling some of that data off a old collection system, it was a pain in the neck, right? Like you had to have for an audit. And now it's like, here's the tool. What do you want to see? Right? Like.
Adam Parks (30:49)
As you provide more information, are you creating more risk?
Sara Woggerman (30:51)
I was just, knew, were you reading my mind? All right, so So yeah, let's talk about that for a minute. Be careful what you wish for because we know regulators love to say things like known or should have known, right? So that phrase gets thrown around in consent orders regularly. It's also in many laws, known or should have known.
And so when we need to as an industry, when people like Mike are giving us these great tools and feeding us all this information, we must do something with that information. To get that information and do nothing with it, creates risk, right? ⁓ And that is where you're going to end up getting your hand slapped.
So sometimes it's best to not have the information and certain instances, unless you have the resources to make informed decisions based on that. So if you're learning from, based on the metrics you're getting from your AI tool, that one of your policies really upsets people, right? and creates complaints and you do nothing about it, you don't evaluate that and maybe adjust it or do something different, then you are going to create risk within your organization. There's no doubt about it.
Mike Walsh (32:22)
You're absolutely right.
Adam Parks (32:23)
I just think back to the original CFPB. At the very beginning of the CFPB, they were coming out and they were finding for nothing. And I think the more information that they have, the more likely they are to do that. Because let's be perfectly frank, the political pendulum swings. And right now it may be pushed really far in one direction, but it's going to come back again. CFPB is not dead. They're not gone.
And they're going to hire again. And when they hire again, they're going to hire the same sycophants that they had at the beginning. And we're going to come into these same challenges. So if we start providing this mountain of new data, they're going to find and invent new ways to create challenges and problems where they maybe don't necessarily exist. But that's not going to stop the risk level. It's not going to stop the fines. And like usual, it starts with the creditors and works its way backwards.
Sara Woggerman (33:18)
Correct. And logic doesn't necessarily rule the day in these, right? So, and I say that very seriously because for those of you who are new to the United States of America and how this works in a regulatory concept, like that's one of the things when I'm talking to people who have launched these tools overseas and now are coming into the US market is that they have this sort of notion that they can logically explain this and it'll all make sense. And it's like, no, that's not how our regulators respond.
That's not how a court case gets decided in the US. Like, I wish it was that simple, folks, because logic does not rule these often, right? And ⁓ what they deem as consumer harm might not be harmful at all. It's just their sort of opinion.
Mike Walsh (33:59)
Not here.
Sara Woggerman (34:18)
⁓ in some of these cases. ⁓ And yes, the more information they have, the more subjective they can be with what they extract from that to sort of create that narrative. And so we need to always be thinking about how could this be twisted the other way, which is what I do all day, is like, all right, how do we make sure that ⁓ this is improving the experience for the consumer? ⁓
That we can show it's improving the experience for the consumer, that we're being more compliant ⁓ and that this isn't creating undue harm. And I think, I really do believe, I don't just think, I actually believe that AI will help us accomplish those things. ⁓ It's just going to be about how you evidence it and making sure that you are, the information you are getting back from the AI tools that you are disseminating that information so that into your compliance management system and doing what a compliance management system is intended to do, which is identify risks and fix and remediate them, right?
Change policies as needed, change your practices and then document that so that you can show Mr. Regulator, Mrs. Regulator, look at what we've done as a result of this, right? So it's a positive experience.
Adam Parks (35:49)
You just made me think about what it really is. No, you made me think about this a little bit. Can you explain this to a jury of your peers? And I know that sounds like a kind of a simple concept, but I don't know when the last time either of you have actually gone and done jury duty, but it is a shakingly scary thing when you realize what a jury of your peers actually means and what level at which this needs to be explained in order to meet that standard.
Sara Woggerman (36:18)
Yeah. No, I think that's important to remember. I love that, Adam.
Mike Walsh (36:23)
And I think compliance has to be part of the decision making for these tools, right? Like when you're buying a tool or building a tool, you have to like, I think there's a couple of things people, especially people who build and have run into that come back to us and say, updating this thing with all the laws is a nightmare, right?
There's 21 AI laws on the books right now. Sara, you're on the committee with my colleague. And I was like, what? And 21 different ones. And you know, you gotta be able to go in and change it and change it by location of the consumer. Then just, think that some creditors, like I always think of the fair lending acts, right? Like that is a big part of, I think the European AI law is kind of based on some of those and risk modeling.
If you've been doing this a long time, we started 2018 with this product. So we've kind of seen a little swing in different directions. And I was at the CBA when the CFPB declared war. They were going to sue most of you and or some of you were suing us and we're going to defend ourselves. Like I was like, whoa, this is crazy.
But, yeah. So you have to build a product that's going to withstand regulatory compliance. and you have to part of the, like, I think part of the problem that I've heard from other people is they run into a tool that they just can't get what they need to show. Like being able, I think that's a great way to say it them is to go into court and show why it did it and how it protected this consumer. The beauty of AI.
Adam Parks (38:08)
What happened? What happened? And I remember the Chopra incident that you're referring to at the consumer bankers or the consumer bankers association and what that actually sounded like. I look, it was a scary day, I think, for all of financial services and for the consumers themselves because of the impact that some of those statements have. But on the tail end of that regulator, if you can't, if the regulator refuses to understand, not can't understand, but refuses to understand.
Mike Walsh (38:15)
That's great. And he's left.
Adam Parks (38:36)
In the end, it ends up with jury of your peers. And can you explain this to a jury of your peers? This is where you have to start thinking about the average reading level of an American, the average understanding level of an American, and make sure that you can explain these decisions and this tool set to that population. It's unfortunately the reality of
Mike Walsh (38:59)
And those rules will change, right? Like it has to be flexible enough to change with the times. It has to be.
Adam Parks (39:04)
And, can you unwind it once it learns? So if we're training something and the rules start to change, what capabilities do we have to be able to work backwards and to get these tool sets to be able to comply with it? Now, Sara, I got an interesting question for you because I know you work with both tools, you work with the debt buyers that are deploying tools, and you work with AI first agencies or AI first organizations.
Do you see a difference in their outlook of compliance between those three segments of the market? Those that are building the tools, those that are deploying the tools, and then those groups that are actively building themselves as an AI-first organization? Is there any difference in their compliance posture?
Sara Woggerman (39:49)
Of course I do, yeah. Yeah, I do only because, ⁓ you know, if you're a large debt buyer, consent is going to be something that ⁓ you really have to evaluate from a risk standpoint, right? Because you've got...there's some murkiness in our laws. So a deregulatory environment creates a whole bunch of gray areas sometimes.
And that has happened with the TCPA. And um whether or not consent is passed through. And then you've got the question of how reliable is that consent. So for outbound, this is tricky business for the debt buyer space.
Mike Walsh (40:35)
And a collection agency space, right?
Sara Woggerman (40:36)
And so until we get some clarity around that, that's going to be, you know, they've got different reservations. Whereas a direct creditor relationship, there's not those kind of reservations because they know what their source of their data is coming from. ⁓ so when it comes to sort of how they view sort of at least the TCPA is the first big consideration.
The other thing I would say is, between those groups is the understanding of the FDCPA and the interpretation of the least sophisticated consumer. For people who are not from our industry traditionally, they don't fully appreciate oftentimes how that gets twisted in a lawsuit. And understanding what the least sophisticated consumer really means in front of a judge.
If you are deploying the technology as some as a company like a debt buyer or third party agency and all the traditional debt collector folks are still there, they're going to build in and have oversight over those things and be looking for those things. If you don't have those types of folks integrated into your business, you're probably going to get tripped up and that's going to be that's going to be a problem. Right. And so, you know, all the all these different groups
⁓ I love when I see the tech people and the industry people really coming together and solving these problems together and being really collaborative. I think that's the way this has to be deployed because we have really ⁓ we have PTSD for really good reasons related to taking some of these risks, right?
We have experienced the class actions, the consent orders. We've seen we've seen our peers, you know, make the headlines. And we don't want to relive those days and we don't want to take those kind of risks necessarily, right? We might be okay with taking certain risks, but not jumping both feet in and just kind of caution to the wind. ⁓ I think most industry people don't have interest in doing that.
We've had a lot of starving lawyers ⁓ when it comes to TCPA suits, since the Facebook case and they're getting new life in them because of artificial voice. So just keep that in mind.
Adam Parks (43:17)
There's always a quote unquote consumer attorney looking for an ambulance to chase and given them.
No, look, and I evaluate their websites and we can already see the traps that they're setting up in search engine optimization and even within the LLMs to try and draw consumers in. I was having a conversation recently with John Bedard and some of the folks on the defense panel for ACA talking about some of the challenges that they're starting to see in terms of the pro se litigants even leveraging these LLMs and how is this going to start to right?
How are we going to handle this from the other side? But I've kind of one last question for the group here and I, I'm curious to get your take on this. one of the things that's kind of come up ⁓ over the past couple of months is the idea that we're going to have the, the bot wars. And what I mean by that is, know, we've got a bot on the collection side, the consumers got a bot that that settlement company's got a bot calling in, you know, what happens as these bots start arguing with bots and what are we going to start to see as the consumer attorneys rollout bots that are built for the purpose of trying to trip up the collection bot.
Mike Walsh (44:36)
I think it's already happening to be honest, right? So yeah, like, and, you know, you, this is where I think to what Sara's point was industry experience on the building of the tool and the maintenance of the tool really matters, right? Like, again, for my talk, the first question is ask them some collection questions, like, use some collection jargon.
If they don't know what you're talking about, ask why don't they don't know what you're talking about and who at their company does right? Like that's the easiest you want to see if it's a collection AI tool. If they don't know the jargon and the meeting move on, right? Like, like, because if you didn't build this with industry expertise, there's so many ways to get caught, but I don't, I'm not worried about it, to be honest.
Like the great thing about AI, like we, it's a great topic, but most of the compliance people that are my customers are very happy, right? Like it's the same process. Like there was something today that came across like, Hey, you texted this person at this time. And it's like, So yeah, but they texted us one second before, you know, like it's the trail is so together there are no bad days like it doesn't have a bad day like you know their team lost in the NCAA tournament and they're mad and they're you know they're just gonna start screaming at like it is pretty when you get it up and running and it it is you know adjusted to your business and how you do business it's pretty like we we do not run into this AI calls us talks our AI we don't really care right like
Adam Parks (46:02)
Predictability of the behavior.
Mike Walsh (46:30)
You know, if it's trying to trick it, it's going to trick up. Like it's a great question. And I think probably the point of the question is what does it do if it knows it's trying to get trapped into conversation not to do with its debt, right? Like that's what they're going for. So there has to, your vendor has to explain that, you know, for us, it's can you repeat that? I didn't understand. It goes again and like, I'm having a tough time. Let me transfer you my supervisor, even through text. Same thing.
Adam Parks (46:59)
Okay.
Mike Walsh (46:59)
I'm not understanding you. Let me get you. Part of the protection is locking it down, not adding more. Right? Like if you restrict it, if you ask you, if it's in the one, the Superbowl and where can I get, you know, the better banking rate? It's going to, it's going to dump them and move on. And that's get them to a person or in some cases, like if it's a repeat offender, it's just not going to take the call. Really like it's, it's, it's so there's, there's the tool has to have that protection in it is basically what you're saying.
But it's a great question to ask, right? Like what, what does it do if it's someone's trying to trick it up or something?
Adam Parks (47:47)
Sara, what do you think about the pending bot wars?
Sara Woggerman (47:49)
So I think a lot about this actually. ⁓ So I see a feature where on inbound calls or even outbound calls ⁓ where there might be a question that is asked, are you a live person or is this a bot coming from us asking the consumer, right? Ultimately. And then do you have a defense if you third party disclose to a bot that isn't technically authorized, right?
So, right? because then you've done your CYA, right? You've said, well, they said they were like, see, I've seen this, ⁓ like this is in my, in my groups. This is something we've talked about, right? Is, do you have consent? Does the TCPA apply for them using a bot to us, right? ⁓ There might be an O4 opinion that might open that up. Like, I mean, there is like all sorts of, this could get really wild folks.
So ⁓ I think that for companies, it's important that you have a stance on either whether you're going to engage with a consumer bot or not. And because the risk could be a third party disclosure risk. ⁓ How that plays out in court, I don't know.
Mike Walsh (49:14)
That's a good point Sara. That is a good point. And like, what was your disclosure? Like what was your verification question? Right? Like was it up to snuff? It's gonna be, ⁓ and to be honest, like most of our clients pick those, right? Like whether it's year of birth, last four of SOSH, like something where, is this Mike Walsh? Yes. Okay. Just to verify. Boom. And it's gonna identify on our end, it's a virtual agent.
And then how are they going to get through to the next level of that account information, right? Like there's got to be a protection layer in there. And if it is, and someone, I saw one, they're like, they just use, you know, your voice. I'm like, you can't do that. I could record my voice. Like my voice is already in AI. Like I don't have to read all my promos anymore, right? which I'm terrible at. So that's not right there. So you're right. Like anybody could do a voice recording now.
Like, so you have to start thinking there has to be a question. There has to be a question that, and basically a disclaimer that says, okay, verify this is you. And here's the question. They answer the question. And then if it's me, Mike Walsh is a bot called.
Sara Woggerman (50:12)
And if they say, if it is a bot and they say it's the live person, like, okay, well then what are you supposed to, I mean, how are you supposed to know?
Mike Walsh (50:35)
Yeah, what is your collector supposed to do, right? Yeah, you know, what is anybody supposed to do? So I think that point is either there's a fraud going on, which we try to detect those frauds, right? Or, which usually the questions identify as possible fraud and report that back. Or the person did it on purpose and wants, you know, for convenience, the bot to handle.
Like when I saw Heath talk about, our friend Heath talk about those bot conversations, the average bot inbound to an agent is 10 minutes of phone time. And there's usually never a payment because they're just negotiating the payment terms. And then they're called back and make the payment or go on your website and make the payment because they don't want to talk to you. So, you know, if it's bot to bot 10 minutes, it doesn't cost you much. Right.
If it's bot to human, man, you just spent a lot of money collecting nothing. Right. Like you got to hope your, your website to do you. Right. Like, so. ⁓ But it's very interesting too, like the scan box are going to come. They're already here too. I just talked to one today, it was fun. it's something to think about for sure.
Adam Parks (51:46)
This is where it's going to start getting real interesting. And I feel like we're going to have a whole follow-up conversation at some point in 2026 as these bots start to roll out because being able to create a bot of yourself and use it for these purposes, I think is definitely what we're going to see. And I think it's going to be an attack vector of these quote unquote consumer attorneys to try and trip some, to try and trip up a collector or to trip up another bot in order for them to be able to file these frivolous suits.
But that's probably another conversation for another day. I want to thank both of you for coming on today, having this conversation. This is quickly becoming my favorite series talking about artificial intelligence and the practical applications that we've talked about today of being able to roll these things out and getting past the theoretical discussions I think is really going to have an impact on our industry.
Sara Woggerman (52:16)
Great.
Mike Walsh (52:39)
This was fun. Thanks, Sara, for coming. Yeah.
Sara Woggerman (52:39)
Thanks for having us.
Adam Parks (52:43)
Thank you everybody for listening to Applying AI, where we explore how to make artificial intelligence work in the real world of regulated industries. Subscribe to the show on your favorite podcast platform or YouTube and find more insights at receivablesinfo.com. We'll see you all next time.
