Using AI Safely: The Entrepreneur's Guide to Compliance, IP & Privacy

[00:00:00] Speaker A: Hello, everybody. My name is David Pustolski. I am an intellectual property and patent attorney and senior partner at a law firm called Gerhardt Law in Summit, New Jersey. I'm a big believer in education and empowerment. So I created this series about five years ago to educate and empower entrepreneurs on all topics from concept to exit. I have a special relationship with this particular topic because my scientific background happens to be in chemistry and computer science. So I have been practicing for about 21 years, and AI has not been new for me, for sure. And for. And for some of you, it's probably not new either. Without further ado, I want to just welcome everybody to this month's ess. We titled it Using AI Safely the Entrepreneur's Guide to Compliance, Intellectual Property and Privacy. I'm going to let our speakers introduce themselves. I've known the first speaker, Paul Roberts, for quite a few years. I maybe lost track. I think it's probably like we were trying to figure it out ourselves. It could be. It could be maybe a little bit over 10. I've always wanted the opportunity to work with him, so I'm glad that I'm getting to do it now under this. Under this pretense. But I'm going to turn it over to Paul to introduce him. Give a little short intro if you can, Paul, or as long as you want. Actually let them know kind of who you are, where you are, what you're doing, and why this topic. [00:01:37] Speaker B: Thank you for the introduction, David. I think it's probably more than 15 years, at least. [00:01:41] Speaker A: Yeah, it could be like that. [00:01:43] Speaker B: I met David in the American Bar association when we were still part of the Young Lawyers Division. That's how long ago it was. But for myself, I'm intellectual property attorney. I specialize in software. I have worked as outside counsel for a number of years at some big law firms like Hogan Lovells and Foley and Lardner. I've worked for the government as a staff attorney writing patent applications. And most recently I was in house counsel at Geico, managing patent applications as well as AI policy. And currently I'm a director of TransUnion for intellectual property, where I pretty much manage almost all the intellectual property risks, whether they deal with new software, new trademarks, copyrights, contracts, all those kinds of things. Predominantly, I work in software. And so artificial intelligence applies to most algorithms we're using today. So if you're looking for guidance on what you can protect and how to protect it and where the wrinkles in the law is and best practices and how to actually develop programs with AI and what's involved in the governance. Glad you joined because that's really what I like working on. Oh, and locations. I'm based out of Reston, Virginia. [00:03:15] Speaker A: Awesome. Thank you, Paul. We'll come back to you later. From old friend to new friend, our next speaker, Erica Watson. It didn't take me long to realize that. Yeah, I gotta have her too. So I gotta have her on this session. And so I'm super excited to have you. It's good to really, it's good to see you and maybe take a moment and let everybody know who you are, where you are, what you're working on. [00:03:41] Speaker C: Absolutely. Thank you, David. And nice to see you, Paul. And I'm so excited to hear all the interesting things that everyone's doing in this space as entrepreneurs. I'm so excited to talk to all of you. I'm Erica Watson. I'm a global privacy and AI attorney. I'm formerly a Chief Privacy Officer. I'm the founder of MyData Mandala and the principal and CEO of Data Strategy Advisors. I've spent over 20 years inside Fortune 100 companies. Regeneron, Danaher, AbbVie Abbott, GE Healthcare. I've built and continue to build global data privacy and governance programs for organizations that are really scrambling to figure all this out right now. I also teach law, ethics, and social issues at Northwestern. I'm also interesting you mentioned the American Bar Association, Paul, but I've been heavily active. Former chair of the Science and Technology Section. I'm currently the American Bar association advisor to the Mental Privacy ULC Committee. So I've been in the boardroom. I've been in the weeds of building these systems and governance programs out. I'm in the classroom thinking about the broader implications, of course, and I bring all of that experience to the people that are attending this podcast. Entrepreneurs, as well as larger organizations that have been established for quite some time. I do that through Data Strategy Advisors and through my Data My dollar. I'm applying these concepts, these principles at an individual level. I'm putting data ownership and the benefits of it directly in people's hands. So I really just live at this intersection that we're talking about today, which why I'm so excited to be here. I live at AI Privacy ip, so I'm sure that this is going to be a very exciting conversation. [00:05:52] Speaker A: Yes, totally. Let's stick on you for now if we can. Let's start with compliance. AI compliance. What does that even mean to you? What does that phrase mean, AI compliance? [00:06:06] Speaker C: Yeah, I know for most people, you're probably like, oh, this is the most boring topic on the planet. But I think about it all the time. It's my world. I think about AI compliance and governance. We're in the most consequential regulatory moment that we've seen in a while in technology since gdpr. And I don't know how many of you are familiar with gdpr, but GDPR is European data privacy law, law that went into effect in 2018 and it set a. A standard across all European countries on how to govern personal data. Right. So it impacted everything, everything from research to development to technology implementation into what we're living through now with AI adoption. Because a lot of the tools that we are utilizing here requires data and it often requires data about people. So we need to look at laws like gdpr. But unlike GDPR in the us there's not one single law to point to. It makes it very, very hard for entrepreneurs to know what they're dealing with. You got in the eu, you have the EU AI act as well, that's in full effect now. You've got a patchwork of state laws here. We have Colorado, Texas, Illinois. They impose some real obligations on the way companies are adopting and utilizing AI. And also across border. So you got sectoral rules. You have rules that are very specific here in the US To a specific sector like hipaa, the FTC act, we have SEC guidance, and all of it has been stretched and applied to AI context that no one really anticipated when those regulations were being written. [00:08:16] Speaker D: Right. [00:08:16] Speaker C: So it's an interesting space that we're sitting in right now, and I'm excited to talk to it in great detail. One question that I get a lot is what do we have to do? [00:08:30] Speaker D: Right. [00:08:30] Speaker C: And I say more than you think and sooner than you planned, because a lot has changed. Just this year. I'm going to talk a little bit about state laws for a second, if you don't mind. David? [00:08:46] Speaker A: Yeah, please. [00:08:49] Speaker C: This year, and I just wrote a newsletter on this for August this year, we're seeing a lot of enforcement happening. That's what has fundamentally changed here in the US We've moved from having this to an era of enforcement, having new laws to an era of enforcement. And we now have 24 states that have enacted comprehensive privacy laws. And those grace periods that a lot of companies have kind of built their timelines around are disappearing. We don't have grace periods anymore. Colorado, the right to cure expired last year in December. And so for those violations under Colorado's consumer privacy law, the regulators can go straight to a $20,000 violation fine, per violation fine. Rhode island just launched their law. There's no cure, period, at all. The, you know, we can fix it when we, when we get a warning, which a lot of companies. It's an approach a lot of companies take. It's not a good strategy anymore. It's gone. And AI training practices are now an explicit enforcement target text. As we just saw, they sued LinkedIn for using, you know, consumer data to train their AI models without complicit explicit consent. So that's a risk right now. And I just wanted to make sure that this audience is aware of that. [00:10:25] Speaker A: Yeah, no, that's, that's great. That, that's great insight. I think, I think it's so easy for users of any sort of technology or app or social media to not even real that they are, they are exposing data and using AI, whether that was voluntary, voluntarily or involuntarily. And so, yeah, it's important that at some point we have some sort of, you know, federal regulation on such acts and we're not there yet. [00:11:01] Speaker C: Clearly we're so far away from it, [00:11:03] Speaker A: we're far away from that. Exactly. [00:11:05] Speaker C: That's right. [00:11:06] Speaker A: Paul, what do you think about this? This the topic of, if I just say AI compliance to you, what comes to. [00:11:14] Speaker B: I think about it from two parts for companies. I think about it from a development and product lifecycle management. What do we need to build to make sure that our software is reliable? What do we need to do to avoid bias and discrimination? How do we check our program for guardrails and security? And a lot of those things tend to be patentable things because they're hard problems to solve. I think about it from the adoption of tools for companies. Lots of lawyers, including me, have to support companies with how they're going to integrate new technologies and what are the impacts of that? How does the information you put into chat cpt, how discoverable is that? When does it fall underneath the work product doctrine? Do you even have work product in your country? Do you have confidentiality? How does that work? What controls and restrictions should be in place? Are they done by policy, by education? Are they put in through like hard software requirements that require authentications and things like that? When I think about AI governance, I think about the policies that make it manageable. You know, do I need AI governance if I'm just putting a request into copilot to proofread my email? Right. Or if I have some large enterprise tool that's going to make decisions on consumer credits based on AI Those are very different scales. And how does a business respond to those different needs and write something that needs to be adjustable as this technology changes? And to me that's, those are some of the problems I think about when you say, let's improve AI governance. Yeah. [00:13:09] Speaker C: Can I piggyback off of that? [00:13:10] Speaker A: Yeah, yeah, please, yeah. [00:13:11] Speaker C: Because I know that when you talk about governance, I've been living in breathing governance forever and more so now around AI governance. So I like to reframe it a bit because, you know, governance as you were walking through, Paul, like those are very important components to consider. But when I think about it, in a whole, it's governance is, is just a decision making structure, right, that you have around your AI, like all the, all the aspects of your AI within your organization. How are decisions being made? Who, who approves what AI tool gets used to, who reviews the outputs before they go out the door? You know, who owns the risk when something goes wrong? And I, I ask that question often. What happens? Or who, who owns the risk if it gets it wrong? You'll be surprised by the answers I get. If you can't answer those questions, then you have. Let me just flip it the other way. If you do have answers for those three, those three questions, you have a beginning of a governance program, right? Because you have, you know, visibility, you have accountability and then you have process. So I just wanted to flag that. [00:14:36] Speaker A: Yeah, I mean this. I want to ask you both a question. We can start with you, Erica. So you heard there are companies on this call today. These are company. We'll talk about tools in a second. I'm talking about companies, people, you know, entrepreneurs that have their companies that are using AI. Should they even be. Should. I'm somewhat being facetious, sarcastic, but also realistic. Some of them are probably saying, what does it matter? Like, do I even have to worry about creating a governance program to use AI? Like, how would you respond to that? [00:15:14] Speaker C: I would say absolutely right. Maybe I'm the wrong person to ask that question to. [00:15:21] Speaker A: No, I mean, I think, I think all of us would probably agree with that answer. I mean, you gotta give a, something you have, you got to give an S about this. Right. At some point. But why, why, why do you. [00:15:34] Speaker C: Yes, well, we've seen, we're seeing a lot happen, you know, with the, the lack thereof, from small two people, you know, run organizations to Fortune 100 organizations. They're all dealing with the same core issue, which is, hey, I'm building or implementing or adopting this AI tool within my business and it impacts People, is it making a decision that will be wrong that would impact those individuals socially health wise? It's very important to make sure that you have some controls around, well, what's getting adopted? How is it interacting with those individuals? Are those outputs biased in any way? So these questions should be governed with some type of program so that you can number one talk to any regulars that's asking a question about what did you put in place to mitigate these risk? Because all of you, if you're everyone that spoke in the beginning of this call, you are regulated, every single one. So if a regulator comes knocking on your door, you need to be able to demonstrate that you have thought through these concerns, these risks and that you have something documented and that is actually implemented to demonstrate that you have started to mitigate those issues. Even if you, even if it pushes something out that's wrong and someone is impacted, you can at least tell the story that we started to put in place these controls. That's the biggest piece about this. And I think everyone on this call, there's a starting point for everyone. You don't have to go full blown AI governance program, but like I said, those three questions, you can start to build from there. Small, small steps. [00:17:39] Speaker A: I love that. Paul. You, you, you, you build these types of policies for companies. So, and, and so what is it? So same question. Should, should, should companies care about having an AI governance policy? [00:17:56] Speaker B: I, I think it depends on the company size and the impact of. It's easy to stand here and say that you need AI governance policy and you need 50 other policies for your five person company that you know a Fortune 100 company has that may not be realistic. Right. Even to understand and then to build implementation for it. But I think you need to consider what are the risks of it and how important it is to your company. There are certain small companies where it's very important. If you own a small news company and using Gen AI to write news, it's high. You know, if you run a gym might not be as high. Right. If you're just using it for social media marketing, right. So you make mistakes on your social media marketing, what's the impact? It could be high. You could infringe somebody's trademark rights because it's because of the AI. Right? But you know, I think it's, it's a scoping thing and part of it is, you know, looking, looking at what's, what's the worst that can happen, where are the risks, how much human review is involved in your process and how much Surrendering of Gen AI is there where we don't, we look at it, but we don't really review it holistically. We just accept what the AI is doing. The more that needs to happen because of capacity and volume, the, the higher the risk would be. I think most companies are struggling to find the right AI policy. There's not a lot of history. You have a lot of so called experts trying to tell you that they have a crystal ball based on their 20 years of gen AI experience. I love to see on LinkedIn as to what it should be, but a lot of, you know, there'll be a lot of, I guess, you know, corporate mistakes along the way and, and you just have to spend the time to review and update and, and collect information from all the, you know, stakeholders involved so that all the risks are being managed against the amount of work it is to, to check them. [00:20:12] Speaker A: Yeah. Also great insight. Well, let's flip that script a little bit because here we are putting, we're putting onus on the company that utilizes AI to ensure that they have policies that govern. And I get that and I think that's important. But what about the people that are creating these AI tools, right? What onus is on them? Right? We've heard there was a few in the, there was a few in the introductions. People are creating awesome AI tools that they're selling to these companies to help them govern. Right. To help them be compliant. So I'm just curious, Erica, we'll start with you first. What onus should be on the people that are creating AI tools to adhere or eliminate some of the, some of the issues that we're talking about here. [00:21:05] Speaker C: So for those that are building. So it's a little tricky question, right, because I think a lot of entrepreneurs here on the call, they're building on top of existing AI solutions, right? So there's a really delicate balance here. There are true deployers of AI and then there are true developers of AI. And I think that you could be a combination of two, which I think a lot of folks here are you are bringing in or you're building on top of Claude or you're building on top of, you know, OpenAI. I'm sorry, Chachi PT. So you have, they have very great legal teams, right? They under. And they've also seen a lot of issues come their way already. So they have in their terms of use very, very clear language about what they're liable for and what they're not. They are not responsible for the output. [00:22:15] Speaker B: Right. [00:22:16] Speaker C: And they want to make that clear? So if you're adopting, you know, or developing a tool that is utilized or being built on top of one of these, you need to ensure that you have the ability to monitor any output. You have to make sure that you're communicating appropriately to your users. What are they getting? What is the expectation? What are the risk and liabilities associated with their use of it? So I think just those basic principles of understanding what it is that you're obligated to do and what risk you're taking on by bringing in an additional, Bringing in an AI solution to help build yours versus you being the true deployer. I don't know if we can do hands in this session to see how many attendees are actually building their own AI solution, not building on top of a tool that's already existing. That'll be really telling for me. [00:23:21] Speaker A: I see Alex just raised his hand, so. Alex. So, so, so, so, so maybe raise your hand if you're a, a, a, [00:23:29] Speaker C: a versus a developer, right? [00:23:33] Speaker A: Deployer rather than developer. If you can, raise your hand. That's interesting because I agree. I mean, we see that from an intellectual property perspective, that's something we always kind of have to look at. Like, exactly what are you truly building that's novel? Or are you using open source to do something and that becomes the, you know, the, you know, line of demarcation in terms of how you can protect the thing that you're building. [00:23:58] Speaker C: Exactly. And you can't, you can't just say, you know, they don't get to point out that OpenAI or anthropic is to blame when something goes wrong. Like I said, the terms of service, the safeguards, the human review process are important because that's really where the liability defense lives. You need to have those elements in place in the very least, if you're building on top of an existing platform. [00:24:28] Speaker A: Yeah, good point. Paul, same question. What do you think? I don't know if your current companies use outside tools or you build your own tools, but if you use outside AI tools, how much, how much are you expecting that tool provider to take responsibility for, for, for, for things that might go wrong? [00:24:54] Speaker B: I don't think companies rely on that very much in multiple areas. And AI isn't any different because you're ultimately liable to your companies or to, to your clients. So with, with AI tools, whether that be a tool that helps you with voice processing, like what Sierra makes, or if you're talking about a tool that optimizes output so it takes less tokens when you send it to Cursor or wherever you might be sending it to. I don't think companies give a lot of confidence in it. They certainly negotiate it a lot. Those type of things are in every contract where we're negotiating enterprise liability and what's your fault, worse, our fault and things like that. But they are still doing the best to take ownership so they don't have to. Even if you have indemnification, ultimately is what we're talking about, you still have to sue to get it. You got to spend money to get it. And it's a risk. Right. And it's business time. It's work. It's, you know, creates loss and focus on other projects. So I think you, you own your own outputs when you're doing it. Maybe somebody's responsible for you. If that company's much bigger than yours, I wouldn't assume that you'll get anything. [00:26:30] Speaker A: Yeah, that's a good point. Let's, let's stick on you for a little bit and kind of talk about intellectual property if we can. I mean, we can, we can start with, you know, patents and, and kind of how it relates to artificial intelligence. I would say not a week goes by that I'm probably not engaging with an entrepreneur to protect their AI solution. But for every person that I'm speaking to, there's probably about 50 others that are creating AI solutions and maybe not getting it patented. Know your thoughts on that. That in general, the patentability and or even need for patents in this world of AI. [00:27:23] Speaker B: Well, the second question is easier. Let me start with that one. The first one's a little difficult to answer. So if you're thinking about your own company as to whether or not you need patents, you think of it. If you have a smaller company, I think of it in two directions. One is, are you building your company to eventually sell to a larger company? And if that's part of your business strategy, patents have some value, for sure. They are one of the assets that will be looked at. And it is part of an argument why you might have a product that might be different than others. If you're thinking about that, you want to get patents because you want to be able to enforce your rights against large companies that might be making your products in the future. Your company needs to be making a lot of money to do that. If you want to sue a company like Apple or Google because they're violating your patents, you know, legal fees could be 10 to 20 million dollars to take it through trial and they won't bat an eye at it. And you'll need to defend countersuits and things like that. A lot of inventors really take personal ownership over what they come up with. That's why I became a patent attorney. I get it. But it's not that easy to enforce your rights against larger companies, much like it's hard to do a lot of things against larger companies. So that's a thing to think about when you budget how much money you're going to spend. If you are going to get patent rights, hire an attorney. It's basically impossible to get something valuable on your own. You might get something through, but patent attorneys will see all the mistakes and it's just going to cost you a lot of aggravation. You know, I would budget it. Costs can vary a lot, but it's probably somewhere between 6 and 20,000 US dollars to file an application on AI and then budget another 15 to 30,000 to get it through to grants. And then keep in mind you'll pay publication and taxes on it every year while you have it enforced too. So they're not an inexpensive thing to get. Your first question about the patentability of AI, it's largely the same as software. There are basic principles that are not patented. That's what the patent office will tell you is an abstract principle. And then there's the specific application, right? How you integrate your new advances in using AI in a specific application. This is also called technical features in some countries. The level of detail and depth you put that in is what's important in the U.S. ultimately, you have to make the argument that you're improving the computer or the algorithms behind it. And you have to give that disclosure so that somebody else could make the same thing. Having read the disclosure, that's really the standard. And it, I think of it, it's beyond what the sales pitch would have. It's beyond what marketing would say. It's what your computer science friend would want to see, to know how it works. That's what you have to have to put in there. And if the complex part is being handled by Claude or whatever, you might not have anything. Right. But if you're the one who's programming it, then you might. Does the prompt matter? Yes, the prompt is part of your instructions. If you have patentable sequences in there, you can get a. Get a patent. I'll give you an example of something that I built that I decided not to patent because I'm not going to sue anybody with it. But I don't know how many competitive card players we have here, but I play Magic the Gathering and I built an AI system that can automatically build starter decks from your huge inventory. And it applied many different rules across many recursions to figure out how to balance the decks automatically. Could I get a patent on it? I'm probably sure I could. Am I going to sue White Wizard Games with it? No, hopefully not going to do that. Maybe I get a patent and offer it to them, but I don't think it would get me anywhere. But for me it was great because now I have lots of starter decks and I can play the game that way. And you just think about how complicated it is to solve a problem, the specific steps and how unobvious they are is really what's kind of controlling ultimately to get a patent on an AI system. [00:32:09] Speaker A: Love it. Yes. You mean you just confirmed. Which is like. Yeah, some do, some don't. Right, for sure. Some do, some don't get patents on AI. Erica, your thoughts on just intellectual property in general when it comes to, to AI? [00:32:27] Speaker C: Can you hear me? [00:32:29] Speaker A: Yes, we can hear you. [00:32:30] Speaker C: Yeah. No, that was, that was so insightful. I think that. Yes, because it's, it's, it's the balance of. Well, should I patent or shouldn't I? I've been thinking a lot about, you know, what's, what's been happening with Fair, Fair use AI models and fair use where like New York Times, the Getty Images, right. They're working through how they scraped copyrighted material and they're feeding their model and then entrepreneurs like the ones in the session are building on top of that. Again, I think for me, where my mind goes is the fact that we need to think about building AI powered products and how we need to understand what the vendors models were trained on as well and whether that creates any exposure from a downstream stream perspective. I wonder, have you, have you thought about that, Paul, is that. I'm sure you have. [00:33:45] Speaker B: Absolutely. I thought that's great. Wasn't part of David's questions he asked me about patents. But let's talk about. [00:33:52] Speaker A: But yes, let's actually transition to Copper because you bring up the Erica. That's hot topic right now for sure. Yeah, please, Paul. [00:34:00] Speaker B: Well, I tell you what the law is and where I think it's going and why and what my personal opinions are. And there's, there's two big parts, three big parts to it. The one is what you do to copyrighted material with your AI system, what you build with it and whether you can protect what you built with it with copyright. Those are, I think the three spectrums Right. And there was three major decisions last year looking at this decision of like reading copyrighted materials. Right. And then training systems with it for different reasons. The course that that looked at this, one of the big parts of whether or not it's legitimate is whether you legitimately had access to it. Did you kind of break into somebody's systems or use unauthorized means to get access, or did you just buy the materials, obtain them somehow, and then scan them? So that was kind of a big factor. But assuming you have an authorized way of getting the copyrighted material, then training an AI algorithm is fine on it, provided you don't sign some covenant not to, which will be in all your agreements now, because the lawyers have learned that cut rate won't work alone. And that's where the doctrine is. So there needs to be some covenant blocking you from using AI on it to basically train the different models or use it as examples. And most commercial databases now will have covenants in there preventing you from doing so, or they'll charge you extra if you want to train your systems. When you use a model, whether it's a commercial model or something you built to produce new material, can that new material infringe somebody else's copyright? It sure can. It's not a defense that you use AI for it if it's too similar. If I use any number of gen AI tools to make, you know, a picture from my website that looks just like Bugs Money, I'm sure Warner Brothers will eventually send me a letter about that. Right? And it won't matter that I didn't access their materials directly. I use an AI system to do it. Those companies are also suing those, those companies for, for generating those tools and this kind of contributory copyright infringement. And then as far as whether you can control what you build with copyright, if it's AI, if kind of the major creative component of it is built by Geni, you'll have little to no copyright protection on it. But if you build the core first and you use AI to improve the lines or maybe to shift something the copyright, the current policy just kind of has you disclaim the parts that were AI made versus what was made by yours as a human, essentially. And it's no different than you use Photoshop to improve the focus of the image, that part wouldn't have been covered, but the original image that you made would be. And patent doctrine is, is similar. You have to come up with the idea. Right. And you have to have conceived it. Right. Which is a legal term you can use AI to help you write the code. Right. To do that, you can still get a patent. If you're doing that, you can use AI to help you write the patent disclosure to do that all the time. [00:37:47] Speaker A: Yeah, Erica comments on that. Or if not, then I have a question for you. [00:37:55] Speaker C: Yeah, no, go with the question. I'm learning from, from, from Paul here on that, that point. I just do want to say though, one thing that I'm often thinking about when it comes to partnership with IP attorney and also with deploying AI within organizations is ensuring that the staff, the individuals that are utilizing the AI understand that they can expose the organization by putting in company confidential information into these tools. Right. I think that that's something that is not always top of mind that these tools and we can, I think we're probably going to go into it in more, more context. So I won't go deep deep, but I think it's important to always think about is it okay for me to put this information into this public tool. And am I, am I, am I, you know, exposing our ip, our protections as a result, A result of that? [00:39:06] Speaker A: Yeah, no, that's, that's exactly, that's, that's exactly where I want to go with you. I think that because that, because we talked about patents, we talked about copyrights, trade secrets, which is also a, a form of intellectual property, you're right, could be exposed if you put it into your own AI or if your law firm is using AI and hopefully that that's not happening. But, but yes, to your point, how does one protect their proprietary information when using AI tools? How do you navigate that? And again, I really don't think people really understand like, like there's, so I will have clients will come in and be like, oh yeah, I just ran this, this potential, you know, I'm not sure I want to file a patent on it. So I may still keep it a trade secret. I just ran it through chat GPT. They say I have a good chance of getting a patent and then they come to us. What did you just like expose this to the world by putting it into chat GPT, Especially if you have a, you know, free account and it's not enterprise. Yeah, that's exactly what you did. And that's terrible. So how do you, how do you navigate that? [00:40:16] Speaker C: No, absolutely. And that's, and I'm, I'm laughing a little bit because it's, it's something that I feel lots of people do as well as entrepreneurs do. They, they want to get a objective review of maybe Their business plan, right? Or is this a good idea or should I how could I improve this? And you're, they're putting it into Claude or ChatGPT and it's a public model. One thing that I, I really encourage and it's something I've been on my soapbox about for a long time. You want to read the terms of use again, you want to read your, the contracts on these sites to understand what are they collecting, how are they using it. And because they are obligated to tell you, you know, what they intend to do, they will say they will give you the opportunity to toggle controls off for the training. [00:41:16] Speaker E: Right? [00:41:17] Speaker C: They'll say, well, you can toggle this off to not use my data and my inputs to train the model, but in the terms of use it will say, well, we will use this for, you know, business development purposes. Any of your inputs will be used for business development purposes. So that means that they can still use it internally for the improvement of the model. And again, there is no protection. Your trait, your, your trade secrets are gone. Your, you. There is an organization, there was a company I just remembered that was going through selling, selling their, their business and at the very last stages when they were trying to understand, you know, what protections they had, they learned that everything was put into jet GPT. There were no protections and so they lost the deal as a result of that. So I think it's really important to, to be extremely considerate about what you put into these tools. Unless you have your own, you know, closed model. Right, that's one way to go. It's very expensive, but I think that we discussed at the very top of the hour. But, but it's something that you need to consider as an entrepreneur. What are you exposing yourself to? How can you protect yourself? How can you limit or change your questions a bit to further protect yourself more? Generalize your questions, I would say to these tools and strip out your identifiable information. [00:42:50] Speaker A: Yeah, 100%. I mean that also goes just for basic, you know, personal identifiable information and stuff like that. You'd be surprised what people put in for sure. Paul, thoughts on this? [00:43:05] Speaker B: Yeah, I can add some things to that. First, maybe to disambiguate what a closed model really is. Nobody on this call has a closed model that is essentially you have built your own AI algorithm and then you have Fedramp server farms to run these models. You may have a license from one of the different companies that run these things, but to have a model that you can actually do anything with, that's going to be competitive. It's just outside of what small companies can do. It's way too expensive. There are plenty of free models available and you know, Llama made their ones available, but either they won't run very well and so you're getting like Gen AI from 2022 or simply to run it requires amounts of server power that you just, you just can't get. It just doesn't work at scale. In fact, it's no secret that even the large companies are losing money running this stuff. Right? They can't run it. You certainly can't run it. So you don't have a closed model, but you may have a license which says something like the company agrees not to use your data and you know, as and, and build it into the model. Right. So they have to kind of keep it confidential that way. So depending on which model you might be working with, you know, if you asked the model whether your invention is patentable, which I don't recommend you do for other reasons, but if you did that, is that a disclosure depends on the term of whatever the company may have to keep it confidential, in which case probably not. You're not going to get great information from that. It can basically tell you yes or no and it's probably not going to be the answer. So I don't really know how useful it is. You know, privilege and communication is also, you know, not 100% clear on those type of things. It might apply in some countries. So which country you're working in makes a very big difference with this kind of thing because it's very, you know, law dependent of, of whether or not, you know, a disclosure like that would count against you. For trade secrets it is a lot like, you know, putting personal information into your search. When you ask the AI system to answer questions for you, it sends out searches to Google and other type places and those are really not confidential. It's going outside the system. If you have MCP connected, it goes to other, other software systems and so those can reflect what's in your documents. So you might be losing confidential confidentiality that way. And just like you have to be careful what you put into search. It's really the same principles with AI. Yeah. [00:46:08] Speaker A: Thank you. Those are, those are excellent insights. I hope people take heed of all of everything that Erica and Paul are saying here. Let's talk about ethics, the ethical use of AI. What are you, what are your thoughts, Erica, on that? How does one use AI ethically? [00:46:33] Speaker C: Boy, that's broad. So I just, I think on the Ethics, I think about ethics and bias together. AI bias is, is not an ethics problem per se. That becomes a legal problem. It's already a legal problem. Right. So, and I say that because we already have laws. We have, you know, fair lending violations, we have employment discrimination liability, we have, you know, all these laws that are in place to help protect against those risks already. So that's why I say it's a legal problem for the audience. If you're. So say, for instance, if you are an AI hiring tool and you're, you're screening out a protected class at a higher rate than, than other classes, that's a civil rights issue. Right. So, so I think when it comes to ethics, it really, we really have to know ahead of time by testing the system to determine whether or not it's fair. Right. It needs to. If you have a tool that you're developing or you're utilizing a tool within your organization, you need to ensure that its output is fair. And that's through monitoring. You wouldn't monitor the outputs. You need the human in the loop. You probably have heard that term a million times. You need a human in the loop. I'll give something specific. Healthcare. I've been in healthcare, pharma, biopharm, med, device, for over 20 years. And so I think about bias and ethics in that context a lot. I teach on ethics, so I think about it all the time. And I think about harm and how harm can impact marginalized populations. Like we've seen with some AI tools that have been deployed specifically in dermatology that performs. These AI tools have been known to perform worse on darker skin tones because the training data was predominantly lighter skin tones. You know, it really goes all the way back to the beginning of your. From the very start of. Well, what is the data that we are looking to utilize to train our model? Is that ethically situated? Have we thought through the impacts appropriately to ensure that this won't misfire and push out something that is biased? Right, so. So that's how I think about it often. And I really encourage all of you to, to really take the time to look at your outputs. [00:49:29] Speaker A: Yeah, that's excellent advice for sure. Paul, what do you think? What are your thoughts on bias, transparency, ethics? [00:49:41] Speaker B: I think, I think they're very different. Different questions. Right. So the ethics, I feel like we as a, as a country, in most countries have a lot of, a lot of work to do with whether or not there are required disclosures involved and what happens if you don't disclose them. Personally, I feel like it should be disclosed. I feel like anytime you have a service and you represent it as being powered by a human and it's powered by AI, there should be some disclosure, but not. But there isn't the laws yet that, that require that bias and discrimination. There are a lot of laws that control that and companies have been sued. Erica mentioned some of the fact patterns behind those suits about you're in protected classes and things like that, whether you're hiring or you're looking for housing for somebody. So those are certainly cases and the law as subtle as could be as well settled in that way. It doesn't matter that you had an AI do it. That's not going to insulate you. It's also not going to insulate you that you didn't realize that your vendor provided this as well. We didn't do it. We didn't know. One of the wrinkles or challenges that I encounter when I write contracts with vendors is, you know, vendors don't want to tell you how the software really works because they want to keep that confidential. But we need to assess risk. Right. And so there's this tension behind a company that wants to use vendor based technology and the vendor that doesn't want to share it. Right. And working through that so that you have enough information or maybe you have to pick another vendor that's willing to provide that so that, you know, companies can, can manage their, their risk of bias and discrimination. [00:51:43] Speaker A: Yeah, I love, can I. Yeah, I [00:51:46] Speaker C: just wanted to add something on the, the legal side of things because I'm in health care for the most part. You know, it's something that flagged to me recently. So Texas, we're starting to see, we're starting to see, you know, the transparency shift from a best practice to, to a legal requirement because Texas now requires physicians using AI for diagnosis to personal review the output and also tell the patient that AI was involved. [00:52:21] Speaker B: Right. [00:52:21] Speaker C: And then California, you know, California is always at the front of this stuff. They, they passed a bill, AB3030. It says if you're, you're using generative AI and a patient communication, you need a disclaimer and a clear path to a human unless the licensed provider already reviewed it. [00:52:41] Speaker B: Right. [00:52:41] Speaker C: It's kind of a backwards way. So I just wanted to flag that there. We're starting to see some legislation requiring transparency around AI use. [00:52:52] Speaker A: Yeah, I love that actually. And this is an important question I think for both of you. Both kind of have danced around, but I think it's important. But Paul will start with you. So if you are looking to bring on an AI vendor, what other things should you be looking for in those types of contracts? Like, I mean again some of it may not be negotiable. We get it. But what could you like, what should companies be looking for? What should what, what should entrepreneurs be looking for? [00:53:26] Speaker B: So many things, those, those contracts are, are, there's a lot of things that they negotiate. I wrote 120 page book on how to negotiate these contracts. But where do you look? Terms of use are one of the first things I look at. What's allowed, what's not allowed. You look at identification that we talked about earlier. You look about confidentiality. You look about what happens when the agreement terminates and the party's ability to kind of keep it a secret. You look about one of the big things that gets negotiated in IP is the outputs. Right. And who owns the outputs and who can do what with the outputs and what's considered pre existing technology. Before you started working with this vendor where you know, dealing with issues. For example, let's say you're working with a company and you're, and you have them involved and you're working with them to help you build this new product and your engineers are building it and their engineers are building it and then like who owns a new thing? You don't have the good agreement, you both own it, you have a joint license most likely and you can both license it, which is probably not the outcome that. Yeah, that's right. So you know setting, setting those assignments properly to reflect what the businesses really wants is an important goal of it as well as making sure that you know, if you're bringing in vendors. One of the recommendations I give companies is file on file a provisional on what you know already before you get contaminated by their information. Because most likely there'll be some kind of confidentiality agreements in there that will prevent you from doing stuff with it unless you already knew about it. Those same confidentiality agreements can set up something called vendor lock where you learn how to build these new tools from working with the vendor. But then if you go ahead and try to build them your own, you're basically, you know, misappropriating their trade secret information and you can get liability for that. That so making sure that those clauses are, you know, have protections and for you, and then you take good steps to make it clear what you know versus what you might have learned from them are important steps I think in negotiating those contracts. [00:56:06] Speaker A: Yeah. Love it. Erica. Did he miss anything? [00:56:10] Speaker C: I. I'm just going to Add two things because I agree with everything you said. I think. But the liability piece is so massive. It's so massive. And I think you want to think through all the scenarios where you're not left holding the bag at the end of the day. [00:56:25] Speaker B: Right. [00:56:26] Speaker C: But the two things I want to add on is data use and training provisions and I alluded to this earlier because you need to know if the data is going to be used to train their model and can you opt out of it. Right. So that's something that I want to make sure that you guys get. I agree with also the termination. Termination piece of it. But one thing to add on is when you terminate the agreement, do you have to take an action for them to delete your data or what is it automatically going to be deleted? Who will be notified of that? That's important. And then audit. I'm sorry I lied. Three audit rights. You want to make sure that you have auto rights. Can you verify what they're doing with your data? So that's, that's all I have to add for that one. [00:57:19] Speaker A: That's awesome. Yeah, no, that's great. The two of you were perfect. I did chat. If anybody. We're at the hour. If anyone wants to ask a question live, please raise your hand. That's probably the best way to do it. I think we may have covered some of the question questions. Well Paul, people want to know what the name of your book is, so maybe chat that and there was S fire's hands raised. But we'll get to that in one second. Chris Southgate has a. Has a question. I. I'll give it to both of you if, if you can answer it. It says what about the generative and AI component of word add ons like Grammarly, could the input be discoverable? [00:58:13] Speaker C: Absolutely. [00:58:15] Speaker A: Yeah. [00:58:17] Speaker D: Yeah. If you. [00:58:18] Speaker B: So I can make it simple for you. Discovery depends on the country, right? So people talk about discovery. They think in maybe us. It's very country specific. Some countries don't even have discovery. [00:58:31] Speaker C: They sure don't. [00:58:32] Speaker B: Let's. Let's say that you're in the U.S. yes. Let's say that you're not a person who has privilege in your work. Like you're not a doctor, you're not a chiropractor, you're not an attorney, you're not one of those people. Do you have any privilege? No. It's discoverable if somebody sues you. If it is relevant to the case, it's coming in for sure. Doesn't matter if you paid a license for the stuff or not it would come in. [00:59:00] Speaker C: Yeah. [00:59:01] Speaker B: If you are one of those professions that have still might come in. And it's going to really depend on whether or not, for example, for the work product doctrine, if the material that you're preparing was intent in anticipate in anticipation of a lawsuit, then maybe would qualify for work product. But if you're just kind of building it, generally probably not. Will it count as attorney client communications? Well, I mean, depends what you're doing with it. If you're just building it for yourself, probably not. Right. If it's to prepare an email for a client, then maybe it would. Right. And that's probably where the courts will go. A lot of it is unsettled at this point. But what people said before, if you're doing it with an unpaid model, then probably it's not confidential enough. And of course would look like that you waived the privilege by using that. So that I think how it will come out, maybe. [01:00:00] Speaker A: Can you hear me? Yes, Chris, go for it. This is Chris, who asked the question. Yeah, maybe a workaround would be to use, let's say Microsoft Copilot, which is [01:00:11] Speaker B: within enterprise Plan, which is more confidential. [01:00:20] Speaker A: I'm an attorney, sorry, I'm asking question again. [01:00:25] Speaker B: The question is if you use a Microsoft Copilot at an enterprise level which is integrated within the fabric of Ms. Microsoft Office, that would be. [01:00:43] Speaker A: Would that be safer? [01:00:46] Speaker C: You mean for discovery? No. [01:00:48] Speaker A: Yeah, for discovery purposes. No, it doesn't work. [01:00:51] Speaker B: It doesn't work that way. Yeah, it goes when you, when you use Microsoft Copilot, it goes up to the same servers that other places do. It's not run locally on your computer. So you would treat it just like any other AI. AI technology. So how, how are the larger law firms dealing with this, just out of curiosity? Well, they have AI governance as to what they can use and when they're using it and, and they're being very deliberate with their decisions. Right. And if they, if they are going to use it, they're going to make sure that depends on the country again. But like in the U.S. for example, you know, a lot of times you're doing work for a client, it's because it's an anticipation of litigation. That's why you hire the biggest firm. So work products will often come in then, or the law firm is preparing this to give information to their client. Right. And so then it would be an attorney client privilege that might attach to it. So I think it's still fuzzy, right? [01:01:56] Speaker A: I mean, it's still. Still fuzzy. Yeah. It's not guaranteed. [01:02:00] Speaker D: Which is. [01:02:00] Speaker B: Yeah, it's not a guarantee. And the rules. And the rules are. They do matter. Right. And. And it's not going to be because especially if you're doing state law, there's different variances on how this stuff works. There's different bar rules. Right. For what you can use not to too. So it is very specific and that's why you don't see general guidance. It really can't be provided. It's going to be specific to your jurisdiction and what licenses you might have. [01:02:28] Speaker C: Thanks, Chris. [01:02:30] Speaker A: Yes, thank you so much. Sfire, can you. Can you ask your question? Oh, sire, I don't know what happens. [01:02:45] Speaker D: Hear me? [01:02:46] Speaker A: Yes, we can hear you. Go for it. [01:02:49] Speaker D: I have a couple of questions first are. Is there going to be a replay of this record recording? Because I missed some things and I've been in and out. [01:02:58] Speaker A: Yeah, there is. As long as you registered, you will receive the email recap and the recording and the links to the podcast. Yes. [01:03:05] Speaker D: Okay. The other question I have is that I am. I'd like to build a crowdfunding platform and I have a mentor. I did some research, a little bit about AI and considering it to fintech, you're collecting data, you're dealing with money. There's a lot of a host of things that you are dealing with. I. It sounds to me as far as building it, what I've learned, I don't know. You can correct me if I'm wrong. You, you want to build a foundation off of humans and you can use AI as an, as a obviously tool, but. But not as what you. It sounds like what, what everyone is saying is that, you know, these platforms, you got to read the terms and conditions and what they can take and what they can't take. So I guess I'm kind of moving away. I don't. I'm concerned about that. So I explained that to the mentor, but they like, they're encouraging me to, you know, it's change. It's not, it's not like that. There's a lot of things that, that has changed, but my gut is still saying I don't feel comfortable with doing that type of platform on just AI. And I don't know these softwares out here that are going to safeguard people's data and all of that. So how do I. Is there. I mean, what they're saying, is this true? As far as AI is concerned, it sounds to me there's a lot of cons and there's a pros. There's Pros. But for this particular situation, I just want to make sure I'm going in the right direction. [01:04:53] Speaker A: Well, actually, this is a good question, Erica. Yeah, maybe you wanted to. Because basically we're talking about best practices here, so. Yeah. Erica, maybe you can, you can address. [01:05:01] Speaker C: I was just gonna ask you're. You're referring to prompt to code, right? Solutions. [01:05:09] Speaker D: Well, I, I just, I'm, I'm built, I'd like to build the platform and I need help to do it. So I'm just trying to find the best direction. Do I build it with AI because that's what the mentor is encouraging me to do, or do. [01:05:25] Speaker C: I can build a prototype. I wouldn't put any real data in there, you know, to start off with, but I think, you know, as you're building out, just the, the tool itself, the concept, I think using a prompt to code solution could be useful. But once you, before you put any real data in there, you want to work. I think I know Paul and David. You want to work with like a developer, someone that understands the risk associated with using highly regulated data, making sure that you're getting the right vendors to store that information, helping to negotiate those agreements, also making sure you have the right advisory board as well to help give you some guidance around some of those risks. So that, that's my input on that. I think, I think as an entrepreneur there, you want to be careful, especially if you're stepping into fintech. [01:06:29] Speaker B: Yeah. [01:06:31] Speaker A: Any thoughts? [01:06:32] Speaker B: Just to add to it. I know and I saw this from, from, from companies, you know, two years ago or two and a half years ago, and they wanted to put their heads in the sand and say, we don't want to take these risks, we don't want to use AI. There's lots of risk. And the lawyers rep, there's risk everywhere. And like we won't use it. And then the business planners were like, we will be out of business if we don't use AI. We must use some AI. You can't just say there's risk. We know there's risk in everything. So would I tell you that you should be developing new software systems where there's coding without using AI? No, it's, you're not going to be competitive. You can't do it as quickly and there's more risk. That is, that is the life that we're in right now. And you have to balance the risk based on what you're building. Right. And go through some of these things that we're talking today. Even at the legal practice, you should be checking your emails with AI if you're going to clients. That's expected. Okay. You should be be confidential. Right. And where that that applies. Right. That those are kind of the different standards that are changing and there's a lot of gray area. Right. But that's always going to be facts specific. [01:07:52] Speaker A: Yeah. Great advice from both of you. Thank you for your question as fire. Jimmy. Jimmy. [01:08:03] Speaker E: Hi guys. How are you? This is my first time attending this kind of meeting. I was. My question for Paul and Erica and all the team over here is that I'm working on a product. It's still coming up. I'm still at the finishing cycle of mvp, but it's more like a vendor marketplace. And before I launch it I always have curiosity whether I'm doing something right towards the launch because I'm always scared if I. If there is a AI and if there are all the, all the keys and all the connections there for the APIs and everything. And then as soon as I launch it and I'm like I'll be get my product will get hacked and then there will be like. Because we. Because I used AI to build it. But the idea is that if, if I launch it and then on the second day I get like a lawsuit from somewhere like oh, you did something wrong. And I'm like I'm just trying something out. And I did not even know what are the. Whether I'm I was doing the right thing or maybe just a knowledge gap somewhere where I cannot bring it to life or just sharing with friends whether you can try this product or not. And there's always like a fear whether I should do it or not. So I built the whole product but I, I'm always scared like whether I should launch it as an app, whether I should take advice. Even the AI can build you privacy and compliance policy. But still I feel like oh what if something is missing or what if something else is needed as well to just as a precaution for myself because I built it and I'm still somewhere worried whether I should launch it or not. So I just need a real advice from you guys whether I need to. My husband says that I should go to venture capitalist, but I don't know who is the venture capitalist who will guide me in the right direction where I need to take the next steps. So I just need your advice. [01:10:36] Speaker C: I just gonna. I'm just gonna say security. We didn't talk about security. And AI and ip we didn't talk about security. But I will say before launching any product that has, again, it has to do with data and people and business information. You want to make sure you have the right security controls. And there's also regulations around this too, depending on the data type. So it's very important to understand, well, what am I obligated to have in place for security? Also, you want to conduct testing, pen testing, to ensure that your environment is secure. And that also goes with who you hire to store the data. Is it aws? Is it Salesforce? Who are you using to. [01:11:30] Speaker E: Using Microsoft Azure for storing all those? [01:11:34] Speaker C: Right. So you want to look at their terms, but again, you're still on the hook to ensure that the data is secure, so you want to conduct your own testing. So that's something I just, you know, I wouldn't say I'm not going to do it because I'm nervous and scared that someone's going to hack into the system. I would just say do what you're supposed to do to meet the legal obligation and also pressure test it for risk, because breaches are going to happen. They happen every day. It's about whether or not you are, you know, doing the right thing and you've done your best to demonstrate that you have mitigated the risk. [01:12:18] Speaker A: Yeah, well said. Paul, thoughts? [01:12:21] Speaker B: Advice in a different direction, which circles back to some of the original advice we gave. When you're using Genai to code or to write, you as a human in the loop need to have the understanding and the capacity to check it, right? If you are a developer and you're using AI to write some code for you, you need to be able to read that code and understand and make sure that doesn't have security vulnerabilities and who knows what other problems it likely will. You know, if you're an expert in the field of whatever this is, and you can look at the code and be like, okay, yeah, that's not going to work, that's not going to work. This is right. That then the risks are low. But if you can't understand it because you don't have the training in that field, you're hoping AI is going to do it right. And it probably won't, despite the commercials that you read about. Oh, well, you just take an idea and you type it in, then, boom, the product's launched. Right? That's what they want you to believe. That's not gonna. It's not gonna work. The amount of errors are too high and, and they're hard to see because it will look superficially fine unless you're an expert in the field to, to catch it. So you know, you have to take that as, as ultimately the answer. Look at your software. How much of it could you have written yourself? Can you understand everything and understand what those security vulnerabilities can be? And they're a lot. The AI development community knows this. There's all sorts of things that they're like, oh my God, you just put the key in plain text right in the file, like what we. And it just. Well, you didn't tell me not to do that. That was the most convenient place. And it goes on and on and on with stuff it might do. Unless you build in some framework for it that you can teach it to do all those things. But even then, sometimes it doesn't do it right anyway. Despite that you having guidelines and guardrails. So you have to be able to have the capacity to check it. [01:14:17] Speaker E: So who will be the right person to take the steps forward? I'm a developer by nature, but I believe that I do have a knowledge gap somewhere versus working for a company where the infrastructure is already set up and trying to build a new product with an infrastructure where something might be missing. And whether I should hire an architect, [01:14:48] Speaker C: a fractional ciso, someone that, that's aware of security, that has worked in technology. They could help to review your code as well as the infrastructure to determine if there are vulnerabilities that can be mitigated. They could also do work with the right partners to do the, the, the testing. But I think someone, someone that has that security expertise would be a good partner for, for your concerns. [01:15:20] Speaker E: So my husband is in security. Interesting part is he's in security, but he says that there will be a lot of compliance and privacy because there were other parts involved related to vendors and stuff. And so I always wonder whether I should, should find a lawyer, but lawyer won't understand the architecture part and whether I should. And the security guy would understand that, but he would still have the missing gap of the knowledge on the lawyer part. So I was, I always like struggle to fight between those thought process that comes together. [01:15:58] Speaker C: I'm gonna pitch David. [01:16:00] Speaker B: Go ahead, Paul. [01:16:01] Speaker C: I was gonna pitch David to kind of bring the right legal minds together, but go ahead. [01:16:04] Speaker B: Those. There's four major fields of tech law, when you get down to at least the way I classify it. And Eric and I are in different ones. Well, really what you're talking about is a third field, which is cyber law. Right, Cyber law, attorneys. This is, this is what they do. They will understand enough. They will teach themselves what they don't know. And they will work with the CISO and information security officers to make sure that all the risks, they evaluate risks and give recommendations from the legal point of view, where the information security officer understands the technology, it can look at it that way, and they work together to make sure the risks are managed for a company. Erica will come in when there's problems and these things are not followed properly and then people get sued as part of that. And I primarily make sure that if you're going to make it, you have exclusive rights to make it and nobody else can copy it from you. [01:17:00] Speaker A: Excellent. Thank you, Jimmy, for your question. That was actually good. It covered a lot of kind of best practices and kind of mistakes made that people do. Final thoughts? Paul, final thoughts on AI in general? [01:17:16] Speaker B: I don't have any. I thought you raised great questions. These are all the kind of things that people need to be thinking about in this field. And a lot of it is the reality that when it comes to risks, AI can do lots of things really quickly and it just increases risk when you, when you do that. And that, that's the, that's the extra part that they don't tell you about in the commercials. And it kind of is the reality. So you still have to kind of have that understanding and put in the work to make sure this system that, that you're relying on doesn't introduce, whether they be legal concerns or data security problems or, you know, just, you know, violates other rights, it can, it can do a lot of harm. And you need to, you need to read the code, check the code and test it. Like Erica said, a lot more than you think you might need to. [01:18:14] Speaker A: Yeah. Amen. Erica, final thoughts? Yeah. [01:18:18] Speaker C: Yeah, well, I was thinking about this because I think this orient really needs to start thinking about, like, what is that first step for governance? [01:18:26] Speaker B: Right. [01:18:27] Speaker C: I think just start off with knowing what AI tools your team is using. Just that, just what are the tools? Create the list and then for the top three, take a look at the data use terms so you, you, you can make one decision about whether you're comfortable with what those tools are doing with your, with your information. That's, that's the start of the governance program. Right. If you just do that thing, everything else builds from there. Okay, so that's what I wanted to just share and that the AI is not going anywhere. It's not slowing down either. So I think we all need to get comfortable with it and as well as running businesses, we have to adopt them. But this is the first step you can take to help get your arms around it from a governance perspective. [01:19:27] Speaker A: Amen to that also. Thank you, Paul. Thank you, Erica. Thank you for this conversation. I'm sure we can go probably another hour breaking down some of the bigger topics that we spoke about. But we thank you for your expertise, expertise and your professionalism and your insights and your opinions and your advice. Thank you, everybody, for attending. We will be back, of course, next month. And so you will you will get information about what our next topic is when we send out the recap, the video and the link. So, Paul, Erica, I think you might have shared your information or some information in the chat. So please, if you haven't, please do. And everybody enjoy the last few days of summer. We'll talk to you soon. See you soon. Thank you.