The following article is presented through the New Jersey State Bar Association's PracticeHQ, a free member resource that offers guidance to help attorneys build and maintain a successful law practice. The article is by Jeffrey R. Schoenberger, a senior consultant with Affinity Consulting Group, LLC.
No matter where you turn today, you read about the opportunities and perils of artificial intelligence tools like ChatGPT. The opportunities need no additional cheerleaders on their squad. And the Cassandras don’t need another armchair prophet of doom. Instead of highlighting extremes, let’s focus on the banalities that legal professionals need to know when approaching AI.
A Writing Aid, But Not (Yet) a Writer
One productive use of AI is to overcome writer’s block. Because ChatGPT works like a conversation, it’s possible to ask questions to help start your own ideas flowing. For example, if I were preparing a webinar on Microsoft Outlook, I could ask, “What are the most popular Outlook 365 features?” ChatGPT instantly returned a list of nine items, with a sentence describing each.
The list may spur inspiration or break a mental logjam, but even this simple request exposes gaps. First, its top four items are, in order, “email management,” “calendar integration,” “contacts and address book,” and “task management.” True enough, but it hardly makes for an illuminating webinar to say, “Outlook does email.” Someone new to the topic finds a starting point or learns the “greatest hits,” but a would-be presenter or trainer still confronts a fair amount of research and customization ahead. Anyone “in the know” on a topic finds ChatGPT’s responses as illuminating as the portrayal of attorneys on TV shows is accurate.
Second, AI tools like ChatGPT are known as large language models (LLM). Companies train these LLMs with an initial data set. ChatGPT and other AIs have cutoffs, after which they add no new data; September 2021 in ChatGPT’s case. So, were I to ask, “What new features of Outlook 365 should lawyers know about?”, ChatGPT would be ignorant of features added after September 2021, such as the forthcoming AI-based Microsoft Copilot.
Third, ChatGPT is verbose. I prompted it: “Write a 500-word blog about the best use of Outlook by lawyers.” It supplied an article of 600 words and concluded, “Embracing Outlook as a comprehensive tool can empower legal professionals to optimize their time, increase efficiency, and ultimately provide excellent legal services to their clients while maintaining the highest standards of professionalism and confidentiality.” It’s impressively grammatically correct, but would benefit from editing. My armchair rewrite uses half the words: “Legal professionals’ skillful use of Outlook saves time and assists them in efficiently providing excellent service to clients.” In fairness to ChatGPT, I presumed “excellent service” from an attorney includes professionalism and confidentiality.
Create an account and try to break your writer’s block here: https://chat.openai.com.
AI Tries Too Hard to Please
On June 22, 2023, Judge Kevin Castel of New York’s Southern District fined attorneys Steven Schwartz and Pete LoDuca $5,000 for filing a brief written by ChatGPT. Unfortunately for the attorneys involved, ChatGPT didn’t merely engage in the above-mentioned foibles. Instead, it invented cases and holdings to fit the attorneys’ propositions. Furthermore, ChatGPT’s manufactured cases included properly formatted, but entirely fake, citations, and it attributed the holdings to real judges. As part of his sanctions ruling, Judge Castel ordered the attorneys to send letters to “the judges whose names were wrongfully invoked”, notifying them of the sanctions.
While this sanctions case uniquely exposed AI’s “creativity,” ChatGPT’s inventiveness is not limited to court cases. We’ve seen similar ChatGPT flights of fancy in our internal testing. Without attempting to trick it, a colloquy asking ChatGPT to describe the differences between horses and unicorns resulted in ChatGPT declaring horses to be mythical creatures. We also witnessed ChatGPT assert a seven-year statutory closed file retention period for a state with no statutory retention period at all.
We sometimes feel rushed to get work out the door, meet a filing deadline, or check off a drafting task before vacation. And while ChatGPT and other generative-AI tools may help overcome writer’s block, legal professionals must review everything AI supplies, particularly for pleadings and attorney work product.
If you want to experiment with legal industry-targeted AI tools, try Casetext’s CoCounsel, Lexis + AI, and Westlaw Edge.
Two Worlds of AI
Stephen Rose, Chief Technology Strategist at Petri IT, was interviewed on the First Ring Daily podcast about Microsoft’s AI announcements at its 2023 Build Conference. He offered a sobering statistic from Microsoft’s Work Trend Index: 70% of employees would hand off as much work as possible to AI. Similarly, a recent study from Fishbowl found 43% of responding professionals had used AI for work-related tasks. 70% of those users had not told their bosses they were doing so.
Additional numbers Rose cites are even more alarming: 70% of respondents are experimenting with ChatGPT. 40% of those experimenting are putting confidential information into it. Only 5% are telling the boss they’re using AI. ChatGPT and most public-facing AI tools store the questions they’re asked. If you or someone in your firm is entering confidential information into an AI tool, stop until you know what the vendor does with the questions asked of the AI.
Rose’s suggested solution for confidentiality is to rely on a “sandboxed” AI rather than a “general public” tool. For businesses, and perhaps larger firms at present, that entails using AI tools available through Microsoft, Amazon, or others, running in a cloud environment that the business controls. For example, a firm on Microsoft 365 could connect to AI resources available on Microsoft Azure. Then the firm could train or “seed” its AI sandbox with the documents it stores in SharePoint while having user permissions controlled through Active Directory. The firm maintains information security while teaching an AI to “think and write” like the firm. Any summer associate could ask the firm’s AI a question and receive in reply an answer as knowledgeable as that from the senior partner.
Although “private AI clouds” aren’t approachable and affordable for small firms yet, it’s only a matter of time. We’re on the way to where a “best of breed” AI can combine reference material and case law with your past work product to create a quality draft in record time. But you should still proof it before filing.