Legastat Blog

Reflecting on 2023, AI Hallucinations & Document Review

Written by Paul Fox | Jan 4, 2024 1:39:48 PM

A very Happy New Year from the entire team at Legastat! We hope you enjoyed your time over the festive period and feel rejuvenated heading into the New Year. As we embark on 2024, it is interesting to reflect on the whirlwind that was 2023, a year that reshaped our understanding of reality and AI in the legal realm.

The term "hallucinate" was notably chosen as Cambridge Dictionary's word of the year. Beyond its usual connotations, it symbolises the phenomenon where AI, like ChatGPT, generates misleading information. This addition to the dictionary vividly encapsulates our journey with AI: powerful yet imperfect, demanding our critical thinking more than ever.

This concept of "AI hallucinations" is not just theoretical. We've seen real-world implications in our legal industry. For instance, there was a notable case where a litigant-in-person, relying on AI, unwittingly used nine fictitious authorities in a First-Tier Tribunal. This incident highlights the challenges and responsibilities we all face in integrating AI into our personal and professional lives. It serves as a reminder that while AI can be a valuable tool, it requires careful oversight and human expertise to ensure accuracy and ethical usage.

Document Review is clearly an area that stands to benefit significantly from the implementation of AI and Large Language Models (LLM). Whilst providers like Legastat have been augmenting our services with AI-driven machine learning and predictive coding for several years, the emergence of generative language models suggests significant changes may be coming. Sidley Austin has written a fascinating review of testing undertaken using ChatGPT to review documents, pitting the AI against human reviewers. It is certainly worth a read. In summary, unlike traditional TAR-based systems, ChatGPT didn't require training; it responded to the prompt it was provided, resulting in greater consistency than human reviewers. However, results depended on the quality of the prompt, in the same way that document reviewers rely on high quality instructions to read in on a matter.

As we journey through this 'brave new world', the potential and challenges of AI are becoming increasingly evident. From Sidley's experiments with GPT-4 in document review to the complexities presented by AI-generated legal scenarios, our industry stands at a pivotal intersection of cautious innovation. As we march optimistically forward in 2024, merging human insight with technological advancement is essential. By doing so, we'll collectively forge a legal industry that is efficient, effective, and ethically robust.

Thank you for all your support in 2023; we look forward to working with you again in 2024, and cheers to a year of growth, learning, and responsible innovation!