A blind study conducted by researchers at Stanford Law School has revealed that law professors judged responses to law student questions generated by AI systems to be superior to answers written by fellow law professors in 75 percent of evaluated comparisons.
Forbes reports that the research, led by Stanford Law School professor Julian Nyarko and published last week, analyzed nearly 3,000 head-to-head comparisons between AI-generated answers and responses written by legal academics. The results showed AI responses were selected as superior 75 percent of the time when law professors evaluated anonymized answers to contract law questions without knowing their source.
The study involved faculty members from 16 different law schools who examined pairs of answers to determine which response would better serve students. None of the evaluators knew whether they were reading text produced by artificial intelligence or by a fellow law professor. The double-blind methodology was designed to eliminate bias and ensure objective assessment of response quality.
In a particularly notable finding, the professors identified AI-generated answers as pedagogically misleading or potentially harmful only 3.5 percent of the time. In contrast, answers written by human professors were flagged as problematic 12 percent of the time, making the human responses more than three times as likely to be considered potentially damaging to student comprehension.
Professor Nyarko, who directs Stanford’s Legal Innovation through Frontier Technology Lab, emphasized that the research team is “not advocating for wholesale adoption of AI tutors,” but added that “our data suggests that blanket skepticism may be equally unwarranted.”
The research team deliberately selected contract law as the testing ground because the subject matter does not lend itself to simple right-or-wrong answers. The 40 questions employed in the study were designed to mirror the types of inquiries students typically pose after class sessions or during office hours. These questions required test subjects to synthesize competing legal arguments and arrive at defensible conclusions rather than simply recall memorized information. This approach allowed researchers to evaluate whether artificial intelligence systems could demonstrate reasoning capabilities in areas where no single correct answer exists.
Breitbart News has reported on numerous cases of Lawyers filing false legal documents by depending on AI to get the details right. This risk is “nauseatingly frightening” according to the head of one of America’s most powerful law firms:
In response to the incident, Ayala was immediately removed from the case and replaced by his supervisor, T. Michael Morgan, Esq. Morgan expressed “great embarrassment” over the fake citations and agreed to pay all fees and expenses related to Walmart’s reply to the erroneous court filing. He emphasized that this incident should serve as a “cautionary tale” for both his firm and the legal community as a whole.
Morgan added, “The risk that a Court could rely upon and incorporate invented cases into our body of common law is a nauseatingly frightening thought.” He later admitted that AI can be “dangerous when used carelessly.”
The legal profession is just one of many fields wrestling with how to use AI for humanity’s benefit without letting the technology run amok. Breitbart News social media director Wynton Hall has written his instant bestseller Code Red: The Left, the Right, China, and the Race to Control AI to serve as the definitive guide on how the MAGA movement can create positions on AI that benefit humanity without handing control of our nation to the leftists of Silicon Valley or allowing the Chinese to take over the world.
Read more at Forbes here.
Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.