Utah’s pilot program lets an AI system renew prescriptions for chronic conditions like diabetes and depression — a first-in-the-nation test of autonomous AI in patient care.
getty
In January, Utah became the first state to let an autonomous AI system renew prescriptions. This has sparked a debate that’s pulled in the American Medical Association (AMA), the state medical board, the FDA, and a growing shelf of academic journals, none of whom agree on what just happened.
Under a deal between the state’s Office of Artificial Intelligence Policy and Doctronic, an AI company, Utah agreed not allow the organization to refill some prescriptions without a doctor being in the loop as is currently required in the rest of the United States.
The regulatory mechanism used was an agreement from the state not enforce its unprofessional-conduct laws against Doctronic as long as the company adhered to a contract built with safety and privacy guardrails.
In the contract, Doctronic’s AI agent is allowed to process 30-, 60- and 90-day renewals at $4 per prescription for roughly 190 drugs used to treat chronic conditions like high blood pressure, diabetes and depression. These are medications a licensed provider already prescribed, not new treatment decisions.
Here’s how it works: a patient confirms they’re physically in Utah, answers a set of clinical questions, and, if the system clears them, the renewal goes straight to a pharmacy. If not, the chatbot hands the case to a doctor on Doctronic’s own telehealth service.
The process affects a narrow slice of what doctors and their offices do. Yet, it is a momentous step in AI, as “one of the first deployments at scale of an autonomous, agentic system in medicine” according to Stanford’s Michelle Mello. By July, a story posted in the Associated Press (AP) asked doctors, lawyers and public health experts whether the country had, without quite meaning to, handed a machine something close to a medical license.
Why AI Is Allowed To Refill Prescriptions In Utah
Here’s the problem the program aimed to fix: only licensed prescribers can authorize prescription renewals. When patients email or call a doctor’s office asking for one, they’re generating unreimbursed administrative work because there is no visit tied to the request.
Meanwhile, many chronic-disease prescriptions often rarely change from refill to refill. The idea is that a well-supervised AI system can absorb that repetitive work and free up physician time. Doctronic co-founder Dr. Adam Oskowitz has framed the company’s ambitions even more broadly, telling the AP he envisions a future where routine tasks like ordering tests and reviewing results get offloaded to AI so doctors can manage “thousands more patients than they can today.”
Early Results From Utah’s AI Prescribing Experience
Utah’s Department of Commerce released five months of pilot data in May. The results are nuanced. In 72% of cases, Doctronic’s AI recommended approving the renewal and routed it to a human physician for sign-off. Physicians agreed with that recommendation 91% of the time. The other 9% they asked for more information, like updated labs.
In the remaining 28% of cases, the AI escalated to a physician on its own, flagging complications or the need for new testing. In those cases, physicians agreed the escalation was warranted 69% of the time. In the other 31%, they judged the system had been overly cautious.
Zach Boyd, who heads Utah’s AI office, has said Doctronic has, if anything, erred toward over-caution, routinely kicking uncontroversial decisions up to a physician. This is caution the state considers appropriate for a system this new, even as it works against the efficiency case for the pilot.
The state’s own design guarantees that overlap between the AI and physicians, for now. Under the agreement, a Utah-licensed clinician must personally review the first 250 cases before anything reaches a pharmacy. The system is expected to move toward operating without that direct human review once it clears an accuracy threshold.
The Program Has Stirred A Hornet’s Nest
The design of the program is to automate human work with AI. That has created substantial pushback, the sharpest from Utah’s Medical Licensing Board who says it learned about the program from news coverage, not from state officials. “We were essentially told: ‘Yes this is going on. And no, you don’t have a say in it,’” board chair Dr. Alan Smith told the AP.
In an April letter, 11 board members asked that the program be suspended, pointing to the risk of renewing medications — including blood thinners on Doctronic’s list — without catching changes in a patient’s condition.
“Many times when I see people after six months I find that their medical history or situation has changed,” Smith said. “Just because something was prescribed before does not mean it’s appropriate now.” The AMA has echoed the concern, warning that “prescription renewals aren’t routine checkboxes.”
The AI policy office responded that the program remains in a fully human-reviewed phase and that the system is barred from touching controlled substances, altering treatment plans or initiating new prescriptions. In May, Public Citizen escalated the fight further, sending its own letter backing the medical board’s call for suspension and warning that “physician supervised” pilots risk sliding into rubber-stamp approval of machine output as volume grows.
There are also two structural risks, according to Mello. First is accountability. The contract between Utah and Doctronic is murky about who is on the hook if an AI-driven prescribing error injures a patient. The second is scope creep where once a vendor’s tool is deployed through a regulatory workaround, it could expand what the tool does without facing the scrutiny that greeted the original launch. According to Mello: “If Utah’s pilot program is the proverbial camel’s nose in the tent, it will be important for someone to be firmly holding the reins when the camel emerges.”
And indeed, that is what has started to occur. In March, Utah signed a second agreement, this time with with Legion Health. It covers a defined formulary of non-controlled maintenance psychiatric drugs like SSRIs and SNRIs, with mandatory escalation for any hint of suicidality, mania or severe side effects.
It has tighter guardrails, with a 98% physician-concordance bar for the first 250 cases and a 99% bar for the next 1,000.
AI Replacing Human Tasks Creates A Regulatory Gap
An issue this program raises is that physicians have licenses to practice medicine, AI doesn’t. According to Eric Bressman at the University of Pennsylvania, “We have crossed a threshold in terms of giving something that is not human a medical license, whether or not we want to call it that.” He compares today’s regulatory patchwork to American medicine before national licensing standards existed, and faults Utah for not requiring outcomes data on renewals before launch rather than after. “Mostly they’re accepting the company’s word on good faith that they’re up to the task,” he said — noting that the only published evidence on Doctronic’s clinical accuracy so far is a company-authored, non-peer-reviewed study finding its diagnoses matched human doctors in 80% of 500 telehealth consultations.
There’s an additional jurisdictional issue. Medical devices are federally regulated by the FDA, while medical practice is regulated state by state. Doctronic sits uncomfortably between the two. Doctronic executives told the AP they consider their tool part of the state-regulated practice of medicine and declined to say whether they’d sought FDA clearance. An FDA spokesperson said only that the agency “has not authorized any AI chatbots” and is taking a hands-off posture for now.
Where The Concept Of AI Replacing Humans In Healthcare Goes Next
Utah won’t be alone for long. Texas and Wyoming are reportedly weaving similar AI carve-outs into their own rules. Additionally, lawmakers in Iowa, Idaho and other states have introduced bills to formally license AI medical services. Many are built on a template from the Cicero Institute, an AI-friendly think tank founded by Palantir co-founder Joe Lonsdale.
Cicero’s health policy director, Adam Meier, has framed the resistance in blunter economic terms: “Whoever goes first is going to take the slings and arrows because there’s economic interests, concerns about the workforce and what that’s going to mean for jobs.”
Utah has said it will publish its findings. Doctronic has said peer-reviewed studies are coming later this year. Whether that evidence arrives before or after the program moves past its human-review phase is likely to determine how this plays out. As Daniel Aaron put it to the AP: “Companies may benefit in the short term by expanding their business models and kind of having the technology go beyond the evidence. But in the long-term, I think they risk compromising public trust and fueling backlash.”