Abstract:
Utah, USA, is further expanding the application scope of medical artificial intelligence and allowing AI systems to directly assess patients, decide on treatment plans and issue prescriptions under specific conditions. As new pilot programs launch, Utah is becoming one of the most aggressive testing grounds for medical AI regulation in the United States.
This new project is launched by medical AI startup Nolla Health, mainly targeting patients with mild to moderate acne. Users first need to take a facial photo through the app and fill in medical history and personal information. AI then analyzes the skin condition, assesses the severity of acne, and then selects a suitable treatment plan from a list of approved drugs based on the medical information provided by the patient.
Different from traditional telemedicine, the core of this process is not to let AI complete the screening first and then make the final diagnosis by the doctor. Instead, it allows AI to make the final decision on whether to prescribe medication and send the prescription directly to the pharmacy designated by the patient.
However, the project did not completely eliminate physician involvement from the outset. In the initial phase of the pilot, every AI-generated prescription will need to be reviewed and signed by a licensed Utah physician. As the project progresses, doctor participation will gradually decrease, eventually allowing AI to issue qualified prescriptions without prior review by doctors.
Under the pilot agreement announced by Utah officials, Nolla Health can currently only treat mild to moderate acne for adult patients, and the prescription range is strictly limited to a small number of topical medications, including designated creams and gels. The system cannot be used to treat severe acne, nor can it prescribe riskier treatment options such as oral medications or isotretinoin.
Patients must also undergo identity and age verification, and prescriptions can only be sent to Utah addresses. For patients with severe acne, who are pregnant or planning to become pregnant, who are breastfeeding, who have compromised immune systems, and who have had adverse reactions to related drugs, the AI system must stop the automatic prescription process and transfer it to human medical personnel.
The reason why Utah can conduct such an experiment is related to the "AI regulatory sandbox" system established by the state in recent years. The system allows companies to test artificial intelligence products under government supervision while temporarily exempting some existing regulatory requirements within clearly defined limits.
This means that the company does not obtain an unlimited "AI medical license", but after signing an agreement with the state government and relevant regulatory agencies, it can conduct tests on specific products, specific diseases and specific groups for a limited time. Businesses will still need to comply with other legal requirements that are not exempted from the agreement.
In fact, earlier this year, Utah became the first state in the United States to allow AI to autonomously handle prescription refills. At that time, Doctronic launched an AI medical service that could re-prescribe some chronic disease drugs for patients who have received long-term treatment without the direct involvement of doctors.
This program covers about 190 commonly used non-controlled drugs, but does not include pain relievers, injectable drugs and some drugs used to treat ADHD. Patients need to first pass identity verification and medical information verification, and AI will then decide whether to allow refilling based on past prescriptions and the patient's current condition.
Utah’s current new policy means that AI medical trials are moving further from “automatic refill” to “first prescription”. There is a clear difference between the two: refilling means that the patient has previously been prescribed a treatment plan by a doctor, while prescribing for the first time means that the AI must make new clinical decisions based on the patient's current condition.
This is why the Nolla Health project has received special attention.
Utah officials require Nolla Health to have two certified doctors review every prescription early in the program. As the project progresses, the audit method can gradually change to post-audit and random inspection, but each time it enters a new stage, it must obtain written approval from the state government.
Businesses must also purchase professional liability insurance and report adverse events to regulators after they occur. At the same time, pharmacies must be clearly informed that prescriptions are generated by AI systems and be able to contact relevant doctors to handle issues.
Utah regulators also require companies not to sell or transfer patient data for other uses, and must regularly submit data on project operations, including indicators such as the degree of consistency between AI and physician judgment.
The Utah government believes that this model can explore whether AI can reduce medical costs in some common, low-risk medical scenarios and alleviate the problem of insufficient doctor resources. For some patients with relatively simple conditions, being able to complete assessments and prescribe medication via mobile phone could potentially reduce the time and expense of traveling to the clinic.
However, there has been obvious controversy over this model in the medical community.
Critics believe that even for seemingly simple diseases, doctors need to consider the patient's age, medical history, other medications, allergies, and some clinical signals that are not easily discovered through standardized questions and answers when prescribing drugs. If you rely entirely on AI for judgment, once the system misses important information, it may cause incorrect treatment.
Earlier this year, a safety test of Utah’s AI medication refill system also caused controversy. Security researchers have used relatively simple prompt word attacks to cause related AI systems to generate incorrect medical advice, including incorrectly increasing the dosage of certain drugs and giving dangerous drug recommendations. Such tests further highlight the differences between medical AI and ordinary chatbots.
One of the biggest risks of medical AI is that wrong answers from ordinary chatbots may simply lead to misleading information. However, once medical AI has the authority to prescribe medicine, errors may directly translate into real-world medical consequences.
Therefore, the current pilot in Utah is actually exploring a very important question: under what conditions, artificial intelligence can transform from an "auxiliary tool for doctors" to an "executor of medical decisions."
Supporters believe that AI does not necessarily need to replace doctors, but can take on a large number of standardized, repetitive and relatively low-risk tasks. For example, simple skin disease assessment, medication refill for long-term chronic diseases, and other situations that comply with clear clinical guidelines may become the first areas where AI will enter the medical system.
Opponents argue that even if some cases appear simple, it cannot be assumed that all patients fit the standard model. Especially when AI has the power to prescribe, the cost of system errors will be significantly higher than ordinary information question and answer errors.
Currently Utah does not allow AI to "become a doctor" without restrictions. Both Nolla Health's acne treatment and previous AI drug refill projects are strictly limited to specific drugs, specific patients and specific processes, and companies must accept continuous supervision.
But from an industry development perspective, Utah’s significance is still very special. In the past, medical AI was mainly used for image recognition, auxiliary diagnosis, medical record collection and doctor decision support, but now AI is gradually beginning to gain actual clinical decision-making authority.
If these pilots can prove that AI can safely complete some low-risk medical tasks under strict restrictions and continuous review, then other US states may refer to similar models in the future to further relax the scope of AI applications in the medical field.
Conversely, if serious medical errors occur during the pilot process, the system is attacked, or AI judgments deviate significantly from those of professional doctors, then the Utah experiment may also become an important case for regulatory agencies to tighten the authority of medical AI.
No matter what the final result is, the ongoing experiment in Utah means a clear change: the stage of artificial intelligence entering the medical industry is gradually moving from "helping doctors work" to "directly making medical decisions within a limited scope." Whether AI can assume this responsibility will no longer be just a technical issue in the laboratory, but will increasingly be tested through real patients and real prescriptions.
Comments