a new study published in The Lancet by artificial intelligence ethicist Dr. Stefan Harrer has advocated for a strong and comprehensive ethical framework around the use, design and management of generative AI applications in healthcare and medicine as it has the potential to go catastrophically wrong.

The peer-reviewed study describes how Large Language Models (LLMs) have the potential to fundamentally transform information management, education and communication workflows in healthcare and medicine, yet remain one of the most dangerous and misunderstood forms of AI.

Dr. Stephen Harer

Dr. Harrer is chief innovation officer at the Digital Health Cooperative Research Center (DHCRC), a major funding body for digital health research and development, and describes generative AI as a “very nice autocorrect” with not real understanding of language.

“LLMs used to be boring and safe. They have become exciting and dangerous,” he said.

“This study is a case for regulation of generative AI technology in healthcare and medicine and provides technical and managerial guidance to all stakeholders of the digital health ecosystem: developers, users and regulators. Because generative AI should be both exciting and safe.”

LLMs are an important part of generative AI applications for creating new content, including text, images, audio, code, and videos in response to textual instructions. Examples examined in the study include OpenAI’s chatbot ChatGPT, Google’s chatbot Med-PALM, Stability AI’s image generator Stable Diffusion, and Microsoft’s BioGPT bot.

Dr. Harrer’s research highlights a wide range of important applications for AI in healthcare, including:

• assist clinicians in generating medical reports or pre-authorization letters;

• helping medical students study more efficiently;

• simplifying medical jargon in doctor-patient communication;

• increasing the efficiency of clinical trial design;

• help overcome interoperability and standardization hurdles in EHR mining;

• make drug discovery and design processes more efficient.

However, his paper also highlights the inherent danger of LLM-driven generative AI because, as already demonstrated on ChatGPT, it can authoritatively and convincingly create and distribute false, inappropriate and dangerous content on an unprecedented scale.

Mitigating risks in AI

In addition to the risk factors identified by Dr. Harrer, he also outlined and analyzed real-life use cases of ethical and unethical LLM technology development.

“Good actors chose to follow an ethical path to build safe generative AI applications,” he said.

“Bad actors, however, get away with doing the opposite: by hastily producing and releasing LLM-powered generative AI tools in a burgeoning commercial market, they are gambling with the well-being of users and the integrity of AI and knowledge databases. dish . That dynamic has to change.”

He argues that the limitations of LLMs are systemic and rooted in their lack of language comprehension.

“The essence of efficient knowledge retrieval is asking the right questions, and the art of critical thinking relies on one’s ability to gauge responses by testing their validity against models of the world,” said Dr. Harrer.

“LLMs cannot perform any of these tasks. They are intermediaries who can narrow down the size of all possible responses to a prompt to the most probable, but they cannot judge whether prompt or response made sense or was contextually appropriate.

Guiding Principles

He argues that increasing the size of training data and building increasingly complex LLMs will not reduce risk, but rather increase it. That is why Dr. Harrer proposes a regulatory framework with 10 principles to reduce the risks of generative AI in health.

They are:

1. design AI as a tool to enhance the capabilities of human decision makers, not to replace them;

2. design AI to produce performance, usage and impact metrics that explain when and how AI is used to support decision making and scan for potential bias,

3. study the value systems of target user groups and design AI to meet them;

4. explain the purpose of designing and using AI at the beginning of any conceptual or development work,

5. disclosure of all training data sources and data attributes;

6. design AI systems to clearly and transparently label all AI-generated content as such;

7. Continuously monitor AI for data privacy, security and performance standards;

8. maintain databases for documenting and sharing the results of AI audits, educate users about model capabilities, limitations, and risks, and improve the performance and reliability of AI systems by retraining and deploying updated algorithms;

9. apply fair and safe work standards when hiring human developers;

10. establish legal priority to define under what circumstances data may be used to train AI, and establish copyright, liability and accountability frameworks to govern the legal dependencies of training data, AI-generated content and the impact of decisions humans make using such data.

“Without human oversight, guidance, and responsible design and use, LLM-powered generative AI applications will continue to be a party trick with significant potential for creating and spreading misinformation or malicious and inaccurate content on an unprecedented scale,” said Dr. Harrer.

He predicts a shit in the current competitive LLM arms race towards a phase of more nuanced and risk-aware experimentation with research-grade generative AI applications in health, medicine and biotechnology, which will result in the first commercial product offering in digital health data management within two years.

“I am inspired by thinking about the transformative role generative AI and LLMs could one day play in healthcare and medicine, but I am also acutely aware that we are far from there and that, despite the prevailing hype , LLM-powered generative AI can only gain the trust and support of clinicians and patients if the research and development community strives for equal levels of ethical and technical integrity in developing this transformative technology to market maturity,” said Dr. Harrer.

The complete study is available here.

READ NOW: 5 experts on how ChatGPT, DALL-E and other AI tools will change the way of working for creatives and the knowledge industry