Session Description

Both traditional and generative AI-driven innovation is increasingly being used in clinical settings with the potential for transformative positive outcomes. Rapidly advancing AI evaluation methods can help ensure AI technologies are integrated in ways that are effective, safe, and equitable. In this talk, the Responsible AI Testing Team at DLA Piper will discuss red teaming generative AI applications in the healthcare context, such as consumer-facing AI chatbots or the use of LLMs to summarize patient-clinician interactions. We will describe recent evaluations where we applied our novel legal red teaming methodology to a healthcare chatbot. We will demonstrate how red teaming can be an effective technique for domain experts seeking to be better equipped to evaluate generative AI systems in healthcare.

Video

view slide deck

Speakers

Danny Tobey | Partner, Global Co-Chair; Chair, DLA Piper Americas AI and Data Analytics Practice

Danny is a medical doctor, successful software founder, and chair of DLA Piper’s AI & Data Analytics practice. He is a noted thought leader on the safe and effective deployment of AI systems, sitting on the World Economic Forum’s AI Governance Alliance and the UN’s Executive Committee of the AI for Good Law Track. In 2023, Financial Times named Danny the Innovative Practitioner of the Year for his pioneering work in legal AI, and he led Insider’s 2022 list of top attorneys helping companies adopt AI.

Danny’s AI clients include Fortune 500 and 100 companies, breakthrough startups like IDx, the first FDA-approved fully autonomous AI, and three of the four major LLM makers, including OpenAI in the first defamation suit against GenAI, and Anthropic as Trust & Safety counsel. Danny was an invited speaker at the AAAI’s inaugural AI, Ethics, and Society conference, Stanford Law School’s CODEX FutureLaw Conference, and the Swiss Embassy on AI innovation in the U.S. Danny advised the American Medical Association on its AI policy and has been recognized by the U.S. Library of Congress for his work on liability and AI.

Dr. Sam Tyner-Monroe | Managing Director of Responsible AI, DLA Piper

Sam Tyner-Monroe, Ph.D., is Managing Director of Responsible AI at DLA Piper, where she leverages artificial intelligence and data analytics to ensure accountability and transparency in AI systems. With nearly a decade of experience at the intersection of data, law, and policy, Dr. Tyner is skilled at turning data into insights, using algorithms responsibly, and communicating complex topics to both technical and non-technical audiences. She is focused on ensuring AI and data-driven technologies are developed and applied ethically for the benefit of society.

Dr. Tyner holds a Ph.D. in Statistics from Iowa State University. After earning her doctorate, she served as a postdoctoral researcher at the Center for Statistics and Applications in Forensic Evidence, applying statistical methods to improve the reliability and validity of forensic evidence. Dr. Tyner also served as an AAAS Science and Technology Policy Fellow at the U.S. Bureau of Labor Statistics, where she contributed her statistical expertise to evidence-based policymaking. With her diverse background, Dr. Tyner continues to drive innovation, champion accountability, and advance the responsible use of AI.

Bogdana Rakova | Senior Data Scientist for Responsible AI Testing, DLA Piper

Bogdana Rakova is a senior data scientist at the DLA Responsible AI Testing team with a background in computer science, ML engineering, and cross-disciplinary socio-technical research in Responsible AI. Previously, she has held positions as a Senior Trustworthy AI Fellow at Mozilla Foundation, a data scientist at the Responsible AI team at Accenture, a research fellow at Partnership on AI, and a senior ML research engineer at the Think Tank Team innovation lab at Samsung Research. She was a lead contributor to the IEEE 7010 Recommended Practice for Assessing the Impact of Autonomous and Intelligent Systems on Human Well-Being standard and her research centers on equity, access, participatory mechanism design, and multi-stakeholder engagement in data and AI governance practices.

other health AI hubs