Amidst the rapid advancements in AI, cybersecurity experts are working hard to keep systems safe. Microsoft’s AI Red Team is at the forefront of this effort, taking a unique approach to identify and mitigate potential risks posed by generative AI tools like ChatGPT and Copilot. The AI Red Team, led by Ram Shankar Siva Kumar, combines traditional cybersecurity methods with a focus on responsible AI practices.
The team includes experts from diverse fields such as neuroscience, linguistics, and national security, bringing a wide range of perspectives to address the complex challenges of AI safety. “We think security, responsible AI, and AI safety are different facets of the same coin,” says Siva Kumar. “It’s important to get a universal, one-stop-shop look at all the risks of an AI system before it reaches the hands of a customer.
Because this area will have massive sociotechnical implications.”
The team operates independently from the developers who build the technology, allowing them to proactively identify weaknesses and potential risks.
Microsoft’s unique AI risk strategy
They adopt various personas, from creative pranksters to known adversaries, to uncover blind spots and region-specific threats.
The team’s global presence, with members speaking 17 languages, helps them address nuanced cultural contexts. To assist security professionals and machine learning engineers outside the company, the AI Red Team has released open-source frameworks like the Python Risk Identification Toolkit for generative AI. These tools help map potential risks, making red teaming more efficient and productive.
When an issue is identified, it is sent to the Responsible AI Measurement Team, which assesses the threat’s severity. Other internal experts and groups then work to address the issue in a three-step approach: mapping, measuring, and managing risks. “Our activity encompasses a wide variety of harms we try to proof for,” Siva Kumar says.
“We quickly adapt and reformulate, and that has been the recipe for our success — not to wait for the forces of change to push us, but to anticipate.”
As AI continues to transform various industries, the work of teams like Microsoft’s AI Red Team will be crucial in ensuring the safety and responsible use of these powerful technologies.
