Visiting Speaker: Jiahao Chen on tools for AI governance

Evaluating algorithm auditing and risk assessments as tools for AI governance

On Friday, Cynthia Liem & Patrick Altmeyer are hosting visiting speaker Jiahao Chen in the Van Katwijkzaal (Building 36), who will talk about algorithm auditing and risk assessments for AI governance.

Abstract

As progress in AI creates new risks for ethical harms, calls for regulations on AI are growing worldwide. Europe is at the forefront of landmark new laws, like the Digital Services Act, that require algorithmic audits and risk assessments as part of production usage of AI systems. Despite this growing interest in audits as tools for accountability and transparency, there is little consensus in industry on what constitutes an acceptable audit, as well as what to do with the results of an audit. In this talk, I survey the nascent landscape of algorithmic audits and their coevolution with legislative and regulatory developments across the world. I also report on my industry experience from building industry tools for internal compliance at financial institutions in the USA, as well as building a startup focused on auditing and audit readiness for AI employment tools in New York City. My main findings are: i) gaps in expectations for costs and outcomes of audits, ii) tensions between stakeholders arising from lack of audit preparedness, data availability and transparency expectations, iii) a general need for improved data science practices in algorithmic auditors, and iv) needs to incorporate audit findings into business processes for change management. I conclude with some preliminary thoughts on how we can address the growing pains in the nascent industry of AI governance and risk management, so that the practice of AI in industry can mature as an engineering discipline.

Bio

Jiahao left an academic career at MIT for industry in 2017. At that time, the deep learning revolution was in full-swing, but many of the impressive advancements were not in universities, but in companies. He became a Senior Manager of Data Science at Capital One focusing on machine learning research for credit analytics and retail operations. Jiahao shipped his first models in production—based on logistic regression and naïve Bayes—and felt like he’d suffered a bait and switch. Instead of the latest and greatest in neural networks, why did he ship a model based on decades-old statistics that he used to teach at MIT?

It turns out that banking regulations require models have behaviors that are understandable to humans, and must be proven to be non-discriminatory before they can be used. Jiahao became interested in fairness and explainability in AI/ML, and how to use state-of-the-art research techniques to address compliance needs. Jiahao started the responsible machine learning group at Capital One as a result, and brought those interests to JPMorgan Chase as a Director of AI Research. By the time Jiahao left in 2021, he was part of the leadership that oversaw the department’s growth to almost 70 full-time PhDs, as well as developing new techniques for fair lending compliance review and model risk management for AI/ML systems. Jiahao then co-founded a startup where he developed new risk management techniques for employment decision systems, and closed his first revenue sustaining customer just six months into the startup.

When still in academia, Jiahao was a Research Scientist at MIT CSAIL where he co-founded and led the Julia Lab, focusing on applications of the Julia programming language to data science, scientific computing, and machine learning. Jiahao has organized JuliaCon, the Julia conference, for the years 2014-2016, as well as organized workshops at NeurIPS, SIAM CSE, and the American Chemical Society National Meetings. Jiahao has authored over 120 packages for numerical computation, data science and machine learning for the Julia programming language, in addition to numerous contributions to the base language itself.

https://responsibleai.tech/about/