There are several excellent resources that I used while preparing for my recent Data Science DoJo webinar on Building Trustworthy AI Solutions (see Building trustworthy AI webinar – Lake Data Insights for background). This post highlights the key resources and provides some context.
I framed the AI trust discussion around a favorite business book, Speed of Trust: The One Thing That Changes Everything: Stephen M .R. Covey, Stephen R. Covey, Rebecca R. Merrill: 8601405449078: Amazon.com: Books. In particular, the four cores of credibility – Integrity, Intent, Capabilities, and Results – cross over very well from human to human trust to human to AI trust.
The AI Guidelines from Microsoft Research (https://aka.ms/aiguidelines) are an excellent starting point for all the considerations that should be considered when delivering an AI solution. The poster with 18 guidelines across the phases of Initially, During Interaction, When Wrong, and Over Time is a useful reference for evaluating your solution.
The open source toolkits for fairness (https://fairlearn.org/) and model interpretability (https://interpret.ml/) are great building blocks for identifying bias and opening up a model’s ‘black box’ respectively.
Last but not least, there’s some great research going on in this area. There’s significant overlap with user experience and the paper on an AI trust score written by my colleagues Jennifer Wang and Angela Moulden is a good example. You can find that paper at AI Trust Score: A User-Centered Approach to Building, Designing, and Measuring the Success of Intelligent Workplace Features (acm.org).
I also learned how a confidence score for a prediction can be more valuable than interpretability in “Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making” by Zhang, Liao, and Bellamy. This paper can be found at https://dl.acm.org/doi/abs/10.1145/3351095.3372852.
The webinar (https://online.datasciencedojo.com/course/Building-Trustworthy-AI-Solutions) knits these resources together into a cohesive story for building trust and provides several examples. Please check it out!
Picture details: Lake Superior, 8/6/2019, Canon PowerShot SD4000 IS, f5.6, 1/800s, ISO-1600, – 1 step