Researchers at MIT and the MIT-IBM Watson AI Lab have introduced a novel onboarding system designed to improve collaboration between humans and artificial intelligence (AI). In scenarios where AI models assist professionals, such as radiologists interpreting X-rays, the system teaches users when to trust the AI's advice and when to disregard it. The fully automated onboarding process incorporates training exercises based on learned rules, resulting in a 5 percent enhancement in accuracy during human-AI collaboration in image prediction tasks.

During onboarding, the system guides users through collaboration exercises, providing feedback on their performance and the AI's performance. Unlike traditional onboarding methods, which often rely on human-produced training materials for specific use cases, this system automatically learns from data generated by humans and AI working together on a particular task. It adapts to different tasks, making it scalable for various human-AI collaboration scenarios, including medical diagnostics, social media content moderation, writing, and programming.

The researchers envision the onboarding process as a crucial element in training medical professionals, anticipating its application in decision-making with AI assistance. The system's automated learning from data enables continuous evolution, aligning with the dynamic capabilities of AI models over time.

The onboarding method involves collecting data on human-AI collaboration in a specific task, embedding data points into a latent space, and identifying regions where collaboration errors occur. An algorithm then formulates rules for collaboration, expressed in natural language, and refines them through contrasting examples. These rules serve as the basis for training exercises, allowing users to practice and improve their collaboration with AI.

The study evaluated the system's effectiveness on tasks like detecting traffic lights in blurry images and answering multiple-choice questions from various domains. Results demonstrated a notable 5 percent improvement in accuracy for the traffic light prediction task. The researchers emphasize the importance of such onboarding procedures in ensuring users understand when it is appropriate to rely on AI suggestions.

In the future, the researchers plan to conduct larger studies to assess the short- and long-term effects of onboarding. They also aim to leverage unlabeled data for the onboarding process and refine methods to reduce the number of regions without omitting crucial examples. The work is seen as a crucial step in addressing the need for effective methods that help users discern when it's safe to trust AI suggestions in various applications.