This episode is not yet released.
the-defeat-of-the-winograd-schema-challenge | episodes



The Defeat of the Winograd Schema Challenge

Our guest today is Vid Kocijan, a Machine Learning Engineer at Kumo AI. Vid has a Ph.D. in Computer Science at the University of Oxford. His research focused on common sense reasoning, pre-training in LLMs, pretraining in knowledge-based completion, and how these pre-trainings impact societal bias. He joins us to discuss how he built a BERT model that solved the Winograd Schema Challenge.

Vid introduced the Winograd schema challenge (WSC) and why it was challenging for deep learning models. He shared how he gathered the dataset he used for training the model. Vid also shared the metric for evaluating models for WSC. He shared the performance of other NLP algorithms on WSC.

Vid shared his thoughts on the imitation game and whether AI is advancing to pass the Turing test. He discussed other problems that still pose a challenge for AI systems.

Resources

Research Paper: A Surprisingly Robust Trick for Winograd Schema Challenge.

Research Paper: A Review of Winograd Schema Challenge Datasets and Approaches.

Vid Kocijan

Vid Kocijan is a Machine Learning Engineer at Kumo.AI. Prior to this, he worked as a Quantum Natural Language Processing Engineer at Cambridge Quantum. Vid holds a PhD in Computer Science from the University of Oxford on the topic of Natural Language Processing, commonsense reasoning and bias in AI. His other research includes work on network analysis at Stanford University and transfer learning in natural language processing at New York University.