A computer that reads a news article and then answers questions about it can be thrown off by a simple rephrasing—a problem this fellowship aims to fix by teaching machines to build mental models of events as they read, much like a human does. Current language models like BERT excel at pattern-matching but fail at genuine understanding. They rely on shallow cues—an entity’s name, for instance—rather than grasping the factual knowledge and causal relationships between events. This makes them brittle: a small change to a question can cause performance to collapse. The research addresses this gap by drawing on cognitive psychology, where successful reading comprehension depends on constructing an internal “situation model” that integrates prior knowledge with new information. If successful, this work could make automated reading systems far more robust and reliable. That matters for any system that processes text at scale—search engines, automated customer service, medical record analysis, or legal document review—where a misunderstood question can lead to wrong answers or missed information. The project is primarily fundamental science, aiming to build a framework for human-like language understanding. Past work in this vein has underpinned breakthroughs in everything from voice assistants to machine translation, so a deeper grasp of event-centric reasoning could open doors not yet imagined.
View original technical description
Natural language understanding (NLU) aims to allow computers to understand text automatically. NLU may seem easy to humans, but it is extremely difficult for computers because of the variety, ambiguity, subtlety, and expressiveness of human languages. Recent efforts to NLU have been largely exemplified in tasks such as natural language inference, reading comprehension and question answering. A common practice is to pre-train a language model such as BERT on large corpora to learn word representations and fine-tune on task-specific data. Although BERT and its successors have achieved state-of-the-art performance in many NLP tasks, it has been found that pre-trained language models mostly only reason about the surface form of entity names and fail to capture rich factual knowledge. Moreover, NLU models built on such pre-trained language models are susceptible to adversarial attack that even a small perturbation of an input (e.g., paraphrase questions and/or answers in QA tasks) would result in dramatic decrease in models' performance, showing that such models largely rely on shallow cues. In human reading, successful reading comprehension depends on the construction of an event structure that represents what is happening in text, often referred to as the situation model in cognitive psychology. The situation model also involves the integration of prior knowledge with information presented in text for reasoning and inference. Fine-tuning pre-trained language models for reading comprehension does not help in building such effective cognitive models of text and comprehension suffers as a result. In this fellowship, I aim to develop a knowledge-aware and event-centric framework for natural language understanding, in which event representations are learned from text with the incorporation of prior background and common-sense knowledge; event graphs are built on-the-fly as reading progresses; and the comprehension model is self-evolved to understand new information. I will primarily focus on reading comprehension and my goal is to enable computers to solve a variety of cognitive tasks that mimic human-like cognitive capabilities, bringing us a step closer to human-like intelligence.
Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.
Is something wrong? Let us know