(My!) class on concepts in NLP
teaching nlp by Vagrant GautamSyllabus for a class I will teach next summer on the conceptualization and operationalization of abstract concepts in natural language processing like interpretability, bias and robustness.
I'm thrilled to be teaching next summer at Saarland University, and not just as a co-instructor on an existing course! I designed a seminar to get Masters students comfortable with conceptual work, something that I'm really enjoying doing these days (see our work on intersectionality, names, and democratization). The course is research-focused and reading-heavy (for computer scientists) and I'm looking forward to the discussions we'll have, because I think conceptual work is critical to think about where we are and where we're going in NLP. I've chosen a selection of concepts in subfields that I'm familiar with—fairness, interpretability and reasoning—but I welcome additional suggestions for conceptual/critical papers. If you're an author of one of the papers I've selected and would be interested in joining our discussion, I'd be thrilled to hear from you.
Syllabus
Introduction
NLP papers commonly use various abstract concepts like "interpretability," "bias," "reasoning," "stereotypes," and so on. Each subfield has a shared understanding of what these terms mean and how we should treat them, and this shared understanding is the basis on which datasets are built to evaluate these abilities, metrics are proposed to quantify them, and claims are made about systems. But what exactly do these terms mean? And, indeed, what should they mean, and how do we measure that? These questions are the focus of this seminar on defining and measuring abstract concepts in NLP.
In two week cycles, we will cover various concepts in NLP research from the selection below. We will read papers that analyze or critique how a given concept is used, and then we will use this as a lens to read, discuss, and critique three recent NLP papers that use that concept. We will also try to reimagine how we would run these projects and write these papers in light of what we have learned.
Course Requirements
- Present a concept and lead a discussion about it (a sample will be given of what this should look like) – 35%
- Engage with presentations about other concepts – 30%
- Write a report designing an NLP project involving the concept that is either novel or a reimagination of one of the papers we saw in the seminar – 35%
Learning Objectives
This course will help you acquire / practise the following skills, among others:
- How to read and critique papers (both interdisciplinary and more conventional NLP papers)
- How to critically evaluate aspects of conceptualization (defining an abstract concept) and operationalization (creating empirical measures of the abstract concept), towards answering questions like: What does X mean? How do we translate our conceptions of X into something that we can observe and measure? How, concretely, can we measure it—with data and metrics and gold labels and an evaluation strategy? What should X mean? How do our choices in defining, conceptualizing and operationalizing X lead to gaps in how we make claims about X?
- How to design NLP projects in ways that address critiques and push the discipline forward
General Reading
- It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance[1]
- If all you have is a hammer, everything looks like a nail: Operationalization matters[2]
- The LLM Evaluation Guidebook[3]
- Reading papers: How to Read a Paper[4], How to Read a Technical Paper[5], Ten simple rules for reading a scientific paper[6], Reading a Technical Paper[7]
List of Concepts
We will choose a subset of the following topics based on class size and student interest: robustness, stereotypes, interpretability, explainability, paraphrases, bias, race, emergent abilities, gender, generalization, names, reasoning. I may add further topics, including understanding, intelligence, accountability, memorization, trust, etc., depending on interest. Each concept is listed below with a paper that does a critical or conceptual analysis (sometimes with a survey) of it. In addition, other papers that use the concept are listed here for us to discuss and evaluate.
Robustness: Beyond generalization: a theory of robustness in machine learning[8]
- How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?[9]
- Sorting through the noise: Testing robustness of information processing in pre-trained language models[10]
- ROBBIE: Robust Bias Evaluation of Large Generative Language Models[11]
Stereotypes: Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets[12]
- SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models[13]
- StereoSet: Measuring stereotypical bias in pretrained language models[14]
- Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models[15]
Interpretability: The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery[16] and Against Interpretability: a Critical Examination of the Interpretability Problem in Machine Learning[17]
- RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations[18]
- Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models[19]
Explainability: Explanation in artificial intelligence: Insights from the social sciences[20]
- Attention is not Explanation[21]
- Attention is not not Explanation[22]
- Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals[23]
Paraphrases: What Is a Paraphrase?[24]
- How Large Language Models are Transforming Machine-Paraphrase Plagiarism[25]
- Paraphrase Types for Generation and Detection[26]
- When and how to paraphrase for named entity recognition?[27]
Bias: Language (Technology) is Power: A Critical Survey of "Bias" in NLP[28]
- Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions[29]
- Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection[30]
- Social Bias Frames: Reasoning about Social and Power Implications of Language[31]
- Intrinsic Bias Metrics Do Not Correlate with Application Bias[32]
Race: A Survey of Race, Racism, and Anti-Racism in NLP[33]
- A Multilingual Dataset of Racial Stereotypes in Social Media Conversational Threads[34]
- The Risk of Racial Bias in Hate Speech Detection[35]
- Evaluation of African American Language Bias in Natural Language Generation[36]
Emergent abilities: Are Emergent Abilities of Large Language Models a Mirage?[37]
- Emergent Abilities of Large Language Models[38]
- Are Emergent Abilities in Large Language Models just In-Context Learning?[39]
- Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study[40]
Gender: Theories of "Gender" in NLP Bias Research[41] and Gender as a Variable in Natural-Language Processing: Ethical Considerations[42]
- The Lou Dataset — Exploring the Impact of Gender-Fair Language in German Text Classification[43]
- On Evaluating and Mitigating Gender Biases in Multilingual Settings[44]
Generalization: A taxonomy and review of generalization research in NLP[45]
- Lexical Generalization Improves with Larger Models and Longer Training[46]
- Crosslingual Generalization through Multitask Finetuning[47]
- Cross-Task Generalization via Natural Language Crowdsourcing Instructions[48]
Names: Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP[49]
- A Rose by Any Other Name would not Smell as Sweet: Social Bias in Names Mistranslation[50]
- Nichelle and Nancy: The Influence of Demographic Attributes and Tokenization Length on First Name Biases[51]
- The Gender Gap Tracker: Using Natural Language Processing to measure gender bias in media[52]
Reasoning: Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema[53]
- Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?[54]
- Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks[55]
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models[56]
Arjun Subramonian, Xingdi Yuan, Hal Daumé III, Su Lin Blodgett ↩︎
Christina R. Steidl, Regina Werum ↩︎
Clémentine Fourrier ↩︎
S. Keshav ↩︎
Jason Eisner ↩︎
Maureen A. Carey, Kevin L. Steiner, William A. Petri J ↩︎
Josef Fruehwald ↩︎
Timo Freiesleben, Thomas Grote ↩︎
Shiyue Zhang, Vishrav Chaudhary, Naman Goyal, James Cross, Guillaume Wenzek, Mohit Bansal, Francisco Guzman ↩︎
Lalchand Pandia, Allyson Ettinger ↩︎
David Esiobu, Xiaoqing Tan, Saghar Hosseini, Megan Ung, Yuchen Zhang, Jude Fernandes, Jane Dwivedi-Yu, Eleonora Presani, Adina Williams, Eric Smith ↩︎
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach ↩︎
Akshita Jha, Aida Mostafazadeh Davani, Chandan K Reddy, Shachi Dave, Vinodkumar Prabhakaran, Sunipa Dev ↩︎
Moin Nadeem, Anna Bethke, Siva Reddy ↩︎
Eddie Ungless, Bjorn Ross, Anne Lauscher ↩︎
Zachary C. Lipton ↩︎
Maya Krishnan ↩︎
Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva, Atticus Geiger ↩︎
Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, Mor Geva ↩︎
Tim Miller ↩︎
Sarthak Jain, Byron C. Wallace ↩︎
Sarah Wiegreffe, Yuval Pinter ↩︎
Yanai Elazar, Shauli Ravfogel, Alon Jacovi, Yoav Goldberg ↩︎
Rahul Bhagat, Eduard Hovy ↩︎
Jan Philip Wahle, Terry Ruas, Frederic Kirstein, Bela Gipp ↩︎
Jan Philip Wahle, Bela Gipp, Terry Ruas ↩︎
Saket Sharma, Aviral Joshi, Yiyun Zhao, Namrata Mukhija, Hanoz Bhathena, Prateek Singh, Sashank Santhanam ↩︎
Su Lin Blodgett, Solon Barocas, Hal Daumé III, Hanna Wallach ↩︎
Mihir Parmar, Swaroop Mishra, Mor Geva, Chitta Baral ↩︎
Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, Noah A. Smith ↩︎
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, Yejin Choi ↩︎
Seraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sánchez, Mugdha Pandya, Adam Lopez ↩︎
Anjalie Field, Su Lin Blodgett, Zeerak Waseem, Yulia Tsvetkov ↩︎
Tom Bourgeade, Alessandra Teresa Cignarella, Simona Frenda, Mario Laurent, Wolfgang Schmeisser-Nieto, Farah Benamara, Cristina Bosco, Véronique Moriceau, Viviana Patti, Mariona Taulé ↩︎
Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, Noah A. Smith ↩︎
Nicholas Deas, Jessica Grieser, Shana Kleiner, Desmond Patton, Elsbeth Turcan, Kathleen McKeown ↩︎
Rylan Schaeffer, Brando Miranda, Sanmi Koyejo ↩︎
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus ↩︎
Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, Iryna Gurevych ↩︎
Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, Ji-Rong Wen ↩︎
Hannah Devinney, Jenny Björklund, Henrik Björklund ↩︎
Brian Larson ↩︎
Andreas Waldis, Joel Birrer, Anne Lauscher, Iryna Gurevych ↩︎
Aniket Vashishtha, Kabir Ahuja, Sunayana Sitaram ↩︎
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske, Ryan Cotterell, Zhijing Jin ↩︎
Elron Bandel, Yoav Goldberg, Yanai Elazar ↩︎
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, Colin Raffel ↩︎
Swaroop Mishra, Daniel Khashabi, Chitta Baral, Hannaneh Hajishirzi ↩︎
Vagrant Gautam, Arjun Subramonian, Anne Lauscher, Os Keyes ↩︎
Sandra Sandoval, Jieyu Zhao, Marine Carpuat, Hal Daumé III ↩︎
Haozhe An, Rachel Rudinger ↩︎
Fatemeh Torabi Asr, Mohammad Mazraeh, Alexandre Lopes, Vagrant Gautam, Junette Gonzales, Prashanth Rao, Maite Taboada ↩︎
Yanai Elazar, Hongming Zhang, Yoav Goldberg, Dan Roth ↩︎
Vagrant Gautam, Eileen Bingert, Dawei Zhu, Anne Lauscher, Dietrich Klakow ↩︎
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, Yoon Kim ↩︎
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou ↩︎
- Previous post: Protect a disabled NLPer; don't come sick to EMNLP!
- Back to the archive