ALT Highlights – An Interview with the PC Chairs of ALT 2021
Welcome to ALT Highlights, a series of blog posts spotlighting various happenings at the recent conference ALT 2021, including plenary talks, tutorials, trends in learning theory, and more! To reach a broad audience, the series will be disseminated as guest posts on different blogs in machine learning and theoretical computer science. This initiative is organized by the Learning Theory Alliance, and overseen by Gautam Kamath. All posts in ALT Highlights are indexed on the official Learning Theory Alliance blog.
This is the fourth post in the series, an interview with ALT 2021 PC Chairs Vitaly Feldman and Katrina Ligett, written by Sutanu Gayen and Michal Moshkovitz.
We had the great opportunity to attend The 32nd International Conference on Algorithmic Learning Theory, held online between March 16-19, 2021, and co-chaired by Vitaly Feldman and Katrina Ligett. Vitaly is a research scientist at Apple AI Research and has done foundational works in machine learning and privacy-preserving data analysis. Katrina is an Associate Professor of Computer Science at the Hebrew University of Jerusalem and has done pivotal works in data privacy, algorithmic fairness, algorithmic game theory, and online algorithms. We asked for an interview with them about their experiences and perspectives as co-chairs, to which they kindly agreed. We are happy to share with the readers the excerpts of this interview.
How it started
How are chairs and program committees chosen?
Katrina: The chairs are selected by the Association for Algorithmic Learning Theory (AALT) Steering Committee (http://algorithmiclearningtheory.org/alt-steering-committee/), and the chairs then select the program committee members. Vitaly and I brainstormed potential PC member names, solicited additional suggestions, and also considered the lists of people who have served on recent ALT and COLT PCs. In building the PC, we had many considerations in mind, including coverage of research areas, and various metrics of diversity.
The chairs’ role
What are the different tasks a chair has? What is the most difficult task?
Katrina: At a high level, the roles of the PC chairs are to build the PC, oversee the reviewing process and create the conference program. Practically, though, there are a lot of decisions that need to be discussed, emails to be sent, and a lot of organizational aspects to tend to—configuring the reviewing platform, sending reminders, chasing down late reviews, and so on. Vitaly and I have a very, very long joint “to do” list—and luckily, it’s now almost all crossed off! We also had additional responsibilities this year because of the move to the virtual conference format, including selecting the technologies, overseeing the pre-recording process, and much more.
Vitaly: I think the hardest and probably the most time-consuming is making the accept/reject decisions on papers. For a large fraction of the papers arriving at the decision requires getting a sense of the results; understanding the main points in reviews, author responses and discussion (while calibrating them to the PC members and reviewers); ensuring that each paper is properly discussed by chasing reviewers, asking questions and often soliciting additional opinions. We also needed to come up with a set of criteria for deciding on borderline cases and make sure that these criteria are applied as consistently as possible. At the end it is a rather long and iterative process that luckily for us has converged to a program we are happy with.
How much time do you spend doing chair tasks? How do you balance chairing a conference (a massive amount of work) with all your other commitments? Do you turn down other service items you would generally accept, etc.?
Katrina: It’s difficult to estimate the number of hours, but I think we have been meeting regularly since early June 2020, and we’re only wrapping up our work now, in late March 2021. It’s a much longer-timeframe commitment than serving as a PC member. I actually am chairing a second conference this year, FORC, and together it makes for a pretty serious load. As a result, I have been declining all other conference-related service. I also have a couple of other pretty substantial service commitments, as well, so I just don’t have bandwidth this year for additional PC and Area Chair-type roles.
Vitaly: I agree that it’s hard to tell how much time we spent in total. My rough estimate is that it’s about a month of full-time work. I also had to decline most other service commitments during that period some of which I’d normally accept. Naturally, it also slows down other work so I definitely had to lean more on my collaborators in some of the ongoing projects 🙂
Can chairs bring their own personality into the conference? How?
Katrina: One area where the chairs enjoy freedom is in selecting the keynote and tutorial speakers. I’m biased, of course, but I think we chose very well, and all of these speakers (Joelle Pineau, Shay Moran, and Costis Daskalakis) gave excellent talks (they were recorded—check them out if you missed them)! We also were fortunate to be able to work with amazing partners who organized the mentoring workshop (Surbhi Goel, Nika Hagtalab, and Ellen Vitercik) and the Women in ML Theory event (Tosca Lechner and Ruth Urner). These aspects of the conference beyond the papers are a way for the chairs to express their priorities.
Vitaly: The chairs have a lot of freedom in choosing how to run the review process, design the conference program and who else will be involved. So, inevitably, the chairs’ personalities and tastes end up being reflected in the final results.
Does the online conference impact the chair job? How?
Katrina: Typically, the PC chairs build the program, and then many details of organizing and running the conference get handed off to local organization chairs. But this year, since ALT was held virtually, there were many unusual tasks that fell to the PC chairs—not just the obvious ones like choosing the technologies and format and negotiating those contracts, but smaller things like chasing down authors who failed to upload their recordings, and developing instructions for people in various roles to interact with the conference platform.
In addition, COVID times placed strains on many people, which made it more challenging to recruit PC members, and resulted in a higher than usual rate of late reviews and PC drop-outs, which of course left us scrambling.
What motivates you to spend time on a conference service?
Katrina: We all rely on the conference system for our personal professional advancement, and for the advancement of our field as a whole. So we all owe that system our service, of course, according to our abilities, availability, and seniority. Also, it’s fun to get this different perspective on the conference review process and on the field. And it’s an honor to be entrusted with shepherding a conference for a year and hopefully nurturing its growth.
Vitaly: I agree that it’s a mix of (1) contribution to the community I’m a member of and, perhaps, an opportunity to improve some of its processes (2) a learning experience that gives one a higher-level view of the research that is happening and people who do it (3) honor and recognition that come with the job.
Awards
How do you decide which papers were chosen as awards?
Vitaly: A necessary condition for a paper to receive an award is that at least one of the PC members/reviewers assigned to the paper is excited about the results. So we start by looking at papers with the highest scores (typically all papers that received at least one “strong accept”) and reading their reviews. This allowed us to narrow down the list to a set of 5-6 candidates. From those we selected the winners by learning more about the results and selecting those, we found the most significant and interesting for the community.
The review process
What are your thoughts about the current peer review process in ALT? What are the downsides and advantages? Do you have suggestions for improvement?
Vitaly: It is common that confidence in correctness of the results in a conference submission is based on higher-level sanity checks and general intuition of the reviewers. Naturally, the more interesting and important result the more likely it is to be scrutinized. At this ALT we did not run into a situation where the authors’ reputation affected our confidence in the correctness of the results. In case of concerns about correctness of an interesting result we would ask either an expert on the PC or an external expert to try to verify the result.
ALT currently relies on a traditional theory conference model of reviewing and for a typical submission has several PC members who are experts in the subarea. The reviewing load is also relatively light (8 papers per PC member). So I think that the overall reviewing quality is pretty much as good as it gets in ML (and is similar to COLT). Naturally, the model is not perfect and there is still variation in the quality of individual reviews. This year many more reviewers and PC members were under unusual time pressure due to the pandemic so perhaps the variation was higher than usual.
The future
What are your suggestions for the next chair?
Katrina: Make sure you have a good co-chair. 🙂 Vitaly has been a great partner for this process—fun to work with, reliable, always willing to pitch in even on the less-fun tasks, and I have great respect for his technical perspective.
Vitaly: I agree that diversity of perspectives and expertise is useful in several ways. Most notably, it gives the chairs a wider network of people to select the PC from. But I completely agree with Katrina, that the most important thing is the ability of co-chairs to work well together: after all, it’s a lot of work and complicated decisions that need to be made jointly. Here, I couldn’t have asked for more: Katrina is amazing both professionally and personally. Working with her was definitely the highlight of being the ALT co-chair and learned a lot from her in the process as well.
About The Author
