CS575-ethics-spring-2022-team-project

Schedule

• Week 3 (3/14): Team matching, Topic exploration
• Week 6 (4/4, 4/6): Project proposal, Peer grading
  • Proposal slide submission deadline: 23:59, April 3, 2022
  • Peer grading: Real-time
• Week 11 (5/9, 5/11): Progress Update, Peer grading
  • Peer grading: Real-time
• Week 15 (6/8): Final presentation, Final report, Peer grading, Teamwork report

Goal

This project aims to replicate an existing paper on ethical issues in machine learning, and then make meaningful improvements/extensions in terms of research question, machine learning model, features, datasets, or evaluation metrics.

Teams

This is a team-based project. Please form teams of 3 students by March 14. If you need help forming teams, advertise yourself and find your teammate in Slack # team-matching.

Project Grading (50% of Course Grade)

• Proposal presentation: 5%
• Progress presentation: 5%
• Final presentation: 15%
• Final report: 10%
• Peer grading: 10%
• Teamwork report: 5%

Note that any team may get up to -25%p for project scores if there is a serious problem with teamwork. That means, the project grade could be 25/50, even if all presentation and report scores are perfect. This can apply to an individual student, or to the entire team.

Deliverables

All project team members should present an important part in the presentation (speak for at least one minute each). You should present your work in presentation session.

• Proposal presentation: 5-8 minute presentation
• Progress presentation: 7-10 minute presentation
• Final presentation: 10-15 minute presentation
• Final report: 4-6 pages report (+ unlimited references) and slides used for final presentation
• Peer grading: 4 peer gradings for proposal, progress, final presentation, and final report
• Teamwork report: (individually written/submitted) description of what each team member did

Instruction

Choose a paper with the following criteria.

• Recent paper published in 2019 or later
• Technical paper that includes a machine learning model and a medium- to large-sized dataset
• Covering any of the following topics: bias, fairness, social impact, interpretability, privacy, deepfake/generative modeling, social good
• From ML/NLP/AI conferences including ACL, EMNLP, NAACL, Coling, CVPR, ICCV, ECCV, NeurIPS, AAAI, IJCAI, ICML, ICLR
• If your paper deviates from the above criteria, or if your project is not based on replication of a previous paper, please email the teaching staff

Based on the paper, there are two options you can choose:

1. Replicate their method and apply it to your own research problem.
2. Replicate their method and improve it. You should find a problematic issue of the existing work and try to resolve/alleviate it.

Project Proposal

Give a 5-8 minute proposal presentation that includes the following:

• Problem: What is the topic of your paper? What is the specific task being tackled? Why is it important to solve this problem?
• Related work: What existing papers have dealt with this problem? What were their approaches?
• Replication approach: Which paper will you use for replication and why?
• Improvement approach: How will you improve upon the replication paper?
• Data: Describe the dataset you will use, even if it is the same one as in the replication paper. If the replication paper does not work with openly available data, explain how you will gather your own data. If using openly available data but you will work with a subset of the data, explain why and how you will select the subset.
• Plan: Explain a week-by-week plan of completing this project, clearly indicating which tasks will be done by which team member, and which tasks will be done together.

Progress Update

Give a 7-10 minute progress presentation that includes the following:

• Introduction: Please briefly explain your problem, approach, and dataset.
• Replication results:
  • How did you replicate the original paper? What problems did you face during the replication, and how did you solve them?
  • Describe the experiment setting, results of baseline implementation, replication of the chosen models and experiments. You may not have complete replication results, in which case just explain the results you have so far and a concrete plan of completing the rest.
• Plan:
  • Again, how will you improve or modify the original paper? Please have a more concrete plan for this based on the replication results.

Final Presentation & Final Report

Give a 10-15 minute final presentation and submit a 4-6 page (+ unlimited references) report in ICML format. Below is the structure of the final report. The final presentation should contain the same content but in a presentation style.

• Abstract: Summary of your work
• Introduction: Background and your motivation for the problem
• Approach: Your approach to solving the problem
• Data & Experiments: Data, model, algorithm, comparisons to baseline models, etc
• Results:
  • Replication results
  • Results of your improved approach: use graphs, tables, whatever is most appropriate to present your results
  • Interpretation of results: how do the results answer your research problem?
• Discussions: Your improvements and the limitations
• References (unlimited pages)

Peer Grading

• We will probably use Google form (TBD) and it will be done in the lecture.
• Everyone (individually) should grade and comment on
  • Proposal presentations
  • Progress presentations
  • Final presentations & reports

Teamwork Report

• We will use Google form
• Everyone should contribute, and everyone should learn something new
• Everyone (individually) should write
  • Who did what for each task (proposal presentation, progress presentation, project presentation, final report) in detail
  • Any grievances
  • Any praises and special recognitions
  • Due date: same as the final report evaluation