Term Project

One of the course objectives is to learn how to pursue a research problem of your choice and communicate your findings with others. You will develop those skills in this class by doing a term project. There are no programming assignments in this course precisely so that this is where your implementation effort goes, and the final deliverable should read like a short research paper rather than a class report.

Strong projects in this area almost always follow: Problem → Threat model → Attack or failure → Method → Evaluation. A measurement or evaluation project ("here is why this published benchmark does not measure what it claims") is equally welcome, and often more useful than yet another attack.

Project Topics

You are encouraged to choose any topic relevant to the security of agentic AI systems, with a research problem not yet explored in the literature. You are welcome to select a topic connected to your current research: if your area is reinforcement learning, robotics, databases, or HCI, there is an agent-security question in it. Directions that are wide open include prompt injection, secure tool use, authorization and delegated authority, memory integrity, multi-agent security, agent identity, runtime enforcement, attack propagation, security evaluation methodology, and human oversight.

[Note] Identifying a research problem is part of the work. Please start the topic search as early as possible—teams that are still searching at Checkpoint 1 tend to run out of time. Your paper critiques are the cheapest source of ideas: the adaptive attack and the missing experiment are project proposals.

Teams and Sign-up

• Up to 2 students. You may work alone, but the expectations are the same as for a team of two, and you still have to put your name on the sheet.
• Please use the course Google Sheet (link posted on Canvas; you need an ONID to access it) to form a team and sign up.
• [Deadline] 09/30. If you are not on the sheet by then—alone or in a team—your final score decreases by 2%.
• If you do not have a partner by then, email me and I will help match you.

Presentations

Your team should present the research progress three times; I have the following expectations:

Checkpoint Presentation 1 — Mon. 10/19, 10 min + 5 min Q&A per team

• Identify a research problem (or your project scope).
• Do a literature review and explain the prior work relevant to your problem.
• Clearly articulate how your problem is different from the prior work.
• State an explicit threat model: what the adversary controls, knows, and wants.
• Provide your next steps.

Checkpoint Presentation 2 — Mon. 11/16, 10 min + 5 min Q&A per team

• Design a set of experiments to evaluate your idea's feasibility.
• Preliminary results—you must implement a minimal (key) functionality to demonstrate the feasibility of your idea. Negative results are fine.
• Provide your next steps.

Final Presentation — Wed. 12/02, 15 min + 5 min Q&A per team

• Deliver your final results, based on the goals you set in Checkpoint 1.
• Address the limitations of your result without being asked. The limitations section is graded.
• Present in class and submit a written report of 6-8 pages excluding references, using this report template [Note: the report is due 12/11, on Canvas]. If you wrote code, include a link to a repository.

Slides are due on Canvas before class on each presentation day (10/19, 11/16, 12/02).
The final report deadline (12/11, 6:00 pm) is firm—term grades are due right after it. If something happens, talk to me before 12/11.
The session before each presentation has no class so that your team can prepare: 10/14 before Checkpoint 1, 11/11 (which is also Veterans Day) before Checkpoint 2, and 11/30 before the Final Presentation. These are marked in the syllabus.

Grading Policy (Evaluations)

Your term project will be evaluated with the following scheme (40% in total):

• 10%: Checkpoint Presentation 1
• 10%: Checkpoint Presentation 2
• 20%: Final Presentation and Write-up
• -2%: No team on the Google Sheet by 09/30.
• +5%: Submitting your final report to a workshop or conference. Several projects from a course like this are genuinely publishable—talk to me before the end of the term if you are interested.