AI systems are moving from passive language models to autonomous agents that plan, call tools, keep memory, read content they did not author, talk to other agents, and act on their users' behalf. That shift changes the question we ask of them. A model that writes a wrong sentence is a quality problem; an agent that sends the wrong email, runs the wrong shell command, or wires the wrong payment is a security problem.
This course studies the security of such systems from three perspectives: attacks, defenses, and human oversight. We start with a systems-level picture of how an agent is actually built, then ask what an adversary can do to each of its parts—reasoning, tools, memory, retrieved content, inter-agent messages, and delegated authority—which architectural and runtime mechanisms defend them, how to tell whether a published security claim holds up, and what a human operator must be shown in order to intervene in time.
We treat this as a systems and security problem rather than an extension of prompt engineering or adversarial machine learning. The course runs on recent research papers and a research-oriented final project. There are no labs and no weekly homework; the work is reading, threat modeling, security reasoning, and open-ended research.
In the end, we expect:
This is a graduate research seminar. It requires a basic understanding of machine learning (e.g., CS 434). A prior course in security (e.g., CS 578) or in trustworthy ML (e.g., CS 499/599 | AI539) helps, but neither is required. You should be comfortable reading conference papers, and comfortable with the idea that a system's security depends on where its trust boundaries are drawn, not on how good its model is.
No programming assignments. You may still implement systems or run experiments as part of your final research project, and most strong projects do.
The University's Code of Academic Integrity applies, modified as follows:
Must: Please write down the names of students who helped you. It will not affect your score. You will learn from this practice how to credit others for their contributions—an essential skill when you collaborate later.
This is a course about AI agents, so let us be precise about using them. Generative AI tools are permitted for the final research project (code, experiment scaffolding, literature search, and copy-editing) and for understanding background material. They are not permitted to produce the substance of your paper critiques or any part of the final exam.
Whenever you use such a tool on a graded artifact, you must disclose, in the submission itself: (1) which tool, (2) what kind of interaction (brainstorming, drafting, debugging, editing), and (3) which portion of the work it contributed to. This follows the EECS guidance on AI use policies. Undisclosed use is an academic integrity violation. You remain fully responsible for every claim in anything you submit, including claims a tool produced.
Oregon State University is committed to making learning experiences accessible to all, which includes consideration of the course design, course learning outcomes, and barriers experienced by the disabled student. Disability Access Services (DAS) staff and students work together to determine reasonable accommodations. Students connected with DAS who have questions about their accommodations should reach out to their DAS Adviser. Students who have not yet completed the process for setting up accommodations, please contact DAS at 541-737-4098, visit the DAS website or email disability.services@oregonstate.edu to explore accommodations for a disability (e.g., mental health diagnoses, chronic health conditions, learning disabilities, physical conditions/injuries, etc.). Students and faculty members are encouraged to collaborate around the implementation of accommodations.
University students encounter setbacks from time to time. If you encounter difficulties and need assistance, it’s important to reach out. Consider discussing the situation with an instructor or academic advisor. Learn about resources that assist with wellness and academic success at oregonstate.edu/ReachOut. If you are in immediate crisis, please call or text the Suicide & Crisis Lifeline at 988.
All students are subject to the registration and refund deadlines as stated in the Academic Calendar.
OSU has twelve established student rights. They include due process in all university disciplinary processes, an equal opportunity to learn, and grading in accordance with the course syllabus: link.