CS 579: Advanced Topics in Agents Research

Spring 2016

Class Information

Time: Fridays 10:30am-1:0pm
Location: SH 124


William Yeoh
Office: SH 173
Email: wyeoh@cs.nmsu.edu
Phone: (575) 646-5666
Office Hours: Tuesdays 3:00-4:00pm and by appointment



  • Jan 22: Overview
    • Discussion of Syllabus and Class Rules [pdf]
    • How to read and critique a research paper [pdf] [pdf]
    • Templates: Critiques [pdf], project proposals [pdf]

Planning Under Uncertainty

  • Jan 29: Markov Decision Processes I
    • Labeled RTDP - Improving the Convergence of Real-Time Dynamic Programming [pdf]
      Pros: Khoi; Cons: Christabel
    • Bandit based Monte-Carlo Planning [pdf]
      Pros: Russell; Cons: Ping
  • Feb 5: Markov Decision Processes II
    • LRTDP vs. UCT for Online Probabilistic Planning [pdf]
      Pros: Khoi; Cons: Ping
    • Efficient Solution Algorithms for Factored MDPs [pdf]
      Pros: Christabel; Cons: Athena
    • Interesting read: Mastering the Game of Go with Deep Neural Networks and Tree Search [pdf]
  • Feb 12: No class due to AAAI.
  • Feb 19: Continuous-State MDPs and POMDPs
    • Symbolic Dynamic Programming for Discrete and Continuous State MDPs [pdf]
      Pros: Christabel; Cons: Russell
    • Planning and Acting in Partially Observable Stochastic Domains [pdf]
      Pros: Russell; Cons: Khoi
  • Feb 26: Reinforcement Learning
    • Double Q-learning [pdf]
      Pros: Russell; Cons: Ping
    • Deep Reinforcement Learning with Double Q-learning [pdf]
      Pros: Athena; Cons: Ping
    • Interesting read: Human-level Control through Deep Reinforcement [pdf]

Plan and Goal Recognition and Design

  • Mar 4: Plan and Goal Recognition
    • Plan Recognition as Planning [pdf]
      Pros: Russell; Cons: Ping
    • Probabilistic Plan Recognition Using Off-the-Shelf Classical Planners [pdf]
      Pros: Christabel; Cons: Russell
    • Interesting read: Goal Recognition over POMDPs - Inferring the Intention of a POMDP Agent [pdf]
  • Mar 11: Goal Recognition Design
    • Goal Recognition Design [pdf]
      Pros: Athena; Cons: Christabel
    • Goal Recognition Design for Non-Optimal Agents [pdf]
      Pros: Russell; Cons; Christabel
  • Mar 18: Spring Break

Distributed Constraint Reasoning

  • Mar 25: Complete DCOP Algorithms
    • A Scalable Method for Multiagent Constraint Optimization [pdf]
      Pros: Russell; Cons: Ping
    • Asynchronous Forward Bounding for Distributed COPs [pdf]
      Pros: Christabel; Cons: Athena
  • Apr 1: Incomplete DCOP Algorithms
    • Bounded Approximate Decentralised Coordination via the Max-Sum Algorithm [pdf]
      Pros: Ping; Cons: Christabel
    • Quality Guarantees for Region Optimal DCOP Algorithms [pdf]
      Pros: Athena; Cons: Khoi
  • Apr 8: Privacy in DCOPs
    • Analysis of Privacy Loss in Distributed Constraint Optimization [pdf]
      Pros: Christabel; Cons: Khoi
    • Max-Sum Goes Private [pdf]
      Pros: Khoi; Cons: Athena
  • Apr 15: Communication Sensitivity in DCSPs
    • Communication and Computation in Distributed CSP Algorithms [pdf]
      Pros: Athena; Cons: Athena
    • The Impact of Wireless Communication on Distributed Constraint Satisfaction [pdf]
      Pros: Athena; Cons: Athena
  • Apr 22: Dynamic DCOPs
    • Superstabilizing, Fault-containing Distributed Combinatorial Optimization [pdf]
      Pros: Khoi; Cons: Khoi
    • Explorative Max-sum for Teams of Mobile Sensing Agents [pdf]
      Pros: Khoi; Cons: Russell
  • Apr 29: Hybrid DCOP/MDP Models
    • Networked Distributed POMDPs: A Synthesis of Distributed Constraint
      Optimization and POMDPs [pdf]
      Pros: Ping; Cons: Khoi
    • Decentralized Multi-Agent Reinforcement Learning in Average-Reward Dynamic DCOPs [pdf]
      Pros: Ping; Cons: Christabel
  • May 6: Project Presentations


Instructor Information

William Yeoh
Office: SH 173
Email: wyeoh@cs.nmsu.edu
Phone: (575) 646-5666
Office Hours: Tuesdays 3:00-4:00pm and by appointment

Required Textbook

The course does not require any textbooks.

Course Overview

The course assumes that students have the pre-requisite knowledge of artificial intelligence concepts taught in Artificial Intelligence I and II (CS 475/505 and CS 575). Building upon those concepts, the course will cover advanced topics in agents and multi-agent systems including planning under uncertainty, plan and goal recognition and design, and distributed constraint reasoning.

Learning Objectives

By the end of this course, you are expected to be able to

  • use decision-theoretic models and algorithms to represent and solve planning and reasoning problems under uncertainty.
  • use reinforcement learning algorithms to solve unsupervised planning problems.
  • use planning models and algorithms to represent and solve plan and goal recognition problems as well as their design variants.
  • use distributed constraint reasoning models to represent simple distributed resource and task allocation problems.
  • understand the tradeoffs between the different agent models.

Course Topics

  • Planning under uncertainty
  • Plan and goal recognition and design
  • Distributed constraint reasoning

Attendance and Class Participation

The class participation portion of the grade will reflect how actively the student participates in class. Participation consists of attempts to answer questions asked of the class, asking questions about the material being discussed, contributing to class discussions, and taking part in classroom activities.

Students are responsible for all lecture material, handouts, and announcements given during class. Too many absences will make it very difficult for the student to complete the paper critiques and participate in the classroom satisfactorily.


Paper Critiques: 20%
Class Project: 50%
Paper and Project Presentations: 20%
Class Participation: 10%

Late submissions of projects and homeworks will not be accepted.

Grading Scale

The intended grading scale is as follows. The instructor reserves the right to adjust the grading scale.
A's (A-,A,A+): >= 90%
B's (B-,B,B+): >= 80%
C's (C-,C,C+): >= 70%
D's (D-,D,D+): >= 60%
F: < 60%

Incomplete ("I") Grades

An "I" grade may be given for possible work that could not be completed due to circumstances beyond the student's control (e.g., severe illness, death in the immediate family). These circumstances must have developed after the last day to withdraw from the course. Requests for "I" grades should be made to the instructor, but must be approved by the department chair. In no case will an "I" grade be assigned to avoid a grade of "D" or "F" in the course.


It is the responsibility of the student to know important dates such as University drop dates. Moreover, it is the responsibility of the student to officially withdraw from any class he or she intends to drop.


Students are expected to check their nmsu.edu email accounts regularly for all official coursework communications.

Students with Disabilities

Section 504 of the Rehabilitation Act of 1973 and the Americans with Disabilities Act Amendments Act (ADAAA) covers issues relating to disability and accommodations. If a student has questions or needs an accommodation in the classroom (all medical information is treated confidentially), contact:

Trudy Luken
Student Accessibility Services (SAS)
Corbett Center, Rm. 208
Phone: (575) 646-6840
E-mail: sas@nmsu.edu
Website: http://sas.nmsu.edu/

Do not wait until you receive a failing grade. Retroactive accommodations cannot be considered.

Academic Misconduct

Students should familiarize themselves with the NMSU Student Code of Conduct (found in the NMSU Student Handbook). Any violation of the Student Code of Conduct (e.g., plagiarism, cheating, etc.) will result in the student receiving a grade of "F" in this course. If you do not have a Student Handbook, this information is available here: http://deanofstudents.nmsu.edu/student-handbook.

Plagiarism is using another person's work without acknowledgment, making it appear to be one's own. Intentional and unintentional instances of plagiarism are considered instances of academic misconduct and are subject to disciplinary action such as failure on the assignment, failure of the course or dismissal from the university. The NMSU Library has more information and help on how to avoid plagiarism at http://lib.nmsu.edu/plagiarism.


NMSU policy prohibits discrimination on the basis of age, ancestry, color, disability, gender identity, genetic information, national origin, race, religion, retaliation, serious medical condition, sex, sexual orientation, spousal affiliation, and protected veterans status. Furthermore, Title IX prohibits sex discrimination to include sexual misconduct, sexual violence (sexual assault, rape), sexual harassment, and retaliation.

For more information on discrimination issues, Title IX or NMSU's complaint process contact:

Gerard Nevarez, Title IX Coordinator
Agustin Diaz, Title IX Deputy Coordinator
Office of Institutional Equity (OIE)
O'Loughlin House
1130 University Avenue
Phone: (575) 646-3635
E-mail: equity@nmsu.edu
Website: http://www.nmsu.edu/∼eeo/

Other NMSU Resources

  • NMSU Police Department: (575) 646-3311 or http://www.nmsupolice.com/
  • NMSU Police Victim Services: (575) 646-3424
  • NMSU Counseling Center: (575) 646-2731
  • NMSU Dean of Students: (575) 646-1722
  • For Any On-campus Emergencies: 911

Academic Integrity Policy

Academic dishonesty includes (but not limited to) the following:

  • Giving or receiving information during an exam.
  • Unauthorized or malicious use of computing facilities.
  • Deception or misrepresentation in a student's dealing with the instructor, teaching assistant, or grader.
  • Inappropriate collaboration on or coping of homework assignments. Students are encouraged to discuss the readings with one another, even when the discussion relates to assignments. As long as the purpose of discussion is to help the student's understanding of the material, and not to reduce or share the work, such discussion will not be deemed inappropriate.
  • Plagiarism, the submission of material authored by another person but represented as the students own work. It does not matter whether the original work author gave permission.
  • Any violation of academic integrity standards described in the student conduct code. Students are expected to be familiar with these standards.

All students are responsible for reading and following the NMSU Student Code of Conduct (found in the NMSU Student Handbook). Any violation of the Student Code of Conduct will result in the student receiving a grade of "F" in this course. If you do not have a Student Handbook, this information is available here: http://deanofstudents.nmsu.edu/student-handbook.