Reading Group: Multi-objective Decision Making in Security and Sustainability

 

Maintained by Jun-young Kwak. If you need any more information or have any suggestions, please contact Jun-young Kwak (junyounk at usc dot edu).

 

Key Information

• Location: SAL 222


• Time: Every Wed 2:00pm - 3:30pm


• Topics: Multi-objective Optimization, Sequential Decision Making under Uncertainty, Multiagent Planning & Learning, Human-agent Interaction/Negotiation, Social Psychology Study, etc.

 

2012 Spring Schedule

No.	Date	Speaker	Paper
1	01/18/2012	-	A survey of recent developments in multiobjective optimization (Altannar Chinchuluun and Panos M. Pardalos). ANNALS OF OPERATIONS RESEARCH, 2007

 

2011 Fall Schedule

No.	Date	Speaker	Paper
1	08/18/2011	-	1. Markov Decision Processes with Multiple Objectives (Krishnendu Chatterjee, Rupak Majumdar, and Thomas A. Henzinger). In STACS, 2006 2. On finding compromise solutions in multiobjective Markov decision processes (Patrice Perny, Paul Weng). In European Conference on Artificial Intelligence Multidisciplinary Workshop on Advances in Preference Handling, 2010
2	08/24/2011	No reading group
3	08/31/2011	-	Survey of multi-objective optimization methods for engineering (R.T. Marler and J.S. Arora). Structural and Multidisciplinary Optimization, 2004
4	09/07/2011	-	Theoretical Considerations of Potential-Based Reward Shaping for Multi-Agent Systems (Sam Devlin, and Daniel Kudenko). In AAMAS, 2011
5	09/14/2011	No reading group
6	10/26/2011	-	Strategy Learning for Autonomous Agents in Smart Grid Markets (Prashant P. Reddy, and Manuela M. Veloso). In IJCAI, 2011
7	11/02/2011	-	Regret-based Reward Elicitation for Markov Decision Processes (Kevin Regan and Craig Boutilie). In UAI, 2009

2011 Spring Schedule

No.	Date	Speaker	Paper
1	03/01/2011	-	Functional Value Iteration for Decision-Theoretic Planning with General Utility Functions (Y. Liu and S. Koenig). In AAAI, 2006
2	03/08/2011	-	1. If multi-agent learning is the answer, what is the question? (Yoav Shoham, Rob Powers, and Trond Grenager). AIJ, 2007 2. Multiagent learning is not the answer. It is the question. (Peter Stone). AIJ, 2007
3	03/15/2011	No reading group (Spring Break)
4	03/22/2011	-	1. Symmetric Primal-Dual Approximate Linear Programming for Factored MDPs (Dmitri Dolgov and Edmund Durfee). International Symposiums on Artificial Intelligence and Mathematics (ISAIM), 2006 2. Symmetric Approximate Linear Programming for Factored MDPs with Application to Constrained Problems (Dmitri Dolgov and Edmund Durfee). Annals of Mathematics and Artificial Intelligence (AMAI), 2006
5	03/29/2011	No reading group
6	04/05/2011	-	Formal Models and Algorithms for Decentralized Decision Making under Uncertainty (S. Seuken and S. Zilberstein). JAAMAS, 2008
7	04/12/2011	-	1. Point-based value iteration: An anytime algorithm for POMDPs (Joelle Pineau, Geoff Gordon and Sebastian Thrun). In IJCAI, 2003 2. Heuristic Search Value Iteration for POMDPs (Trey Smith and Reid G. Simmons). In UAI, 2004
8	04/19/2011	-	1. Where Do Rewards Come From? (Satinder Singh, Richard L. Lewis and Andrew G. Barto). In CogSci, 2009 2. Variance-Based Rewards for Approximate Bayesian Reinforcement Learning (Jonathan Sorg, Satinder Singh, and Richard Lewis). In UAI, 2010
9	04/26/2011	No reading group (AAMAS'11)
10	05/03/2011	No reading group (AAMAS'11)
11	06/14/2011	-	Towards a Unifying Characterization for Quantifying Weak Coupling in Dec-POMDPs (Stefan J. Witwicki and Edmund H. Durfee). In AAMAS, 2011
12	06/29/2011	-	1. Computationally-Efficient Combinatorial Auctions for Resource Allocation in Weakly-Coupled MDPs (Dmitri A. Dolgov, and Edmund H. Durfee). In AAMAS, 2005 2. Mechanism Design for Multi-Agent Meeting Scheduling (Elisabeth Crawford and Manuela Veloso). Web Intelligence and Agent Systems, 2006

 

Resources

1. Multi-objective Optimization

Papers:

• Markov Decision Processes with Multiple Long-run Average Objectives
  (Krishnendu Chatterjee). Foundations of Software Technology and Theoretical Computer Science (FSTTCS), 2007


• Markov Decision Processes with Multiple Objectives
  (Krishnendu Chatterjee, Rupak Majumdar, and Thomas A. Henzinger). In STACS, 2006


• On finding compromise solutions in multiobjective Markov decision processes
  (Patrice Perny, Paul Weng). In European Conference on Artificial Intelligence Multidisciplinary Workshop on Advances in Preference Handling, 2010


• Computing Optimal Stationary Policies for Multi-objective Markov Decision Processes
  (Patrice Perny, Paul Weng). In IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), 2007


• Survey of multi-objective optimization methods for engineering
  (R.T. Marler and J.S. Arora). Structural and Multidisciplinary Optimization, 2004


• A survey of recent developments in multiobjective optimization
  (Altannar Chinchuluun and Panos M. Pardalos). ANNALS OF OPERATIONS RESEARCH, 2007

2. (DEC-PO)MDPs

Papers:

• Functional Value Iteration for Decision-Theoretic Planning with General Utility Functions
  (Y. Liu and S. Koenig). In AAAI, 2006


• An exact algorithm for solving MDPs under risk-sensitive planning objectives with one-switch utility functions
  (Y. Liu and S. Koenig). In AAMAS, 2008


• Efficient Solutions to Factored MDPs with Imprecise Transition Probabilities
  (Karina Valdivia Delgado, Scott Sanner, Leliane Nunes de Barros, and Fabio G. Cozman). In AAAI, 2009


• Bounded-parameter Markov Decision Processes
  (Robert Givan, Sonia Leach, and Thomas Dean). AIJ, 2000


• Towards Exploiting Duality in Approximate Linear Programming for MDPs
  (Dmitri Dolgov and Edmund Durfee). In AAAI (poster), 2005


• Symmetric Primal-Dual Approximate Linear Programming for Factored MDPs
  (Dmitri Dolgov and Edmund Durfee). In International Symposiums on Artificial Intelligence and Mathematics (ISAIM), 2006


• Symmetric Approximate Linear Programming for Factored MDPs with Application to Constrained Problems
  (Dmitri Dolgov and Edmund Durfee). Annals of Mathematics and Artificial Intelligence (AMAI), 2006


• Bandit based Monte-Carlo Planning
  (Levente Kocsis and Csaba Szepesvari). In ECML, 2006


• Optimal Resource Allocation and Policy Formulation in Loosely-Coupled Markov Decision Processes
  (Dmitri A. Dolgov, and Edmund H. Durfee). In ICAPS, 2004


• Computationally-Efficient Combinatorial Auctions for Resource Allocation in Weakly-Coupled MDPs
  (Dmitri A. Dolgov, and Edmund H. Durfee). In AAMAS, 2005


• Resource Allocation Among Agents with MDP-Induced Preferences
  (Dmitri A. Dolgov, and Edmund H. Durfee). JAIR, 2006


• Combinatorial Resource Scheduling for Multiagent MDPs
  (Dmitri A. Dolgov, Michael R. James, and Michael E. Samples). In AAMAS, 2007


• Strategy Learning for Autonomous Agents in Smart Grid Markets
  (Prashant P. Reddy and Manuela M. Veloso). In IJCAI, 2011


• Robust Online Optimization of Reward-uncertain MDPs
  (Kevin Regan and Craig Boutilie). In IJCAI, 2011


• Regret-based Reward Elicitation for Markov Decision Processes
  (Kevin Regan and Craig Boutilie). In UAI, 2009


• Eliciting Additive Reward Functions for Markov Decision Processes
  (Kevin Regan and Craig Boutilie). In IJCAI, 2011


• Markov Decision Processes with Ordinal Rewards: Reference Point-Based Preferences
  (Paul Weng). In ICAPS, 2011


• Compact Mathematical Programs For DEC-MDPs With Structured Agent Interactions
  (Hala Mostafa and Victor Lesser). In UAI, 2011


• Efficient Solution Algorithms for Factored MDP
  (Carlos Guestrin, Daphne Koller, Ronald Parr, and Shobha Venkataraman). JAIR, 2003


• Risk-Sensitive Planning in Partially Observable Environments
  (Janusz Marecki, and Pradeep Varakantham). In AAMAS, 2010


• Formal Models and Algorithms for Decentralized Decision Making under Uncertainty
  (S. Seuken and S. Zilberstein). JAAMAS, 2008


• The communicative multiagent team decision problem: Analyzing teamwork theories and models
  (D. Pynadath and M. Tambe). JAIR, 2002


• Point-based value iteration: An anytime algorithm for POMDPs
  (Joelle Pineau, Geoff Gordon and Sebastian Thrun). In IJCAI, 2003


• Heuristic Search Value Iteration for POMDPs
  (Trey Smith and Reid G. Simmons). In UAI, 2004


• Towards a Unifying Characterization for Quantifying Weak Coupling in Dec-POMDPs
  (Stefan J. Witwicki and Edmund H. Durfee). In AAMAS, 2011


• Point-Based Value Iteration for Constrained POMDPs
  (Dongho Kim, Jaesong Lee, Kee-Eung Kim and Pascal Poupart). In IJCAI, 2011


• Closing the Gap: Improved Bounds on Optimal POMDP Solutions
  (Pascal Poupart, Kee-Eung Kim and Dongho Kim). In ICAPS, 2011


• Policy Iteration for Decentralized Control of Markov Decision Processes
  (Daniel S. Bernstein, Christopher Amato, Eric A. Hansen and Shlomo Zilberstein). JAIR, 2009

3. Multiagent Learning

Papers:

• If multi-agent learning is the answer, what is the question?
  (Yoav Shoham, Rob Powers, and Trond Grenager). AIJ, 2007


• Multiagent learning is not the answer. It is the question.
  (Peter Stone). AIJ, 2007


• A Game Theoretical Model for Adversarial Learning
  (Wei Liu and Sanjay Chawla). In IEEE International Conference on Data Mining Workshops (ICDMW), 2009


• Where Do Rewards Come From?
  (Satinder Singh, Richard L. Lewis and Andrew G. Barto). In CogSci, 2009


• Variance-Based Rewards for Approximate Bayesian Reinforcement Learning
  (Jonathan Sorg, Satinder Singh, and Richard Lewis). In UAI, 2010


• Theoretical Considerations of Potential-Based Reward Shaping for Multi-Agent Systems
  (Sam Devlin, and Daniel Kudenko). In AAMAS, 2011

4. (Sequential) Mechanism Design

Papers:

• Mechanism Design for Multi-Agent Meeting Scheduling
  (Elisabeth Crawford and Manuela Veloso). Web Intelligence and Agent Systems, 2006

5. Applications

Papers:

Links

• Sustainable Energy Project


• The MSDM Workshop @AAMAS'12


• Distributed Partially Observable Markov Decision Problems (Distributed POMDPs)


• Cancer Modeling and Markov Decision Problems


• Planning and Coordination for Multiagent Teams: Distributed (PO)MDPs and DCOPs