Towards Principled Sequential Decision-Making

[The talk is cancelled]

Speaker: Qinghua Liu
         Princeton University

Title: "Towards Principled Sequential Decision-Making"

Date:  Monday, 8 April 2024

Time:  10:00am - 11:00am


Abstract:

Sequential decision-making studies how intelligent agents ought to make
decisions in a dynamic environment to achieve their objectives. Its
applications span diverse fields, from controlling robots to uncovering
faster matrix multiplication algorithms and fine-tuning large language
models (LLMs). In this talk, I will delve into my research on the
theoretical foundations of sequential decision-making.

Firstly, I will talk about reinforcement learning with function
approximation, a widely employed approach for addressing decision-making
problems featuring enormous state spaces. Diverging from previous theories
largely limited to linear-like scenarios, I will demonstrate that the
classical Fitted Q-Iteration algorithm (the prototype of DQN), when
combined with the idea of global optimism, is provably sample-efficient
for a diverse array of problems involving generic nonlinear function
approximation. In the second part, I will focus on partially observable
decision-making in the framework of POMDP, a problem that has long been
considered intractable within the theory community due to numerous
hardness results. Contrary to this belief, I will reveal a rich class of
POMDPs that are of practical interest and can be solved within polynomial
samples using a variant of the classical maximum likelihood estimation
algorithm. Finally, I will turn to multi-agent decision-making in the
framework of Markov Game, where agents must learn to strategically
cooperate or compete. I will introduce a fully decentralized algorithm
capable of learning equilibria strategies with nearly minimax-optimal
sample efficiency.


*****************
Biography:

Qinghua Liu is a Ph.D. candidate in the Department of Electrical and
Computer Engineering at Princeton University, advised by Chi Jin. He works
on the theoretical foundations of sequential decision-making. His research
has developed simple and generic algorithms that provably address
fundamental challenges in decision-making, including but not limited to
large state spaces, partial observability and multi-agency, all while
providing reliability guarantees. His research has been recognized with
the Princeton SEAS Award and a Best Paper Award at the ICLR 2022 MARL
workshop.