Zhi WANG (王志)

Reinforcement Learning | Robotics

I am currently an Assistant Professor at the Department of Control and Systems Engineering, Nanjing University, Nanjing, China, and has the visiting position at the School of Engineering and Information Technology, University of New South Wales, Canberra, Australia.

Previously, I received the Ph.D. degree at the Department of Systems Engineering and Engineering Management, City University of Hong Kong, Hong Kong, China, in 2019, where I was advised by Han-Xiong Li. I received the B.E. degree at Nanjing University, Nanjing, China, in 2015, where I was advised by Chunlin Chen.

Prospective students, feel free to contact me by email if you are interested in pursuing a master or Ph.D. degree in artificial intelligence / machine learning / robotics at the Department of Control and Systems Engineering, Nanjing University. (有意向在南京大学控制与系统工程系攻读人工智能/机器学习/机器人学方向的硕士/博士学位的学生, 欢迎联系!)

njuwangzhi@gmail.com  /  CV  /  Biography  /  Google Scholar  /  Github


Research

I'm interested in reinforcement learning (RL), machine learning, and robotics. Specifically, I work on how learning algorithms can scale RL agents to dynamic environments, allowing them to autonomously adapt to the non-stationary task distributions in real-world domains. This includes a wide range of topics such as incremental learning, online learning, continual learning, lifelong learning, transfer learning, model-based learning, and meta-learning.

Journal Articles

Incremental reinforcement learning in continuous spaces via policy relaxation and importance weighting,
Zhi Wang, Han-Xiong Li, and Chunlin Chen,
IEEE Transactions on Neural Networks and Learning Systems, 2019.
pdf / BibTex / code
In "Incremental reinforcement learning with prioritized sweeping for dynamic environments", an incremental learning algorithm was first proposed for RL in dynamic environments, which only worked in RL problems with a discrete state-action space owing to involving a tabular form of comparing reward functions and the prioritized sweeping process. In this paper, we design a feasible incremental learning method that incorporates with the functiuon approximation framework, thus being capable of working in continuous state/action spaces.

Incremental reinforcement learning with prioritized sweeping for dynamic environments,
Zhi Wang, Chunlin Chen, Han-Xiong Li, Daoyi Dong, and Tzyh-Jong Tarn,
IEEE/ASME Transactions on Mechatronics, 2019.
pdf / BibTex / code
Traditional RL algorithms focus on learning in a stationary environment. We propose a novel Incremental Reinforcement Learning (IRL) algorithm for learning in dynamic environments where the reward function may change over time. IRL provides an appealing option for saving a significant amount of computational resources, while the dynamic environment scenario is supposed to hold in many challenging real-world domains.

Reinforcement learning based optimal sensor placement for spatiotemporal modeling,
Zhi Wang, Han-Xiong Li, and Chunlin Chen,
IEEE Transactions on Cybernetics, 2019.
pdf / BibTex

Dissimilarity analysis based multimode modeling for complex distributed parameter systems,
Zhi Wang, and Han-Xiong Li,
IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2019.
pdf / BibTeX

Incremental learning for online modeling of distributed parameter systems,
Zhi Wang, and Han-Xiong Li,
IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018.
pdf / BibTeX

Conference Papers

Better fine-tuning via instance weighting for text classification,
Zhi Wang, Wei Bi, Yan Wang, and Xiaojiang Liu,
in: Proceedings of the AAAI Conference on Artificial Intelligence , 2019.
pdf / BibTeX / supplementary materials
Previous fine-tuning works mainly focus on the pre-training stage and investigate how to pretrain a set of parameters that can help the target task most. In this paper, we propose an Instance Weighting based Finetuning (IW-Fit) method, which revises the fine-tuning stage to improve the final performance on the target domain. IW-Fit adjusts instance weights at each fine-tuning epoch dynamically. The designed instance weighting metrics used in IW-Fit are model-agnostic, which are easy to implement for general DNN-based classifiers.

Incremental learning based subspace modeling for distributed parameter systems,
Zhi Wang, and Han-Xiong Li,
in: Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2019.
pdf / BibTeX

A novel incremental learning scheme for reinforcement learning in dynamic environments,
Zhi Wang, Chunlin Chen, Han-Xiong Li, Daoyi Dong, and Tzyh-Jong Tarn,
in: Proceedings of the World Congress on Intelligent Control and Automation (WCICA), 2016.
pdf / BibTeX


Invited Talks

Incremental reinforcement learning for dynamic environments,
School of Engineering and Information Technology, University of New South Wales, Canberra, Apr. 2019.

Learning based intelligent modeling for distributed parameter systems,
Department of Control and Systems Engineering, Nanjing University, Oct. 2018.


Service

Journal Peer Review

IEEE Transactions on Neural Networks and Learning Systems
IEEE Transactions on Cyberbetics
IEEE Transactions on Systems, Man, and Cybernetics: Systems