Transfer Learning via Dimensionality Reduction

Overview
In this project, together with my collaborators, I have been developing some novel dimensionality reduction algorithms for transfer learning. More specifically, transfer learning aims at utilizing informative data in some related but different source domains to solve learning problems in a target domain. Currently, we focus on the case that the source and target domain data are represented in the same feature space but follow different data distributions. Our motivation is that if two domains are related to each other, then there may exist several common latent variables that dominate the observed data. Some of them may cause the distributions of the observations to be different, while others may not. On the other hand, some of these latent variables may preserve the original data configuration, such as statistical properties or geometric properties, while others may not. Thus, if we can uncover the latent factors that do not cause distribution change across domains and can preserve data configuration in different domains as much as possible, then the corresponding lower-dimensional latent space spanned by the latent factors can act as a bridge between domains, which is we are looking for. In the future, we will study how to learn a transfer latent space in the case that the source and target domain data come from different feature spaces, which means the types of data among domains are different.

Proposed Methods
Publications
  1. Sinno Jialin Pan, James T. Kwok and Qiang Yang. Transfer Learning via Dimensionality Reduction. [BibTex] In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI-08). Chicago, Illinois, USA. July 13-17, 2008. Pages 677-682.


  2. Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok and Qiang Yang. Domain Adaptation via Transfer Component Analysis. [BibTex] To Appear In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI-09). Pasadena, California, USA. July 11-17, 2009.
Collaborators

James T. Kwok (Assosicate Professor, HKUST), Ivor W. Tsang (Assistant Professor, NTU, Singapore), Qiang Yang (Professor, HKUST),


Datasets
References
  1. Karsten M. Borgwardt, Arthur Gretton, Malte J. Rasch, Hans-Peter Kridgel, Bernhard Scholkopf and Alexander J. Smola. Integrating Structured Biological Data by Kernel Maximum Mean Discrepancy. In BIOINFOMATICS-06.
  2. Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Scholkopf and Alexander J. Smola. A Kernel Method for the Two-Sample-Problem. In NIPS-06.
  3. Le Song, Alexander J. Smola, Karsten M. Borgwardt and Arthur Gretton. Colored Maximum Variance Unfolding. In NIPS-07.
  4. Le Song. Learning via Hilbert Space Embedding of Distributions. Thesis, the University of Sydney, Australia, 2007.
  5. Kilian Q. Weinberger, Fei Sha and Lawrence K. Saul. Learning a kernel matrix for nonlinear dimensionality reduction. In ICML-04.

Back to Sinno Jialin Pan's Rearch Project Page