CCMR: Cloud Computing Platform at HKUST

The CCMR platform is used by faculty members at HKUST and elsewhere, including the Departments of Computer Science, Biochemistry, Electrical and Computer Engineering, Physics, and Civil Engineering. The proposed system is an elastic computing platform for large-scale data analytics, scientific computing, and massively parallel simulations. It will also facilitate multidisciplinary research with data-intensive computations.

Existing computational facilities have a few drawbacks that make them inflexible and difficult to use in scientific research projects. If not planned and designed carefully, a computing system would scale poorly when the data size multiplies, the problem under investigation evolves, or the underlying technology changes. MapReduce based processing, initially created by Google, overcomes these limitations to a large extent, and Hadoop, the open source version of MapReduce, makes this technology available to the academia and research communities. However, multiple constraints imposed by the MapReduce model makes it difficult to extend this computational methodology to general parallizable computation. In multiple research projects, we feel a pressing need to construct a computing center that supports flexible computational processing with elastic resource management and reliable scalability in storage, compute power, and network performance.

These observations motivated us to construct the CCMR (Cloud-computing Center for Multidisciplinary Research) platform. Augmented with an easy-to-use control interface and a flexible parallelization framework, this platform will enable scientists from multiple disciplines to index very large bioinformatics data sets, develop high-performance data mining systems, and solve problems previously thought to be intractable. Moreover, the CCMR platform provides a realistic testbed for the research on system software for cloud computing systems.

Several groups from multiple departments (CSE, ECE, PHY, CIVL, and BICH) have found the CCMR platform useful to their research. Quite a few researchers from other institutions also expressed interest in using and potentially contributing to this platform.

Detailed information on how to use CCMR can be found on the baijia forum. Below is a list of some groups that have actively used CCMR in their research projects.

• Qiang Yang's group (CSE): data-intensive machine learning
• Lei Cheng's group (CSE): modeling, large-scale database operations, and data mining
• Xue Hong's group (BICH): bioinformatics
• Lionel Ni's group (CSE): social network analysis
• Oscar Au's group (ECE): modeling and simulation
• Weichuan Yu's group (ECE): scientific computing
• FYP projects (CSE, CPEG): web indexing and search experiments
• Lin Gu's group (CSE): system software for cloud computing