Accelerating Intra-Party Communication in Vertical Federated Learning with RDMA

MPhil Thesis Defence


Title: "Accelerating Intra-Party Communication in Vertical Federated 
Learning with RDMA"

By

Mr. Duowen LIU


Abstract

Federated learning (FL) has emerged as an elegant privacy-preserving 
distributed machine learning (ML) paradigm. Particularly, vertical FL 
(VFL) has a promising application prospect for collaborating organizations 
owning data of the same set of users but with disjoint features to jointly 
train models without leaking their private data to each other. As the 
volume of training data and the model size increase rapidly, each 
organization may deploy a cluster of many servers to participant in the 
federation. As such, the intra-party communication cost (i.e., network 
transfers within each organization’s cluster) can significantly impact the 
entire VFL job’s performance. Despite this, existing FL frameworks use the 
inefficient gRPC for intra-party communication, leading to high latency 
and high CPU cost. In this paper, we propose a design to transmit data 
with RDMA for intra-party communication, with no modifications to 
applications. To improve the network efficiency, we further propose an 
RDMA usage arbiter to adjust the RDMA bandwidth used for anon-straggler 
party dynamically, and a query data size optimizer to automatically find 
out the optimal query data size that each response carries. Our 
preliminary results show that RDMA based intra-party communication is 10x 
faster than gRPC based one, leading to a reduction of 9% on the completion 
time of a VFL job. Moreover, the RDMA usage arbiter can save over 90% 
bandwidth, and the query data size optimizer can improve the transmission 
speed by 18%.


Date:  			Friday, 8 January 2021

Time:			4:00pm - 6:00pm

Zoom meeting: 
https://hkust.zoom.us/j/93030860857?pwd=WnRTR3pQTXJ6dCtWdWo5bUM5eGpFZz09

Committee Members:	Dr. Kai Chen (Supervisor)
 			Dr. Xiaojuan Ma (Chairperson)
 			Dr. Yangqiu Song


**** ALL are Welcome ****