The average consensus problem of multi-agent systems (MASs) with switching directed network and time-varying delays is discussed in this paper, where the switching topologies are kept weakly connected and balanced. Firstly, the n-dimensional MASs model is reduced to (n-l)-dimensional subspace. Then, some sufficient conditions for average consensus are achieved based on reciprocally convex approach. Furthermore, by Lyapunov-Krasovskii functional method, the upper bound of time-varying communication delays is obtained, which is less conservative. Finally, numerical example illustrates the effectiveness of the proposed method.