Background Exercise is accepted as an important contribution to the rehabilitation of patients with cardiovascular disease (CVD). This study aims to better understand the possible causes for lack of consensus and reviews the effects of three exercise modalities (aerobic, resistance and combined exercise) on central hemodynamics, arterial stiffness and cardiac function for better rehabilitation strategies in CVD. Methods The electronic data sources, Cochrane Library, MEDLINE, Web of Science, EBSCO (CINAHL), and ScienceDirect from inception to July 2017 were searched for randomized controlled trials (RCTs) investigating the effect of exercise modalities in adult patients with CVD. The effect size was estimated as mean differences (MD) with 95% confidence intervals (CI). Subgroup analysis and meta-regression were used to study potential moderating factors. Results Thirty-eight articles describing RCTs with a total of 2089 patients with CVD were included. The pooling revealed that aerobic exercise [MD(95%CI) = -5.87 (-8.85, -2.88), P = 0.0001] and resistance exercise [MD(95%CI) = -7.62 (-10.69, -4.54), P<0.00001] significantly decreased aortic systolic pressure (ASP). Resistance exercise significantly decreased aortic diastolic pressure [MD(95%CI) = -4(-5.63, -2.37), P<0.00001]. Aerobic exercise significantly decreased augmentation index (AIx) based on 24-week exercise duration and patients aged 50–60 years. Meanwhile, aerobic exercise significantly improved carotid-femoral pulse wave velocity (cf-PWV) [MD(95%CI) = -0.42 (-0.83, -0.01), P = 0.04], cardiac output (CO) [MD(95% CI) = 0.36(0.08, 0.64), P = 0.01] and left ventricular ejection fraction (LVEF) [MD(95%CI) = 3.02 (2.11, 3.93), P<0.00001]. Combined exercise significantly improved cf-PWV [MD(95%CI) = -1.15 (-1.95, -0.36), P = 0.004] and CO [MD(95% CI) = 0.9 (0.39, 1.41), P = 0.0006]. Conclusions Aerobic and resistance exercise significantly decreased ASP, and long-term aerobic exercise reduced AIx. Meanwhile, aerobic and combined exercise significantly improved central arterial stiffness and cardiac function in patients with CVD. These findings suggest that a well-planned regime could optimize the beneficial effects of exercise and can provide some evidence-based guidance for those involved in cardiovascular rehabilitation of patients with CVD.