This study aims to investigate the synergistic effect of quaternary blends applying supplementary cementitious materials on sustainable Ultra-high Performance Concrete (UHPC) pastes. The hydration kinetics, pore structures, fresh behaviour, strength, fibre-to-matrix bond, shrinkage and environmental sustainability of 14 UHPC pastes are determined and analysed. The results show that limestone powder contributes to better environmental sustainability and fresh behaviour, but enlarged shrinkage and diminished strength, and application of silica powder is an effective measure to overcome those disadvantages. Slag cement possessing a relatively lower Ca/Si ratio (2.45) is preferred to a lower amount but finer silica (3% nano silica) in the presence of limestone powder, compared to the Portland cement with a higher Ca/Si (3.22) that needs more silica even with coarser particle size (5% micro silica). Quaternary blends with cement-slag-limestone-silica in UHPC pastes have considerable advantage of reducing embedded CO2 emission and and improving sustainability efficiency. Furthermore, positive synergies in term of strength, fibre-to-matrix bond and total free shrinkage are observed in UHPC pastes with quaternary binders compared to binary and ternary ones.