Instruction and learning are increasingly based on competencies, causing a call for assessment methods to adequately determine competency acquisition. Because competency assessment is such a complex endeavor, one single assessment method seems not to be sufficient. This necessitates Competency Assessment Programs (CAPS) that combine different methods, ranging from classical tests to recently developed assessment methods. However, many of the quality criteria used for classical tests cannot be applied to CAPS, since they use a combination of different methods rather than just one. This article presents a framework of 10 quality criteria for CAPS. An expert focus group was used to validate this framework. The results confirm the framework (9 out of 10 criteria) and expand it with 3 additional criteria. Based on the results, an adapted and layered new framework is presented.