Computational Intelligence (CI) is a sub-branch of Artificial Intelligence (AI) that focus on studying adaptive mechanisms to enable intelligent behavior in complex environments. CI techniques have been successful in solving complex problems in many different knowledge areas. However, despite their usefulness, developing solutions based on CI techniques is not a trivial activity, since it involves the codification/adaptation of algorithms to specific context and problems. In this paper, we present and validate through a quasi-experiment a new paradigm to develop CI-based solutions using a more mature version of Athena (2.0): Computational Intelligence as a Service (CIaaS). Using this tool, both researchers and practitioners can design and evaluate CI-based solutions by dragging and dropping components in a visual environment, in a cloud-based platform. The results of the quasi-experiment suggest that our approach can help researchers to design and evaluate CI-based systems in a simple, reliable and fast way. © 2018 Elsevier Ltd