Interference due to the sharing of common spectrum band among links and congestion due to the contention among flows sharing the same link have become obstacles to good performance in wireless networks, especially in multi-hop cognitive radio networks (MHCRNs). Consequently, the high end-to-end throughput for MHCRNs calls for a framework of cross-layer optimization design. In this paper, by taking into account the problem of joint optimal rate, power, and spectrum allocation (JORPS), we propose a new cross-layer optimization framework for MHCRNs under spectrum underlay manner using orthogonal frequency division multiple access (OFDMA). The formulation is shown to be a mix-integer non-linear non-convex optimization problem, which is NP-hard in general. To solve the problem, we develop a partially distributed solution, which has been proved to converge to the global optimum with the tradeoff complexity and convergence speed.