In this paper, we propose a general delay and stability performance analysis in Heterogeneous Cellular Caching Networks (HCCNs), based on queuing theory. We introduce new performance metrics in HCCNs and propose an optimization problem which minimizes the average experienced delay for users by ensuring the stability of the network. In addition, from the design perspective, we address the problem of finding the minimum cache size for the small cell base stations (SBSs) for having a tolerable average delay and also a stable network. Finally, the analytic expressions derived in this paper are validated through real trace-driven experiments on traffic of YouTube video requests.