This work proposes a stochastic approximation (SA) algorithm for downlink power control in femtocell networks based on inherent channel measurement report feedback in macrocell signaling. Without requiring extra backhaul signaling from a macro base station (MBS), a femto base station can update its downlink transmission power by overhearing typical feedback signals from nearby macrocell user equipment (MUE) to the MBS, such as the channel quality indicator and ACK/NAK signals. We analyze the algorithm convergence in detail with respect to instantaneous feedback and discounted sum estimate of the MUE rate. Furthermore, when multiple MUEs with different scheduling probabilities and rate requirements are impacted by the femtocell, we prove that a sufficiently small coefficient $\rho$ used as a discount factor for rate estimate can guarantee the convergence for our SA algorithm. To improve convergence speed and to reduce the sensitivity of our algorithm to the choice of parameter $\rho$, we further present an improved algorithm for the multiple-MUE scenario.