Stereo matching is a fundamental task in vision applications. we propose an adaptive cross-scale aggregation method for stereo matching, which is introduced by solving an optimization problem. Unlike the original approach which introduces the same regularization term based on the inter-scale regularizer parameter to control the cost consistency among the multi-scales for all regions of the input images. We propose an adaptive regularization term in order to take into account the local structure of the image. For this purpose, we use the popular three-component image model to parse the reference image, then obtain the edge, texture and flat regions. Then, for each region, a regularizer parameter is defined. Experiments were conducted on the KITTI benchmark and obtained results have demonstrated the efficiency of the presented algorithm since significant erroneous disparities are effectively reduced.