We consider a small cell network enhanced with the possibility of D2D communications. In such networks, there are two possible transmission modes: 1) two-hop or indirect transmission via a small cell base station (SBS) and 2) one-hop or direct transmission. Transmission via an SBS is performed on a channel allocated by the SBS, and hence it is interference-free. Direct transmission, in contrast, is performed only if a primary channel is idle, and is impaired by the interference resulted from the lack of coordination. Despite being interference-free, the indirect mode might be inefficient due to high radio resource consumption, latency, and complexity. Since implementing a central controller as well as obtaining channel/network knowledge is costly, users need to choose the optimal transmission mode in a distributed manner given no prior information. In this letter, we first characterize the utility of each transmission mode. We then solve the mode selection problem using a bandit model.