This paper proposes a sensorless chatter detection method for a ball screw drive system used in milling, by introducing the mechanical energy factor (MEF). The MEF is an indicator for detecting self-excited chatter, based on the power factor theory used in electric systems, and monitors the phase difference between dynamic cutting force and displacement. By monitoring the MEF, the self-excited chatter can be detected in real time with low computational load and a unique threshold, independent of the vibration amplitude. Furthermore, the MEF can be monitored without any additional sensors by applying a sensorless cutting-force estimation technique, based on the disturbance observer theory. In addition, we conducted mode decoupling to the two-inertia model of the ball screw driven stage, and derived its rigid body and vibration modes. Each mode is regarded as a one-inertia model and is independent of the other, and the MEF is monitored in the vibration mode. The validity of the proposed method was evaluated through several milling tests, and the results showed that the self-excited chatter could be detected.