For a safety critical task like driving, it is very important for the driver to be vigilant at all times. In this study, we explore a driver drowsiness monitoring and early warning system, which uses machine learning techniques based on vehicle telemetry data. The proposed system can ensure safe driving by real time monitoring of driving pattern. This proves to be a very cost effective technique over biometric and camera based techniques since it doesn't involve expensive sensors. The detection of drowsy state of a driver is modeled as a binary classification issue. We outline the design methodology followed and challenges faced in developing a machine learning based classification scheme on vehicle telemetry data, which is essentially a time-series data. Even though our study is focused on an exemplary application like driver attention monitoring, the results can be equally applied for vehicle prognostics, driving style analytics and various other telematics applications in automotive industry. We have done our entire development and validation on a driver-in-loop simulation platform.