Depression is the most common mood disorder in the world which is also the leading cause of suicide. Depression might be diagnosed using varying modalities, as it affects many aspects of the patient, not only the patient will appear melancholic, but the patient's sleep quality and life quality might also be disrupted. In this study, we aim to extract facial landmark features unique to depression patients and build a machine learning model to classify the depression severity. The facial landmark activities considered in this study include: speed statistics of the landmarks, the standard deviation of eye pupils, and statistical features of mouth area. We found several facial landmark activity features with significant differences between healthy volunteers and depressed patients. We also successfully built machine learning models for automatic depression severity prediction.