Background: Psychiatric ill-health is prevalent among prison inmates and often hampers their rehabilitation. Rehabilitation is crucial for reducing recidivistic offending. A few studies have presented evidence of the positive effect of yoga on the well-being of prison inmates. The conclusion of those previous studies that yoga is an effective method in the rehabilitation process of inmates, and deserves and requires further attention. Aims: The current study aimed to evaluate the effect of 10 weeks of yoga practice on the mental health profile, operationalized in the form of psychological distress, of inmates. Methods: One hundred and fifty-two volunteer participants (133 men; 19 women) were randomly placed in either of two groups: to participate in weekly 90-min yoga class (yoga group) or a weekly 90-min free-choice physical exercise (control group). The study period lasted for 10 weeks. Prior to and at the end of the study period the participants completed a battery of self-reported inventories, including the Brief Symptom Inventory (BSI). Results: Physical activity (including yoga) significantly reduced the inmates' levels of psychological distress. Yoga practice improved all primary symptom dimensions and its positive effect on the obsessive-compulsive, paranoid ideation, and somatization symptom dimensions of the BSI stayed significant even when comparing with the control group. Conclusions: Yoga as a form of physical activity is effective for reducing psychological distress levels in prison inmates, with specific effect on symptoms such as suspicious and fearful thoughts about losing autonomy, memory problems, difficulty in making decisions, trouble concentrating, obsessive thought, and perception of bodily dysfunction.