The purpose of this study was to longitudinally examine the effects of stigma on the development of children living in out-of-home care situations, specifically with regards to self-esteem and antisocial behavior. Using a stratified sampling method, data from 341 children aged 11 or 12 living in out-of-home care (residential institutions, group homes, and foster homes) were collected five times, from 2011 to 2015. The results of latent growth modeling (LGM) analyses were as follows. First, individuals demonstrated different levels of stigma at ages 11 or 12, and the levels of stigma were likely to decrease over time (mean for intercept: 2.146; p < 0.001, mean for slope: −0.038; p < 0.001). Second, the initial stigma at ages 11 or 12 had a statistically significant negative effect on their self-esteem after four years. In addition, the results on stigma slopes during the five-year period between ages of 11–12 and 15–16 showed that stigma had a negative impact on self-esteem and was an influential factor of antisocial behavior. The results suggest that it is necessary to provide interventions for children in out-of-home care, and practitioners must place emphasis on early screening, monitoring children’s lives, and psychosocial health.