There is an emerging area of scientific research that explains exactly how early experiences can have life-long impacts. By understanding the role of genes and how they relate to development is known as epigenetics. 
Experiences very early in life, when the brain is developing most rapidly, cause epigenetic adaptations that influence whether, when, and how genes release their instructions for building future capacity for health, skills, and resilience.
The good news is that the research is showing that the effects can be reversed.