Ezzard grew up on the island of Saint Kitts and spent most of his childhood in a town known as Sandy Point. He grew up with three brothers, four sisters, a mother, and a father. His father was a sharecropper who did subsistence farming on the foothills of the mountain, growing crops and rearing animals. Throughout his childhood, while attending school, he worked on the farm to help support his family. He was a victim of child labor and suffered child abuse both in school and at home. During his childhood, he started working early (as early as two or three o'clock) in the morning. He attended school at an early age, from preschool, all the way to high school, in the town of Sandy Point. During those years, he took part in the local school sports, which eventually led to his success in representing Saint Kitts in athletics. Following his graduation from high school in 1973, he worked as an elementary school teacher, educating students in grade 4 at the Verchilds Public School, in Saint Kitts, until his departure to the United States in August of 1976, to attend Idaho State University. He earned a BS in biology with a minor in physical education from Idaho State University and, later, his MS in environmental science from New Jersey Institute of Technology. He also completed a large amount of coursework at the PhD level in education. Due to his athletic ability, he made indelible marks both in Saint Kitts and in college athletics in the sport of track and field, a remarkable feat that will stand for many years to come.