The Hayes House was built in 1887 for Reverend George Finley Bovard who later became the fourth president of the University of Southern California. Milton Scott Wilson purchased the house in 1893. He served as Justice of the Peace for the San Gabriel Township and many weddings took place n his parlor. In 1904, his daughter, Mary Leticia, married Edwin Hayes who helped organize the first San Gabriel City Council and the San Gabriel Union Church.
Their daughter, Mary Ruth Hayes (1907-1990), was born in the house and lived there all her life. She was a teacher and administrator in the school district for more than forty years. She willed the home and its contents to the San Gabriel Historical Association. The home reflects the lives of one family in San Gabriel for nearly 100 years.
The small stone milk house was used as a jail on weekends to hold revelers from local saloons until they could be taken to the Los Angeles County Jail on Monday morning. It was later moved to the Wilson-Hayes property.