The Lyon family erected a wooden tobacco factory building at the southwest corner of Pine Street (now South Roxboro) and East Pettigrew Street. The building burned in 1884, and a brick structure designed by William H. Linthicum replaced it until a 1910 fire closed it. The factory was then most likely converted into stables, which functioned into the 1920s. The building was torn down in 1930 and replaced by the Southern Railway freight depot and office. In 1993 the Southern Railway office building was torn down in preparation for the new Durham County Detention Center, which began operation on the site in 1996.