The museum building dates back to the Japanese rule of Taiwan when it was constructed as a sugar refinery factory in 1901 and completed in 1902. The factory underwent second and third stages of construction in 1905-1911 and 1911-1945 respectively which focused on the construction of leisure and religious needs of the employees. During World War II, two factories were damaged during bomb raids. They were then restored. Due to the declining of world's sugar price, the area was turned into museum and was opened in 2006.