The city of Eksjö most likely appeared sometime in the medieval ages when it was the centre for the thing, a regional council. Its first mentioned is from the 14th century. It was given its charter in the early 15th century by the King Erik av Pommern.
In the 16th century Eksjö was one of the six Swedish cities in the province of Småland. The others were Jönköping, Kalmar, Vimmerby, Västervik, and the seat of the diocese: Växjö.
After the crowning of King Gustav Vasa in the 1524, the partisan leader Nils Dacke led a rebellion. Dacke took control of the area and was supported by the locals, also in Eksjö. After Dacke had been killed by the Royal army, his body parts were hung in public places, as to quench any notions of new uprising. Eksjö was one of the towns were body parts of Dacke were on public display. This may also have contributed to the decision of Gustav Vasa to revoke the city charter of Eksjö in 1544. There are several monuments and folk museums honouring these times around the province and in Eksjö municipality, and in the dense Smålandian forests where Dacke lived.
During the Nordic Seven Years' War Eksjö was burnt to the ground in 1568 and subsequently rebuilt at a somewhat different location. The construction was led by the Dutch Arendt de Roy.
The city emerged as a centre for the oxen trades but never really prospered and remained a small town until a heath outside the town became the point of assembly for the Småland Regiment (Smålands Regemente). The city continued to be in the center of military establishments, with the coming of the engineering battalion and the Husars of Småland, in southern Sweden, hence the lack of large industrial establishments.