In the theory of random matrices, the circular ensembles are measures on spaces of unitary matrices introduced by Freeman Dyson as modifications of the Gaussian matrix ensembles. The three main examples are the circular orthogonal ensemble (COE) on symmetric unitary matrices, the circular unitary ensemble (CUE) on unitary matrices, and the circular symplectic ensemble (CSE) on self dual unitary quaternionic matrices.
The distribution of the unitary circular ensemble CUE(n) is the Haar measure on the unitary group U(n). If U is a random element of CUE(n), then UTU is a random element of COE(n); if U is a random element of CUE(2n), then URU is a random element of CSE(n), where
Each element of a circular ensemble is a unitary matrix, so it has eigenvalues on the unit circle: with 0 < θk < 2π and k=1,2,... n. (In the CSE each of these n eigenvalues appears twice.) The probability density function of the phases θk is given by
where β=1 for COE, β=2 for CUE, and β=4 for CSE. The normalisation constant Zn,β is given by
Generalizations of the circular ensemble restrict the matrix elements of U to real numbers [so that U is in the orthogonal group O(n)] or to real quaternion numbers [so that U is in the symplectic group Sp(2n). The Haar measure on the orthogonal group produces the circular real ensemble (CRE) and the Haar measure on the symplectic group produces the circular quaternion ensemble (CQE).
The eigenvalues of orthogonal matrices come in complex conjugate pairs and , possibly complemented by eigenvalues fixed at +1 or -1. For n=2m even and det U=1, there are no fixed eigenvalues and the phases θk have probability distribution 
with C an unspecified normalization constant. For n=2m+1 odd there is one fixed eigenvalue σ=det U equal to ±1. The phases have distribution
For n=2m+2 even and det U=-1 there is a pair of eigenvalues fixed at +1 and -1, while the phases have distribution
This is also the distribution of the eigenvalues of a matrix in Sp(2m).
These probability density functions are referred to as Jacobi distributions in the theory of random matrices, because correlation functions can be expressed in terms of Jacobi polynomials.
Averages of products of matrix elements in the circular ensembles can be calculated using Weingarten functions. For large dimension of the matrix these calculations become impractical, and a numerical method is advantageous. There exist efficient algorithms to generate random matrices in the circular ensembles.
- F.M. Dyson (1962). "The threefold way. Algebraic structure of symmetry groups and ensembles in quantum mechanics". J. Math. Phys. 3: 1199.
- V.L. Girko (1985). "Distribution of eigenvalues and eigenvectors of orthogonal random matrices". Ukr. Math. J. 37: 457.
- F. Mezzadri (2007). "How to generate random matrices from the classical compact groups". Notices of the AMS 54: 592. arXiv:math-ph/0609050.