The diagram algebra introduced by Brauer that describes the centralizer algebra of the n-fold tensor product of the natural representation of an orthogonal Lie group has a presentation by generators and relations that only depends on the path graph A n - 1 on n - 1 nodes. Here we describe an algebra depending on an arbitrary graph Q, called the Brauer algebra of type Q, and study its structure in the cases where Q is a Coxeter graph of simply laced spherical type (so its connected components are of type A n - 1, D n , E6, E7, E8). We find its irreducible representations and its dimension, and show that the algebra is cellular. The algebra is generically semisimple and contains the group algebra of the Coxeter group of type Q as a subalgebra. It is a ring homomorphic image of the Birman-Murakami-Wenzl algebra of type Q; this fact will be used in later work determining the structure of the Birman-Murakami-Wenzl algebras of simply laced spherical type.