G e o m e t r i c and F u n c t i o n a l A n a l y s i s

1016-443X/92/0300341-3951.50+0.20/0

Vol. 2, No. 3 (1992)

~) 1992 B i r k h ~ u s e r Verlag, Basel

THE BILLIARD IN A REGULAR

POLYGON

WILLIAM A. VEECH

List o f C o n t e n t s 1. 2. 3. 4. 5. 6. 7. 8. 9.

Introduction Aff(w~) Representation of Aft(an) on H1 Representation of E(n) in SL(2,ZN), N--- ~(~) Identification of Fo(n)=p(Eo(n))(modN) Enumeration of the cusps Residue formulae Residue calculation (Part I) Residue calculation (Part II)

1. I n t r o d u c t i o n It is possible to associate to a regular planar n-gon Pn a nonuniform lattice F,, C_ G = PSL(2, R) and to express the zeta function of the length spectrum of the billiard in Pn as a holomorphic linear combination of the Eisenstein series associated to the cusps of Fn ([V]). One infers from the Ikehara Tauberian theorem that there exists c~ > 0 such that the growth function of the length spectrum satisfies t2

N(P~,t),~c~,,p,,l, II

(t ---, cx~)

(1.1)

(llpn]l = area(Pn)). The values c3 = 1 and c4 -- 3 can be calculated directly from elementary considerations of lattice points. However, the value of cn, n > 4, Research supported by NSF-DMS-8521620 and by L'Institut des Hautes Etudes Scientifiques.

342

W.A. VEECH

GAFA

will be seen to depend rather subtly u p o n the prime factorization of n, and this dependence seems not to be so easily exhibited. In preparation of the s t a t e m e n t of the main result, introduce a multiplicative n u m b e r theoretic function c(it) ( c ( m n ) = c(m)c(n) whenever ( m , n ) = 1), by giving its values for prime powers, n = p~, u > 0, as Tn i c(n) =

Also, let r

6n 2 1 + l - - n - : '~

(n = 2") (it =

2<

P

prime)

.

(1.2)

= 1 or 2 as n is odd or even.

1.3 T H E O R E M . Let Pn be a regular n-gon, n > 4. The asymptotic relation (1.1) is true with the constant c~ given by

cn = 4 8 r

c(it) - n 3 - 2) r

( n > 4) .

(1.4)

14 In 107 and Cs = 3--~" T h e first few values of cn are c5 -- i6s__,~,_5 c6 = 16, p(p-1)(p2+l) general, i f p > 2 i s p r i m e , then cp = 48(p-2)~ ; i f n = 2", u > 2, then

The reasoning which leads to (1.4) does not apply in the cases n = 3,4 and (1.4) does not give the true values of c3,c4. A well known construction (cf. [ZK]) associates to the billiard in Pn (or any rationM angled simple polygon) a closed surface Y and an atlas w for Y (minus a finite set) with transitions z --~ z + c. Aft(w) denotes the group of orientation preserving h o m e o m o r p h i s m s 7 of Y such t h a t T is affine relative to w-charts. T h e derivative v -~ D T ( e SL(2, R), cf. [V]) determines a h o m o m o r p h i s m Aft(w) -% G, where a(r) = [:l=Dr]. P(w) = a(Aft(w)) will be proved to be a subgroup of a Schwaxz triangle group (2, n, cx~), here denoted En. (As a group of fractional linear transformations on the upper half plane En has fundamental domain a hyperbolic triangle with angles ~ and0) n~ n Define w(n) = n 1-I (1 + 1) (p = prime) and a(n) = (4, n). In Section pin 5 it will be proved t h a t if N = ~-~n), t h e n Cn

=

n~(n--1) 16(n--2)Tr"

[ E n : F(w)] = w ( a ( N ) )

N = r

(1.5)

This fact determines the Poincar6 covolume of F(w) because En has covolume - ~ l r . In Section 6 we shall determine the n u m b e r of cusps,

Vol.2, 1992

THE BILLIARD IN A REGULAR POLYGON

C(n), of r(w)

343

to be

C(n)=s(n)

H O~ap

pap~2( 1 + ~ )

H

even

2PI'P/21'

(1.6)

apodd

where n = l i p aP (it is a curious fact that when n = 2 ~, there are exactly p two cusps, while if n is a square of an odd integer, there are more than cusps). Continuing with Pn and w, the length spectrum of w is defined to be the set of lengths, with multiplicities, of equivalence classes of parallel closed w-geodesics. (This length spectrum coincides with the billiard length spectrum, provided a certain convention is made concerning holonomy (Remark 1.11). If C(n) is the number (1.6) of cusps of F(w), there exist entire functions fj such that the zeta function of the length spectrum, ((w, s), satisfies c(~) ~(w,2s) = E fj(2s)Ej(i,s), i = x/L-f, where Ej is the Eisenstein series j=l

associated to the jth cusp [V]. Sections 7-9 are devoted to evaluation of the residue at s -- 1: c(n) ReSs=l ~(w,2s)--

V(F(w)) -1 ~

fj(2) .

(1.7)

j=l

V(.) stands for covolume, and this is computed, as indicated earlier, using (1.5). The sum on the right-hand side of (1.7) presents a more difficult problem for calculation. The surface Y above sits naturally on an N-sheeted unbranched cover of a surface X of genus [ - ~ ] , equipped with its own (singular) flat structure u. The covering group (~ ZN) is normalized by Aft(w) (Section 2), and therefore we focus on the group Afr0(u) C_Aft(u) of maps which admit lifts to Y. Afro(u) satifies F(w) = a(Afr0(u)). Let J be a singular set of u, and let A be the canonical representation of Aft(u) on H1 (X, J) (Z coefficients). In Section 4 we exhibit a base A for H1 (X, J) and a finite index subgroup E C Aft(u) such that (1) a(E) -- a(Aff(u)) -- F(u) and (2) calculation of A in the base A reveals a representation A(E) ~ SL(2, Zn) with E0 def E ["1 Afr0(u ) = )~-ltt-lS g where SN C SL(2, ZN) is a lower triangular subgroup. This (unanticipated) representation is of critical importance for the evaluation of the sum on the right in (1.7). Appartently intractable calculations with A(E) are reduced to manageable ones with p()~(E)).

344

1.8 Remark:

W.A. VEECH

GAFA

Let c(n) be the sequence defined in (1.2), and set up the oo

generating function Q(s) = ~ c(n)n -8 (set c(1) -- 1). An elementary n----1

calculation, which will not be given, yields an expression for Q(s) in terms of the Riemann zeta function: (20

-Jr 2 s ) ( 2 s - 4

--

1) ((s - 4)((s - 2)

Q(8) = ( 2 s _ l _ 1)(8s_ 3 _ 1)

(1.9)

1)

Q(s) is holomorphic for Re s > 5 with a simple pole at s = 5, and Q(s) is otherwise analytic for Re s -- 5. From (1.10) we have Res Ss=hQ(S) = 52 r 45 4 ( 4 ) " N ~r 4 As ~(4) = ~6, the Ikehara Wauberian theorem implies lim ~ ~ n - - - ~ OO

c(~) n4

~t ~--~1

~-~" 1~ ~ k )-

(It is not difficult to verify this directly using the definition (1.2) of c(n).) Theorem 1.a, especially (1.4), then implies the sequence {c n 172 ~_~3} in (1.1) satisfies lim 1 L N-~N

e(n)cn n3

13 - 67r5r

) .

(1.10)

n=3

1.11 Remark: Let P be a simple planar polygon, and let 7 be an unoriented closed billiard trajectory in P. If 7 makes an odd number of reflections before (first) closing, then nearby parallel trajectories will close only after travelling twice the distance, with twice as many reflections (see the figure below).

Therefore, we adopt the convention that the length assigned to a closed trajectory 9' is its euclidean length or twice its euclidean length as 7 makes an even or odd number of reflections before (first) closing. With this convention each periodic trajectory is interior to a maximal band of parallel periodic trajectories of equal length. Each side of the band is comprised of a union of billiard trajectories which have vertices on either end. (An

Vol.2, 1992

THE BILLIARD

IN A REGULAR

POLYGON

345

additional convention m a y be adopted when one angle of P is rr/m, but this does not enter for Pn, n > 4.) When the angles of P are rational multiples of 7r, the length s p e c t r u m associated to the m a x i m a l bands of closed (unoriented) trajectories coincides with the length s p e c t r u m of a corresponding w above. An i m p o r t a n t theorem by H. Masur [M] contains the statement that when the angles of P are rational multiples of 7r, the growth function N(P, t) satisfies a "Tchebychev theorem", at 2 <7 N(P, t) < bt 2, large t. It is u n k n o w n and perhaps unlikely that all such P enjoy a prime closed trajectory theorem, i.e., an asymptotic formula N(P, t) ~ ~ (t --~ oc), much less what (other) geometric significance the constant c might have. In view of the expression (1.10) for ~(3), the question of (further) geometric interpretation is of (potential) interest even for ca, n > 3.

1.12 Remark:

The author thanks P. Sarnak ([$2]) for the observation that if the group Fn in the first paragraph of this introduction has no "small" eigenvalues, the asymptotic relation (1.1) is true with error at m o s t 0(t4/3). This follows from results in [G], especially Theorem 4 (p. 116). In this regard it m a y be noted that the lattices associated to certain isosceles triangles are Hecke groups, i.e. (2, n, cx))-Schwarz triangle groups, and these are known to have no small eigenvalues ([S1]). In particular, if n > 4 is odd, and if T~ is an isosceles triangle with equal angles 7r/n, then [V] combines with [S1,2] and [G] to yield N(T.,

t) =

1) 48(n -- 2)7r [IT,t ~2I-----~+ 0(t4/3) "

(1.13)

It would be interesting to know whether O(t 4/3) can be improved in this case. If F(n) is a (2, n, oc)-Schwarz triangle group, F(n) admits a presentation (a, b la 2 = 1 = b") from which one readily constructs surjective homomorphisms p : r ( n ) --* PSL(2, Z,~). If n is odd, the group Fn (above) associated to the regular n-gon can be seen to contain ker p for one such p. It would be of interest to determine whether ker p in general admits small eigenvalues since a negtative answer would have implication for F~, n odd.

2. AU(w.) To fix dependence upon n the objects Y and w of the introduction will be denoted Y(n) and Wn. It is a standing assumption that n > 4. We begin below with the definition of (Y(n), w,,) and with the observation that it

346

W.A. VEECH

GAFA

occurs as an ~--~-sheeted( ) u n b r a n c h e d cover Y(n) ~

X ( n ) q*un = w,~,

where ( X ( n ) , u~) is a pair which arises in [V]. (Recall t h a t e(n) = 1 or 2 as n is o d d or even). We then prove 2.1 PROPOSITION. With notations as above Aff(wn) normalizes the (cyclic) group of the cover Y ( n ) ~-L X ( n ) . If A f r o ( u , ) is the set of a E Aff(un) which admit lifts ~ to Y(n), then (a) a(~) = a(a), a E Affo(Un), and (b) if Fo(u,,) -- a( Affo(U~)), then

r(w.) = r o ( u . ) .

(2.2)

T h e picture (2.3) below illustrates the definition of Y(n) is the cases n = 5 a n d 6. 2 2 3

3

1

5

5

2

5

6

3

3 4

6 1

1

4

1

5

1

1

5

5

3

3

Figure (2.3) T h e picture in each case consists of two columns, each containing parallel congruent regular n-gons. Pairs of p o l y g o n s in different c o l u m n s are

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R

347

POLYGON

congruent by translation followed by 180 ~ rotation (and by translation when n is even). Each edge with label k in one column is glued by translation to the unique parallel edge with label k in the opposite column. There are n equivalence classes of vertices whether n is even or odd. Each vertex class is characterized by the common labels ((k, k + 1) or (n, 1)) of adjacent edges. Y(n) is the surface determined by the gluings just defined, and w~ arises naturally from C. w~ is singular at the vertex classes because n > 4. Define g : Y(n) ~ Y(n) by sending the top polygon in each column to the b o t t o m polygon in the same column and otherwise sending each polygon to the one above it in the same column. Each m a p is translation so that Dg = Id as an affine map in w,,-coordinates, g is fixed point free and has order - ~ ) , exactly. Let X(n) be the quotient of Y(n) with deck group generated by g, and let u~ be the w~-quotient structure. Denote the covering by Y(n) -L ~ X(n). The direction definition of ( X ( n ) , u ~ ) is illustrated below for n = 5, 6:

v

w

x

0

v

y

y

v

w

Figure (2.4) Edges with the same label are glued by translation. To fix notation, let P(n) be the regular polygon whose vertices are the n th roots of unity, and let Q(n) -- - P ( n ) . We assume X(n) and Y(n) are constructed from copies of P(n) and Q(n). Observe that when r/is computed using wn and un charts, it is affine with D r / = Id. If~ E R2-{0}, and if ~ ( w n ) and ~ ( u n ) denote the foliations of the respective nonsingular sets determined by lifting the foliation of R 2 by lines parallel to ~, then r/~r = ~'~(un); moreover, this relation persists when each foliation is oriented by ~. 2.5 LEMMA. With notations as above let ~ be a saddle connection for ~(un). Then r/-16 is a union of N = ~ saddle connections for ~ ( w n ) on each of which r~ is one-to-one. In particular, if t3 is any one of these latter connections, r/-16 = {9k13 [ 0 < k < N}.

348

W.A. VEECH

GAFA

Proof: Choose a connection/3 for ~(w,~) such t h a t r//3 = 6. If u E Z, and if /3N gU/3 contains a nonsingular point, the fact/3 and g~/3 are connections for the same foliation (~d~(wn) = g ~ ( w n ) ) implies g~/3 --/3. As g~ preserves leaf orientation of ~ ( w n ) , gu fixes the endpoints of 8. As g~ is affine on /3, g~ = Id on/3. Two affine maps, ~ and Id in this case, agree identically as soon as they have the same derivative and agree at a nonsingulax point. Therefore/3~ = Id, and the l e m m a obtains. Before proceeding with the proof of Proposition 2.1 we digress with an indication of its strategy. Let ~ l ( u n ) and ~l(Wn) be the vertical foliations, i.e. ~ with ~ e R*(~ Given T e Aff(w~), it will be established that there exist al E Aff(un) and saddle connections 6, 61 for ~ l ( u n ) such t h a t T?7--1~ = ?~--10"1~ 1 ,

(2.6)

and also such t h a t Dal (o) = cOy(O) for some c > 0 (actually, c = 1). We assume these facts for now. L e m m a 2.5 implies there exists a saddle connection fl for ~ ( w ~ ) , f = Da(~), such t h a t r/-la161 = {gk/3 I 0 _< k < N}. Let a be a saddle connection in ~-16. From (2.6) and what has gone just before there exist k and e such that v a = gk/3 and TgOz = g t ~ . Therefore, if u = e - k, we have 7goz = g~TOZ. It now follows, as in the proof of L e m m a 2.5, t h a t Tg -~- gU'r. This is the main assertion of Proposition 2.1. (The remaining assertions, that a(~) = a(a) and F0(un) = r(w~), follow because D q = Id and Dg = Id.) In preparation of the proof of the above reduction of Proposition 2.1 we introduce some notation and recall some facts from [V]. Let m~ = m~ (n) = [~ ]. If n > 4, ~1 (u,,) decomposes X (n) into m l m a x i m a l cylinders of closed leaves which we denote by U 1 , . . .,U~m. 1 If n is even, let ~2(un) be ~ ( u ~ ) for the vector f = c 2=i/n - 1. ~2(u~) decomposes X(n) into m2 = m l - 1 m a x i m a l cylinders of closed leaves, U 2 , . . . , U2~2. It is a general rule that each side of OU~ consists of two saddle connections of unequal length. The exceptions to this rule are listed below with the present numbering k --* U] a s s u m e d to be compatible with numbering in [V], Section 5: (i) (ii)

n odd n even

U~

U~, U;1~/2

one connection on each side one connection on each side

(iii) (iv)

~ odd ~ even

U~nq.2)/4

two connections of equal length on each side

U12/4

two connections of equal length on each side (2.7) following l e m m a is contained in L e m m a 8.11 of this paper. ThereThe fore, the proof will not be given in this section.

Vol.2, 1992

THE BILLIARD IN A R E G U L A R P O L Y G O N

349

2.8 LEMMA. Let notations be as above. If n is even, then both q - l u ~ and r/-1U,1/2 consist of a single maximal cylinder for ~l(w,). If n/2 is even,

then for every a E Aff(un) the set q-lcrU2n/4 is a union of n/2 maxima/ cylinders = Do(~ In preparation of the statement of L e m m a 2.9 let T E Aff(w,), and define ~l(r) = Dr(~ If n is even, let ~ -- e 2~i/n - 1, as a real column vector, and set ~2(T) = DT~. If 1 < j _ s(n), ~ j ( ~ ) ( w , ) = rRdi(wn ) has (only) closed leaves and saddle connections, and therefore ~ . ( ~ ) ( u , ) = ~/~r also has closed leaves and saddle connections. Sections 2,3 and 7 of [V] imply there exist k(j), 1 <_ k(j) <_ e(n), and aj E Aff(Un) such that aj~k(j)(u~) = ~j(~)(u,~), 1 <_ j <_ c(n). The l e m m a to follow has non-trivial content only when c(n) = 2. 2.9 LEMMA. With notation as above k(1) = 1. That is Gl~l(ltn)

= a~,(r)(ltn)

(2.10)

9

Proof: It is no loss to suppose e(n) = 2. If hi2 is odd, we claim (2.7) implies ~lrRdl(wn) ~ a~2(u,~) for all a E Aft(u, 0. Indeed, being affine b o t h ~r and a scale distances along leaves of foliations. This implies that properties such as inequalities between lengths of saddle connections are preserved. Therefore ~/r~l(W~) has at least one cylinder which is b o u n d e d by saddle connections of one fixed length, while a~2(un) has no such cylinder. Therefore, k(1) = 1 (and k(2) = 2) when hi2 is odd. If hi2 is even, apply L e m m a 2.8 to conclude that q-1alU2/4 is a union of hi2 maximal cylinders isometric to a1U2/4; on the other hand, L e m m a 2.8 also implies ~F~,(~)(w~) has two cylinders on each of whose sides are hi2 saddle connections of equal length. As hi2 > 2, it m u s t be that k(1) = 1. The l e m m a is proved. 2.11 LEMMA. With notation and assumptions as above

r l-lU

=

l-lalU 1

T(•-I(u11L.J Unl/2)) = 7]-1((71 vl LJ(71U1/2)

(n odd)

(• even).

(2.12)

Proof: If n is odd (resp. if 4In), (2.7) and (2.10) imply each side of the first line (resp. second line) of (2.12) contains exactly those maximal cylinders for ~r whose boundaries are composed of saddle connections of equal length. Therefore, (2.12) holds in these cases. If n is even, and if hi2 is

350

~V.A. VEECH

GAFA

odd, (2.7) (iii) implies each cylinder in l]-lfflU~n§ has an even number of saddle connections of equal length on each side. On the other hand because n/2 is odd (2.7) (ii) implies each cylinder on either side of (2.12), line two, has an odd n u m b e r of saddle connections of equal length on each side. It follows that (2.12) is true in this case, and the l e m m a obtains. 7 E Aft(w.). With notation as above there exist saddle connections 6 and bl [or ~l(U,,) such that

2.13 LEMMA. Let

~--lal~ 1 = T~--16 .

(2.14)

Proof:

Because C r l ~ l ( t t n ) -~ ~ [ ~ l ( r ) ( U n ) , i t is t r u e t h a t D a l ( ~ ) = c ~ l ( v ) , c E R*. If c < 0, replace al by a l a , where a exchanges the polygons (2.4) with a 180 ~ rotation, and reletter so that c > 0. If n is odd, (2.12) implies Trl-IU ) = rl-ialu~. If n is even, L e m m a 2.8 implies both rl-lU~ and ~-1U1/2 consist of a single cylinder. In this case (2.12) implies there is a choice k = 1 or n/2 such that 7rl-lu~ = ~-lalU~. In order to work in this notation set k = t when n is odd. The four cylinders U 11, ~lU~, 1 7/-1U~ and vr/-tU~ all have left hand sides determined by the orientation of ~1(7). The choice of al and definition of ~l(T) imply that all maps above, i.e., a l , T, ~] and g m a p left sides to left Sides. Let 5 be the left hand side of U 1, and let 51 be the left hand side of U~. Then ~-r/-15 = r/-la151, and the l e m m a is proved. Proposition 2.1 is now a consequence of L e m m a 2.13 and the discussion which follows the proof of L e m m a 2.5.

3. Representation of Aff(u,~) on/'/1 Let n > 4, a n d let X ( n ) : (P(n) U Q(n))/,-, be as in Section 2 ((2.4)). If r = 1 or 2 as before, the set J(n) C X ( n ) of vertex classes has r elements. Orient the edges of P(n) counterclockwise and label them 7 1 , - . - , "r,, as indicated in the figure below for n = 5, 6:

Vol.2, 1992

THE BILLIARDIN A REGULARPOLYGON

75

351

7~,3.~ 75

71

74

S 2

73

~

76

J 72

75 Figure (3.1)

The set F = {Tj [ 1 < j < n - 1} projects to a base for H1 (X(n), J(n)) (integer coefficients). We identify 7j with its class [Tj]. (F is not to be confused with F(un), F(wn).) Of course,

n--1 'Yn = -- ~_~Tj 9 j=l

(3.2)

Let Z (n) denote the Z-linear span of F, Z (n) ~ H1 (X ( n ), J (n) ). If T is a homeomorphism of X(n) such that -rJ(n) = J(n), then T.: Z(n) ---* Z(n) is well defined. Let A(r, F) E GL(n - 1, Z) be the matrix defined by

n-1 T*Tj = E AijTi

(Aij = Aij(r,F)) .

(3.3)

i=l

In what follows we shall recall the definitions of homeomorphisms ~o,0 and when n is even, ~, from [V]. The group they generate will be denoted

352

W.A. VEECH

GAFA

E(n),

and we recall also that F(un) -- a(E(n)). Our interest now is in the representation (3.3) of E(n) in GL(n - 1, Z). For unity of notation we shall write n = 2r + e, where c = g(n) and

r =r(.) = [~]. Introduce cycles vj,

n-jTe--2 vj = 7j + 7 n - j + ~ - I + 2

E

(l_
7i

(3.4)

i=j+l

uj represents the core of the cylinder

U~+e_j, in

the notation of Section 2. n--1

W h e n e = 2 and j = 1, replace the t e r m 7n in (3.4) by - ~

j=l

7j, as in (3.2).

Then n--1

~1 = Z ~,

(~(~) = 2 ) .

(3.5)

i=2

W h e n n is even, define cycles #k, 1 < k < r, by the formulae

n-k #k=Tk+l+2

E

71+Tn-k+l

(l
(3.6)

i=k+2

#k represents the core of the cylinder U~+l_k, 2 again in the notation of Section 2. W h e n k = 1, use (3.2) to find n--1

.~ = -~

+ ~

~.

(3.7)

i=3

The h o m e o m o r p h i s m p effects a Dehn twist about the core of each cylinder U], 1 _< j _< r + c - 1. p can be defined to be affine, but for the m o m e n t we require only its action on H1, which is given by

"fj-q-l/j ~*Tj=

73"-ua-j+~-i

(1 < j < r + c - -

(r+c
1) (3.8)

The minus sign on the second line of (3.8) is due to the fact that 7j passes from right to left across its cylinder when r + e _< j < n. Using the definition (3.4) of uj, (3.8) becomes

I 2 n_J+~i~2~i + "fn--j-l-e--1

l_
~o.Tj =

--2 i=n--j+E~i -- "fn-j+e-1

r+e
Vol.2, 1992

353

THE BILLIARD IN A REGULAR POLYGON

W h e n ~ = 2 and j = 1, (3.5) yields for ~,71 (as do (3.9) and (3.2)) n--1 ~*~fl ---~ E ~i 9 i=1

(3.10)

T h e matrix A(~, F) is determined by (3.9)-(3.10). W h e n n is even, the h o m e o m o r p h i s m ~b is defined to effect a Dehn twist about the core of each cylinder U 2, 1 < k < r. An elementary calculation yields { "fj -[- ]Aj_ 1

7j-P,~-j+I

1 _< j _< r + 1 r+2_
(P0 = 0) (#~+1 = 0 )

(3.11)

It is useful to express (3.11) in terms of ~ , . One finds ~b,Tj = Yv,Tj + 7~-j+1 + 7~-j+2

(1 < j < n - 1 , 3'n+1 = 71) 9 (3.12)

For example, (3.6) and (3.4) imply for 1 < k _< r -t- 1 ]2k--1 ~-~ lYk Jr- ' ) ' n - k + 1 nu " ) ' n - k + 2

and (3.12) holds for 1 < j _< r + I. The remaining cases are easily checked. Finally, the h o m e o m o r p h i s m 0 is defined by rotating b o t h P(n) and Q(n) counterclockwise through an angle 27r/n. 0, satisfies

0,7j

=

{Tj+n~ 1 _ %

4. Representation of

l_
E(n) in

SL(2,IN), N -

(3.13)

~-~

In this section we make a change of base in Z(n) and observe t h a t when {7, [ r e E ( n ) } are represented by matrices with respect to the new basis, there arises a natural representation of E(n) into SL(2, ZN), N -- ~(~). Calculations involving F(un) and P0(un) are m a d e tractable by this representation. Let B be the (n - 1) x (n - 1) matrix defined by

1

(4.1)

354

W.A. V E E C H

GAFA

and use B to define A = {51,... ,~n-1} in terms of F: n-1

5k----EB~kT~

(1 < k <

n-l)

.

(4.2)

B -1 is the matrix B ~ 1 = [B,k[. Indeed, n--1

n--1

E [ B a b l B b c = E ( - - 1 ) b + c ( b - 1 ) 1( b=l

b-1

b=l

(1]--_c

c- 1 a 1

_l)b

C-- a b a

b= a

----" (~ac

(Kronecker 5). If A(% A) is the matrix of ~-, with respect to the base A, then of course A ( T , A ) = B -1A(T,F)B. In what follows we use the notation (1) R(A) for the first two rows of the matrix A and (2) xq to denote a number x modulo q. In particular, 0q stands for a number which is zero modulo q. 4.3 LEMMA. Let ~, 0 and r (n even) be as above. Let N = ~-~). Then

{ (-o-4ooo oo) 4~5n0n...0n (Lln- 4n0n...0n ln3n0n...0,~

(lnOnOn...On) R(0, A) = \ l ~ l n 0 n . . . 0 S

R(r

)

(~(n) = 1) (4.4)

(~(n)=2) (4.5)

]

(1,, - 4 ~ 0 n . . . 0 n

= \ 0~ln0n...0n )

(~(n)=2).

(4.6)

(1 _< j _< n - 1)

(4.7)

Proof: Define ( n - 1)-vectors a(T) and c~(T) by n-1

ai(~) = Z ~,~(~,r) i----1

i=l

1

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

355

The first (resp. second) row of R(v, A) is a(T)B (resp. a(T)B). If 7 = ~o or r = 2), t h e n (3.9), (3.10) and (3.12) imply aj+I(T) = ad(T ) --4, j < n--2. (It is only necessary to c o m p u t e this for j = r + e - 1 in (3.9), i.e. for the transition from line one to line two.) Therefore, aj(T) = al ( T ) + 4 - - 4 j , 7 = ~O or r = 2). Prom the form (4.1) of B it is i m m e d i a t e t h a t (a(r)B)k = O, 2 < k _ n-l, T = ~ o r r Also, by direct calculational(~o) = - 3 n or - l n as s(n) = 1 or 2, and al(~b) = ln. T h e corresponding values of a2 are a2(~9) = - 7 n or - h n a n d a2(r = - 3 ~ . Therefore, ( a ( ~ ) B ) l = - 3 ~ or - l n , (a(~)B)2 = - 4 n , a n d ( a ( O ) B ) l = 1,~, (a(~b)B)2 = - 4 n . If T = 0, then a(O) = ( 1 , 1 , . . . , 1 , ) a n d a(O)B = ( 1 , 0 , 0 , . . . , 0 , 0 n ) . T h e relations (4.4)-(4.6) are therefore correct in the first rows. To c o m p u t e c~(c2) use (3.9)-(3.10). A simple calculation yields o~j (99)

~---

n - j + a - 2 + {(n - j + c - 2)(n - j + e - 3) - (j - 1)(j - 2)}

= (7 - 2c)j + ( c - 2) 2 - 2

(modn)

(1 _< j _< r + ~ - 1) (4.8)

and c~j(~) = - ( n - j + e - 2 ) -

{ ( j - 1 ) ( j - 2) - ( n - j + e - 1)(n - j + e = ( 7 - 2e)j + ( ~ - 2) 2 - 2

1) . (4.9) Again the form of B implies (a(T)B)k ---- 0n, 2 < k _< n - 1. Also, cq(~o) = {(7-

+

-

- 2}.

(modn)

2)}

= 4n or 1 . and " 2 ( V ) -

(r + ~ _< j < n -

I(V) = 5 . or 3 . as

c(n) = 1 or 2. In the case of r = 2), (3.12) and (4.8)-(4.9) combine to imply cU(r = (j - 1), a n d a ( r is the second line of (4.6). Finally, a direct calculation shows that c~(0) = ( 1 , 2 , . . . , n - 2 , - ( " 2 1 ) ) = (1,2,...,n2, ( n - 1)N), N = ~(~). Therefore, c~(O)B = (1, 1 , 0 , . . . ,0,0N). T h e l e m m a is proved. Recall t h a t E(n) has b e e n defined to be the group generated by p, 0 and r = 2). It is i m m e d i a t e from (4.4)-(4.6) t h a t if 7 E E(n), and if p(r) is the 2 x 2 m a t r i x Pij = Aij(T,A), then r ~ fl(r)N (i.e. p(T) ( m o d N ) ) is a representation of E(n). We define F(n) = p(E(n)) ( m o a N ) . Define (~(q) = (q, 4). 4.10 LEMMA. With notation as above

F(n)={(;

d ) ESL(2, ZN) l b = O ( m o d a ( N ) ) }

.

(4.11)

356

W.A. VEECH

Pr~176I f a ( N ) = l ' t h e n F ( n ) c ~ because the latter matrix is p(r

GAFA

(11 01 ) and 1 ( 0

-4)1 . Thisis

when e(n) = 2 and p(O)p(~)p(O)-1 when /-

N O W assume N is even, and let Fl(n) be the set on the right of (4.11). It is evident from (4.4)-(4.6) that F(n) C Fl(n). Let n = 2kq, q odd. From the first part of the proof there exists, for each AI 9 F~ (n), a A 9 F(n) such that A -- At (modq). Therefore, we may and shall restrict attention to A~ 9 Fl(n) such that A~ =_ I (modq). A1 has the form

A1 = ( 1 + O/q a(N)/3q 7q

1 + t~q )

(a(N) -- 2 or 4) ,

and in particular (1 + a q , N) : 1. Choose t so that 7q = 2 ( l + a q ) ( m o d N ) and m so that a(N)q(1 +aq)m +/3) = 0 ( m o d N ) . If ~ =

(10 o(N)) 1

( 9 F(n)), then p(O)eAl~mq = Diag(cl,c2)(modN). We must

therefore prove Diag(cl,c2)(modg) 9 F(n). To this end observe that - I = p(O)p(~)p(r and therefore it is no loss to suppose cl = c I - - 1 and # = -c2,. Then p(O)'~p(O)~ -c~ 1 ( m o d a ( Y ) ) . Define , = o(g) Diag(cl, c2) (mod N). The lemma is proved.

4.12 Remark: Let F ( n ) C_ SL(2, Z) be the group generated by p(0),p(p) and p(r (n even). The canonical representation .~(n) ~-~ SL(2, ZN) determines a representation of E(n) as we have seen. If N is even, the canonical representation F(n) ~ SL(2, Zn) does not determine a representation of E(n). The reason of course is that the second row of R(0, A) contains one (last) element which is 0N but not On. On the other hand, because the element R12(T,A) is even, for all T, SO that R12('r, A)0N = On, ~n(p(xy)) and rn(p(x))rn(p(y)) have the same first rows. Also, for all ~" e E(n) it is true that Rlk(T, A) = On, 2 < k _< n - 1, and R12(T, A) = 04.

5. I d e n t i f i c a t i o n of FoCn) = p(Eo(n)) ( m o d N ) This section is devoted to certain calculations concerning the cover Y(n) X(n) discussed in Section 2. Before we begin, it is necessary to make one remark for even n: (Recall that J(n) C_ X(n) is the puncture set.)

Vol.2, 1992

THE BILLIARD IN A REGULAR POLYGON

357

5.1 LEMMA. Assume n is even, and let T be an orientation preserving homeomorphism of X ( n ) such that TJ(n) = J(n). I f u = ur = (1, - 1 , 1 , . . . , - 1 , 1) E Z ~-1 , there exists e = e(r) = 4-1 such that

u r A ( r , r ) = e(r)ur

(s(r)

(5.2)

= 4-1) .

n--1

Proof: I f 7 = •

xjTj E Z(n), then 07 = u r . x ( [ v l ] - [ v 2 ] ) ,

wherex =

j=l

( X l , . . . , xn-1) and [vj], j = 1, 2, are the vertex classes. If Zo(n) = {7 [ 07 = 0}, then Zo(n) = ur~ Cl Z(n). As T.Zo(n) = Z0(n), there exists r E Q such that uFA(T,F) = e(~-)ur. Clearly, e(T) E 7, and as det A = 4-1, it must be that C(T) = 4-1. The lemma is proved. The cover r/is determined by a character X on H1 ( X ( n ) ) (integer coefficients) with values in the group ZN = Z / N Z , N - ~n)" Inspection of (2.3) reveals that if n = N is odd, and if wr = (1, 1 , . . . , 1) E Z n - l , then

0

n--1

x(7) = w r . x (modn) j=l

If n is even, and if 7 E Zo(n), then w r . x(7) E 2Z, and one finds

X ( 7 ) = ~FW 1

. X (modN)

For even n define Yr = 89

X(7) = ~ r - x

(7 = ~ xjTj e Zo(~)).

(5.4)

+ u r ) E Z n - l . Then (5.4) can also be written

(modN)

(7 = ~

xjTj e Zo(n) .

(5.5)

j=l

5.6 LEMMA. Let ~- be an orientation preserving homeomorphism of X ( n ) such that TJ(n) = J(n). The necessary and sufficient condition that r admit a lift to Y ( n ) is that there exist c E Z*N and, when n is even, d E -s such that w r A ( r , V ) = cwr ( m o d n ) (n odd)

~ r A ( r , I ' ) = c~r + dur (modN)

(n

even)

(5.7)

358

W.A. VEECH

GAFA

Proof: If ~" admits a lift, then T,(kerx) = ker X. This implies that T, induces an automorphism of Z ( n ) / k e r x , n odd, or Z o ( n ) / k e r x , n even. In either case the quotient is ZN, and the induced automorphism is given by multiplication by c E Z~v. Define z = wrA(T, F) - cwr, n odd, and z = ~ r A ( T , F ) - C~r, n even. Then z . x(7 ) = 0 ( m o d n ) , 7 e Z(n), n odd, and z . x(7) = 0 ( m o d g ) , 7 E Zo(n), n even. The first relation implies z = 0 ( m o d n ) , n odd, and the first line of (5.7) holds. As for even n, define "s = z - zlur, where zl is the first component of z. Now ~- z(7) = 0 ( m o d g ) for all 7 E Z(n). (Z(n) is spanned by 3'1 and Zo(n).) Therefore, 2 = 0 (mod Y), and the second line of (5.7) is true. Conversely, if (5.7) is true, with c E Z~v and d 9 Z, then certainly T.(ker X) = k e r x , and therefore r admits a lift to Y ( n ) . The l e m m a is proved. In what follows we use the notation wa, u a and ~ a for the vectors w r B , u r B and WrB. Referring to the definition (4.1) of B we have

= (1,0,...,0) u~ = ( 1 , - 2 , 4 , . . . , 2 n-2)

eve ) even).

WA = ( 1 , - - 1 , 2 , . . . , 2 "-3 )

(5.S)

An i m m e d i a t e consequence of (5.7)-(5.8) is 5.9 LEMMA. Let r E E(n), and suppose p(T) =

d

" If n is odd,

necessary and sufficient conditibn that T E Eo(n) def E ( n ) M Aff0(un) is b = 0 (modN) .

(5.10)

Proof: For any T E E ( n ) wzxA(r, A) _-- (a, b, 0 n , . . . , 0n) in the notation of Section 3. Therefore, (5.7) holds if and only if b = 0 (mod n). The conclusion of the l e m m a is true for even n as well, although the proof is slightly more complicated. 5.11 LEMMA. Let n be even, and otherwise let the notation and assumptions be as in L e m m a 5.9. Then T e Eo(n) if, and only if, (5.10) holds. Proof: Fix T E Eo(n), and choose r = r that u A A ( r , A ) = cu~x

= 4-1, c E Z~v and d E l such

"~AA(T, A) -- c~,x q- du~x ( m o d N ) .

Vol.2, 1992

359

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

Substitute the first of these equations in the second to obtain

89

A) = -~wa c + -c--~ uea

+ dua

(modN) .

The first row of A(T, A) has the form, by L e m m a 4.3 and Remark 4.12, (a, b, 0 n , . . . , 0n) with b well defined m o d N . Therefore, if we equate second and third coordinates above, b

=-(c-c-2d)

(modN)

i0 2 n----0g----2(c--r

(modN).

It follows b -- 0y, as claimed. Conversely, suppose b - ON. If N is odd, choose a e Z~v so t h a t 2a = 1 ( m o d N ) . Then ~ = a(wa + u a ) ( m o d N ) , and -~AA(T, A) ---- (ha, ab, O n , . . . , 0n) + C~XUA. It follows that (5.7) holds with c = a a and d = ac. Finally, assume that N is even and that fl(T) s a t i s f i e s b = 0 ( m o d g ) . The first row of A(T, A) has the form (a, b, 0 n , . . . , On), and moreover 41b (Remark 4.12). As (a,b,N) = 1, a + b is odd, and our assumption b = 0N implies c = a + b E Z~V. Define d

@,,A(T,A) =

,-~,ON,...,ON

-=-~CWA-{--~ r

~(r)-a-b2 . T h e n

+ -~uA , UA

=c~a+~ r = c~a + du,~

--

(modn)

u~ (modN) (modN).

The l e m m a is proved. Recall the definition F(n) = p ( Z ( n ) ) ( m o d N ) from Section 4.

We

bj)EF(n)ib:0 N~. In what follows we shall ) (\ C determine the size of Fo(n)\F(n). Introduce the notation U(m) = {(a,b) E Z 2 I (a,b,m) = 1}. It is

now d e f i n e F 0 ( n ) :

~(a

equivalent to require that there exist (c, d) E Z 2 such t h a t det

d

=

l ( m o d m ) . Define the relation (a',b') " m (a",b") to m e a n there exists s e Z * such that a" = sa' ( m o d m ) and b" = sb' ( m o d m ) . Define K ( m ) H ( m ) / '~m. Also, let Sm be the lower triangular subgroup of G m = SL(2, Zm).

360

W.A. VEECH

GAFA

5.12 LEMMA. T h e n o t a t i o n as a b o v e K(m)

Sm\Gm

~

Proof: The canonical map S m \ G m

is onto by the remark on equivalence in the paragraph which procedes the 1emma. It will be sufficient to establish t h a t if x = that

--, K ( m )

(a :)x, (: 01)xx c

'

=

E G m , there exists u such

d

The equations to solve are ua = c' - c (mod m) u b = d' - d

.

(modm)

Express a and b as a = a l o : and b = blfl where (c~, m) = 1 = (fl, m) and every prime factor of a l b l is a factor of m. Of course (al, bl) = (a, b, m) = 1. From a d - bc = ad t - bc' (mod m) we obtain d' - d a

bl

c' - c -

f l - -

,

al

(5.13)

where, as the notation suggests, d'-d c'-c E Z. Let 7 = (aft) -1 E Z*, bl ' a l and define u = 7ad'b: d = 7 f l da-l c ua = uolal = 7 f l ~c'-c olal The l e m m a is proved.

From (5.13) and the definitions we have

= c' -- c (mod m). Similarly, ub = d' -- d (mod m).

Returning to H ( m ) , defined in the paragraph before the lemma, it is clear t h a t (m, n) = 1 implies H ( m n ) = H ( m ) N H ( n ) . This plus the Chinese Remainder Theorem imply there is a canonical isomorphism K ( m n ) ~- g ( m ) x g ( n ) , (re, n) = 1. In particular, m I g ( m ) l is a multiplicative n u m b e r theoretic function. 5.14 PROPOSITION. Define w ( m ) = m 1-I (1 + ~), the p r o d u c t b e i n g over Ptm the p r i m e f a c t o r s of m . T h e n IK(rn)l =w(m)

.

(5.15)

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

361

Proof: Suppose m = p~, u _> 1, p prime. The following set of classes for Nm covers g ( p ~ ) : {[(0,1)1, [(pC,b)], 0 < c < v (b,p) = 1 and [(1,b)], 0 <_ b < p ~ } . I f 0 < c < u, and i f ( b , p ) = 1, t h e n (pr (pC, b) if, and only if, x - 1 ( m o d p ' - C ) . There axe pc such values of x (modp~). We account for these multplicities and conclude

II,'(p )l

v--1 p~, __ p V _ l = 1 + pU +

~_,

__ pV ..l_ p V - 1 = ~ ( p V )

-pT

c=l

The l e m m a is proved. It is useful to record the cardinality of Gin. Because where ~ is Euler's ~-function, we have IGm] = m ~ ( m ) l K ( m ) I

ISml = m~(m), (5.16)

= m~(m)a~(m) . L e m m a 4.10 implies t h a t if a ( N ) = (N, 4), the canonical m a p SL(2, 7N) -~ SL(2, Z~(N) ) satisfies F(n) = ~r-IS~(N). It follows from (5.16) that

IF(n)l =

X~(N)co(N)

As Fo(n) = SN by L e m m a s 5.9 and 5.11, we have

IF~

"4N)

(5.17)

It is a consequence of the definitions that there is a canonical isomorphism Fo(n)\F(n) ~- Eo(n)\E(n) . (5.18) Also, from Section 5 of IV] we have that a (E(n)) = r(u~). If fact, we claim a(.) induces an isomorphism,

E o ( n ) \ E ( n ) ~-

(5.19)

where as before F0(un) = a(Aff0(un)). In the first place a(.) induces an onto m a p from the left side of (5.19) to the right side. If Eoal and Eoa2 have the same image, there exists a e Aff0(un) such that a(a) = a(ala21).

362

W.A. VEECH

GAFA

If a l a 2 1 a -1 9 Eo(n), then a 9 E ( n ) N A f f o ( u n ) = Eo(n) and Eoal = E0a2. W h a t is certain is t h a t a(ala21a -1) = 1, and therefore T = a l a 2 1 a -1 satisfies D v = 4- Id. Consideration of the shortest distance between vertices in the polygons (2.4) implies T maps each edge in (2.4) to itself (modulo the equivalence relation (2.4)) or to a parallel edge. There are either two (n odd) or four (n even) T with this property. In all cases 7- could be 7 = Id or the m a p which exchanges polygons with a 180 ~ rotation; when n is even, there is also the exchange of polygons by translation. One sees directly that each of these maps lifts to Y ( n ) . Therefore, T 9 Eo(n) and (5.19) is true. Collecting results we have 5.20 PROPOSITION. With notations as above there is a canonical isomorphism Fo ( n ) \ Y ( n ) ~- r 0 ( u n ) \ r ( u ~ ) . (5.21) Also,

[r(u~) : r0(un)] =

~(N) ~(~(N))

(N = ~/~(~)).

The canonical m a p 7r : F o ( n ) \ F ( n ) -+ Fo(q)\F(q) ~-

Sq\Gq

is onto, and each fiber has the same size,

I~-~xl = [~--'sq : F0(n)] The value (5.17) for is [r-lxl =

IFo(n)\F( )l

~(2 R-l) W (O'(2R-1))

_

and

2 n-1 ( 4 , 2 R-1 )

(x 9 Fo(q)\F(q)).

IFo(q)kF(q) I implies this

fiber size

(x 6 Fo(q)\F(q), n = 2Rq, q odd).

(5.22) The possibilities in (5.22) axe

[Tr-lxl-----

1

R_<3

2 R-3

R_>4

(x 6 Fo(q)\F(q)) .

(5.23)

The relation (5.23) will be useful in evaluating sums over Fo ( n ) \ F ( n ) when the s u m m a n d s are constant on each fiber ~r-lx.

Vol.2, 1992

THE BILLIARD IN A REGUL A R POLYGON

363

6. E n u m e r a t i o n o f t h e c u s p s

Let F be a nonuniform lattice in G, and let Fo C_ F be a subgroup such that

[r: ro] < ~ . Let [Aj], 1 < j < s, be the cusps of F, and suppose A~ C_ Fo is a maximal unipotent subgroup. A~ is contained in a maximal unipotent subgroup A C F, and there exist 1 < j < s and c~ E F such that A = c~Ajc~-1. ce is unique up to c~Aj, and A is uniquely determined by [A~ up to /~A/~-1, /3 E Fo. Therefore cr is uniquely determined by A~ up to Foc~Aj. On the other hand, if ~1, oL2 ~ F axe such that FoalAj r Foc~2Aj, the corresponding maximal unipotent subgroups of Fo, c~lAjc~-1 fq Fo and a2Ajc~2-1 fq Fo, are not conjugate in Fo. Therefore, the disjoint union $

=

[.J ro\r/Aj

(6.1)

.4=1

provides an enumeration of the cusps of Fo in terms of those of F. In what follows we specialize to F = F(un) and s -- s(n) (i.e. F(un) has s(n) cusps

[v]). With notation as in (6.1) for F = P(un), define Lj = a-lAj C Aff(un), and let Aj C F(n) be the image of Lj N E(n) under the map v ~ p(r) (mod N). The isomorphisms (5.18)-(5.19) determine isomorphisms

ro(u0)\r(un)/Aj

~

Fo(n)\F(n)/Aj

(1 < j _< E(n)) ,

(6.2)

by factoring through the space Eo(n)\E(n)/Lj A E(n). In practice we shall take L1 = (qo), the group generated by qa, and when E(n) = 2, we take L2 = (r We shall employ a simple counting principle. If a E F(n), define f~(a) = Fo(n)\F(n)/ (a), and let Fo(n)\F(n) _5, Fo(n)\F(n)/ (a), be the canonical map. If t(x) = caxd(Tr-lx), and if f is any function on f2(a), then :(x) xEf~(a)

1

=

(6.3)

xEFo(n)\F(n)

In particular, ]f](a)[ -- card (f2(cr)) is given by

Ia ( ~

=

xEFon\F(n)

t(

1 y) "

(6.3~)

364

W.A. VEECH

GAFA

Lemmas 5.9 and 5.11 are used to compute the value of t(Try) in (6.3)(6.3'). In terms of y = Fo(n)ay and a (in (6.3)-(6.3')) t(Try) has the value

t(~ry) : min {t > O l(ayata~l)1 ~ = 0} .

(6.4)

Of course, the '0' on the right side of (6.4) is i n / N . We shall return to (6.4) with more explicit values in the cases of particular interest. Let ~ and r (n even) have the same meanings as in Section 4, and define al -- p ( ~ ) ( m o d N ) and a2 = p ( r Of course, a2 enters only for even n. As Aj is generated by a(p) (j -- 1) and a(r ( j - - 2, n even) ([V]), the relations (6.1)-(6.2) imply ~(n) (6.5)

n = U ~(~J) 9 j=l

The elements aj can be read we replace a by a~ and reletter so that a2 =

O"1 ----_

a2 =

{( (1

(10

1

-34

-45 )

(n odd)

-11

-43 )

(n even)

"

Wooowre o

(

(6.6)

0

It is easy to check that

) ( 1 -~,4t 1-4t +4t

4= (1-2tt

(n odd)

-4t ) 1 +2t

(n even)

a(N)t ) If O'y --

(~ c

(6.7)

(n even)

1

, an easy cMculation reveals

(auata~l)12 =

-4t(a - b) 2 -t(2a - 5) 2 ta(N)a 2

(n odd, a = al) (n even, a = al) (n even, (7 = a2) 9

(6.8)

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

365

Using (., .) for g.c.d, we can now express t(Tvy) -1 for y as in (6.8) as

{

~((a-b)2,n) t( ~ry)-1 _=_ -~ ( ( 2a - b)2, N) -~(a(N)a2,N)

(n odd, a = al) (6.9)

(n e v e n , o = o l ) (n e v e n , o ----

.

We first take up the case of (6.3') for odd n > 4. Right multiplication

by p(O) = (11 01) permutes elements of Fo(n)\F(n) and transforms the function f(Foay) = ((a - b)2, n) to the function f(Foayp(O)) = (a 2, n). Therefore,

1

E (a2'n) [(a,b)]eI((n)

(6.10)

Next, we take up the second line of (6.9). Let n = 2Rq, where q is odd. Recall from R e m a r k 4.12 t h a t if N is even, and if p(T) =

(a

d

E E(n),

then 4lb. This implies (1) 2-1b is an integer and (2) a - 2-1b is odd. If N is odd, define 2 -1 E ZN using the fact 2 e Z~v and observe that in all cases ((2a - b)2g) -- (4,2R-1)((a -- 2-1b)e,q) .

(6.11)

As a function on Fo(n)\F(n), the right h a n d side of (6.11) is constant on fibers of the canonical m a p to Fo(q)\F(q). Therefore, the s u m of (6.11) over Fo(n)\F(n) can be replaced by a sum over Fo(q)\F(q) with a suitable 2R-1

factor, given by (5.22) as ~ . obtain [f~(a,)[ = ql

E

Since t(~ry) -1 = ~ ( ( 2 a -

( ( a - 2-1b)2, q)

(n even) .

b)2,N), we

(6.12)

[(a,b)]eI((q) As above the sum (6.12) can be replaced by the sum on the right side of $ (6.10) with n = q. (For example, choose v E Zq so that 2v = 1 (modq), and observe that if h(Foay) = ( ( a - 2-1b)2, q), then h(Foa~p(O~)) = (a2,q).) Collecting results, we have proved 6.13 PROPOSITION. Let n = 2nq > 4 with q odd. H a l = P(9~)(modN),

then (a2,q).

q [(a,b)]EK(q)

(6.14)

366

W.A. VEECH

GAFA

We shall finally take up the case of a2 on the third line of (6.9). This

time we ~

that if n = 2Rq' q ~

and if y = F~

( ac ~ ) ' then

((a(N)a2, N) = a(N)(a2,q). This is either because N --- q is odd and a ( N ) -- 1 or because N is even, meaning b is even and a is odd ((a,b,N) -1). Of course, a(N) -- (4, N) divides N. As before replace the s u m of (a(N)a2,i) over Fo(n)\F(n) by the s u m over Fo(q)\F(q) to obtain 6.15 PROPOSITION. With notation as above [~(a2)[ = 1 E (a2'q) " q [(a,b)le1~'(q)

(6.16)

It has been noted in the discussion which precedes Proposition 5.14 that the function q --. K(q) is multiplicative in the sense that if ql, qz = 1, then If(q1, q2) -~ K(ql) x K(q2). It follows easily that the s u m on the right of (6.14) (and (6.16)) is a multiplicative n u m b e r theoretic function of q. To evaluate (6.14) or (6.16) for q a prime power, q = p~, enumerate K(p ~) as in the proof of Proposition 5.14 and find v--1

ql

E (a2,q)_q [(a,b)]EK(q)

+q+

E (p2C,p~) pC (pV _ p V - 1 ) } c~-I

t89

~,-1

c=1 = p[89 +p~,-[ 89

1+

p~ ~'

= p[89

[89 + pU-2[89

v even .

6.17 T H E O R E M . Let n > 4, and let Fo(un) = a ( A f f o ( U , ) ) . Let n = 2R rI paAn) be the prime [actorization of n. ro(Un) has C(n) cusps, where p>2

C(n) is given by

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

367

Fo(Un) has index ~(n)~(N) in a Schwarz triangle group (2, n, c~).

7. R e s i d u e F o r m u l a s

We begin with some notation for the indexing of cusps of Fo(un). Let Lj be the stabilizer of ~j(Un) (Section 2) in E(n), and define Aj = a ( i j ) C_ F(un), 1 <_ j _< E(n). Choose/3j E G so that

Define e(n) -- C ( n ) / z ( n ) to be the number (6.14) or (6.16), i.e.

e(n) = Fo(un)\r(Un)/Ajl

(1 g j g g(n)) .

(7.2)

Then select first a complete set {ajk I 1 < j <_ ~(n) , 1 < k < e(n)} of representatives for the double coset space (s) r0\r/A~ and secondly elements ajk E E(n) such that a(ajk) = ajk. If fljk = cUk/3k, then (7.1) implies

fl'~) Ajkl3jk = A0

(Ajk = ajkAja-f• ) .

(7.3)

Set up vectors ~j and ~jk in R2, defined up to a scalar +1, as

In the notation of Section 2, ~j (Un) has mj maximal cylinders of closed leaves, denoted, along with their lengths, as U~ and h~, 1
ojk;b (un) = ; ~ . (an).

(7.5)

Let U~k = ajkU~, and let h~k be the common length of closed leaves in U~k. There exists a constant cjk > 0 such that

h{ ~ = c~h~ IliJkll = cJkll~ll

(7.6) 9

368

W.A, VEECH

GAFA

There exist integers d~ such that -1 U~J is a union of d~ maximal cylinders (of lengths ~ h ~ ) . J Similarly, T]-I U~~ is a union of d~k maximal cylinders of heights H~ k, where

H•k =

~

r$

jkcjkh~

9

(l
l
s, l
.

(7.7) jk Later on we will look more closely at d~ ; suffice it to say for now that in

general d~k # d{. Next, define A~ = Ajk f3 Fo(U~), and define an integer ujk by

. ~ = [A~: A~

(7.8)

., 1 / 2 --112~ If Ajk = -r~. ~mgtvjk , ujk ), then (7.8) and (7.3) imply ,9 ) •j k- -P~Ij-k- i ~0 lxjkPjk

jk =

A0

9

(7.9)

If ~)o = fljkAjk(lo), then I1,:o,~1t =

(7.10)

'"

Introduce the entire function fjk(s) as mj

fjk(S)

= IIq0 , E ~----1

=~jkll~;ll ~ d { ~ E(

tk

(7.11)

s

= via

H~jlls E(d{k)l+~(h~) -s 9 ~=1

For the last two lines of (7.11) we have used (7.6)-(7.7). Let E j k ( z , s ) be the Eisenstein series associated to the cusp [A~ of F0(u,~) ([K]). Let ~(wn,s) be the zeta function associated to the length spectrum of wn (on Y(n)). By [V], Section 3, we have ~(wn,2s) =

E l
fjk(2s)Ejk(i,s)

(i = X / ~ ) .

(7.12)

Vol.2, 1992

THE BILLIARD IN A REGULAR POLYGON

369

Recall that F(u.) has a flmdamental domain in the upper half plane of Poincar6 volume e(n)(.:2)zr. Proposition 5,20 implies F0(Un) has Poincar6 covolume w(N) e(n)(n- 2) ~ (n > 4) . (7.13) v(~) = ~(~(N)) As Res~=l Ejk(i, s) = V(n) -1, (7.11)-(7.12) imply

e(n)w(a(N)) n(n-2lw(N)~

Res~-l((w.,2s) = -

ms

~-, ~kll(Jll2~-~(d{k)a(h{)-2" ~<_i<_,(,,) ~=1

(7.14 / Formulas (6.1), (6.2) and (6.8) of IV] yield (

2 hj

II Jll ( e )

-2

i

~

~

~ ~

=

0=1

sin~ (2e-1)~

1

x

"

,

l
(j = e ( n ) = 2

,

1 < f < -~)

9

(7.15) The area of P(n) (see (2.4) ft.) is equal to

ile(~)l[ =

l n s i n 2__~ . 2 n

(7.16)

Use (7.15)-(7.16) in (7.14) to obtain the value of Res,=l ((w., 2s): 1

ne~=~((w~,2.) =

Res.=l ((w=, 2s) =

8(n

w(a(N)) 1 ''(M1 + Me) II 4 ( n - 2)w(N)Tr II"P(n)

M~ =

d~ ) k=l s

i~

1

~ y:(~V 2)~v(N)~ lip(~)II k=l e=l

EE = k=l t=l

~ (de)

.sin2

~1~

sin2 (~-~)"n (n odd) (7.17) (n even)

(7.18)

m~ =

(7.19)

n

~ s i n 2 2tE~ n

~-2 m2 =

370

W.A. VEECH

GAFA

7.20 Remark: Each of the sums (7.17)-(7.19) over k (with g fixed) can be viewed as a s u m over for j = 1 or 2, as appropriate. The corresponding factor vjk in each s u m m a n d is the cardinality of the preimage of the point F0(rjkAj under the canonical projection F0\ F --+ F 0 \ F / A j . In the next section the sums will be taken (for fixed e) to be sums over F0\ F, and the factor vjk will be removed thus leaving the total unaffected.

8. Residue Calculation (Part I) Let X be the character defined in (5.3)-(5.4), and let vi, #j be the cycles defined in (3.4)-(3.7). In this section we shall compute the value of X(a,~), A = v~ or #j as a function o f p ( e )

= ( ac

db ) ' a E Aft(us).

Letting

X(a,A) e ZN be represented by an integer, the n u m b e r d = (X(a,A), N) is independent of the choice of representative. (d = N when X(a,A) = 0.) Moreover, ~7-1a, A is a union of d closed curves, each of wn-length ~ times the wn-length of a,A. In order to calculate it is necessary to evaluate the first two coefficients in the expansion of ~, ), = vi or #j, with respect to the basis A from Section 4. T h e definitions of gl and #j in terms of the original basis are given in n--1

n--1

(3.4)-(3.7). Let ui = ~[~ seibe and #j = ~ Qj(be, 1 < i < [~], 1 _< j <_ n-____22 ~=I

~=i

(n even). T h e fact qo, yi = ~/i + b'i, 1 < i < [~], implies that 81i = ai(~9 )

--

1,

(8.1)

where ai(~) is defined in (4.7). Similarly,

tlj--aj+l(r

(lgjgn--~2)

9

(8.2)

8.3 LEMMA. With the notation as above

sli=

-4i 2-4i

(rood n) (modn)

tlj = - 4 j ( m o d n )

(nodd) (neven)

(n even) .

(8.4) (8.5)

Vol.2, 1992

371

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

Proof: (8.1)-(8.2) imply i --~ sli, j ~ tlj satisfy the same recursion as i -~ ai(~), j --* aj(r (see (4.7) ft.). The values (8.4)-(8.5) axe obtained

from saa and tal. The calculations for s2i and t2j the second line of (4.7) we obtain

are

made in a similar fashion. From

1 < i < - - n -72 )

s2i = ai(P) - (i - 1)

.

(8.6)

As for t2j, it satisfies t2j = ~ j + l ( r

8.8 LEMMA.

(1 < j <

~-~)

.

(8.7)

With notation as above

s2i --

4i 2i- 1

(modn) (mod n)

t2j = 0 (modn)

(n odd)

(8.9)

(n even)

(8.10)

(n even).

Proof: Substitute the value (4.8) for ai(~) in (8.6), and (8.9) follows. The discussion after (4.9) contains aj(r = ( j - 1)(modn), and therefore (8.10) follows from (8.7). 8"11 PROPOSITION" Let a e Aff(un)' and supp~

x(o..,) =

-4i(a - b)

(modn)

- ( 2 i - 1)(a- 6)

(mod.)

x(a,#j) =-2ja

(modn)

p(a) = ( ac

b)" Then

(n odd) ( . even)

(8.12)

(n even).

(8.13)

(In the second line of (8.12) use the fact b is defined m o d n to define b.)

372

W.A. V E E C H

GAFA

Proof: If n is odd, the first line of (8.12) is i m m e d i a t e from L e m m a s 8.3 and 1 8.8. If n is even, then X(') is determined by the vector ~z~ = ~(wz~ + u~) from Section 5. As UA is an eigenvector for A(a, A), and as u~ is orthogonal to Zo(n) equipped with the A-inner product, ~z~ = 89 A) = (a/2, b/2, 0~r O N , . . . , 0 g ) defines the character X o T.. Now (8.13) and the second line of (8.12) also follows from L e m m a s 8.3 and 8.8. T h e l e m m a is proved. It will be useful to alter slightly the formulas (8.12)-(8.13) for applications. If n is odd, observe that because

p(aO) = ( ac++bd b ) ( m o d n ) , we

have

k'((aO).~'i) = -4ia

(modn) .

(8.14)

If n is even, let n = 2Rq with q odd, and select X E Z such that 2X -- 1 + sq. T h e first row of p(aO l-x) has entries a + b _ bsq and b. In this regard we recall (Remark 4.12) t h a t b is even, and if N is even, then 4lb. we claim

(X((aOl-x),ui),N) = ( ( 2 i - 1 ) a , q ) = ( ( 2 i - 1 ) a , N ) .

(8.15)

If N is odd, t h e n N = q, a n d (8.12) implies the left side of (8.15) is ( ( 2 i - 1)(a-~sq),q) = ( ( 2 i - 1)a, q); if Y is even, then a is odd, ~sq is even (see above), and ((2i - 1)(a--bsq), N ) ---=((2i - 1)(a -b~))q = ((2i-1)a,q). Therefore (8.15) is established. Let us now return to the sums (7.17)-(7.18) (and (7.19)). As indicated in R e m a r k 7.20, we replace the sum over k for fixed j, which is a s u m over r o ( u n ) \ r ( u , ) / A j , by the s u m over r 0 ( u , , ) \ r ( u o ) w i t h the same s u m m a n d s

ujk. Also, the index set F 0 ( u ~ ) \ F ( u ~ ) will be replaced by the (canonically equivalent set Fo(n)\F(n). b u t for the removal of the factor

Here in principle is how the numbers d~k are computed. Let ),~ represent the core of the cylinder so that (ajk),)~ represents the core of UJtk. T h e n (see the first paragraph of this section),

U~,

(8.16) T h e prescription (8.17) is a function of Fo(n)\F(n). We now record a version of (7.17), i.e. Ress=l ~(wn, 2s) for n odd. To this end recall t h a t vi is the core of U1,1_2, (as remarked after (3.4)). 2

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

373

As-4 ( ~ ) = 2 ( 2 i - 1)(modn), it follows from (7.17), (8.14) and (8.17) and Lemmas 5.9, 5.12 and 4.10 that

t./21~ ((2A: 1)a,.)~

1

ReSs=l ~(~.,28)

8(n -

2)w(n)~TIIP(n)[I [(a,b)]Eg(n) 2"-" 2-" ~=1

sin 2 ~r2'-1 n

(n odd) (8.18) 1 = - ( 2 i - 1)(mod n).

For even n, vi is the core of V,+,_2,, 1 and 2 ( ~ ) 2

Therefore, (7.19), (8.15) and (8.17)imply ~

M1 =

E

( ( 2 e - 1)a, N) 3 si---~-(2t:----i)-~

Fo(n)\ F(n) e=l

Also, (8.13) directly implies (n-2)/2 (2ga, N) 3 E sin 2 e_. 2 ._~ V0(n)\F(n) ~=1 n Putting these together with (7.18) we have

w(a(N))

V'

Res~=l ~(w,,2s)-- 4(n - 2)~(N)~]]P(n)]]

~ (ea'N)3 sin---2~ "

z_.,

Fo(n)\F(n) e=x (n even) (8.19) If we take into account the fact for odd n that n - (2i - 1) is even, then with an additional factor 89(8.18) becomes 1 nes~=l < ( w , , 2 s ) = 1 6 ( n -

2)w(n)~rl]P(n)lI

E

n-z

(ea, n) 3

E

sin-----2~

Fo(n)\F(n) g.=l

(n odd) (8.20) In Section 9 we shall explicitly evaluate the right hand side of (8.19)(8.20).

9. Residue Calculation (Part II) In order to evaluate the resudue (8.19)-(8.20) it is necessary to evaluate the sum

7(n) =

E

Fo(n)\F(n)

sin 2 e~ ~=1

n

r

"

374

W.A. VEECH

GAFA

Let 1 < D < n be a divisor of n, and let us consider the portion of the sum (9.1) such that e = kD ,

(k,m)=l.

n = mD ,

(9.2)

Of course, 7r~= ~r•m ' and the contribution to (9.1) from values (9.2) n of ~ is n-1 (Da, N ) 3 (9.3) TD(n) = Z ~ sin2 ~k Fo(n)\F(n) (le,m)=l t~=l

m

9.4 LEMMA. I f m > 1, then

~(.~)~(m)

1

m--~

Zk----1

sin 2 ~_. kk rtl -

(t-,m)=1

3

-

(9.5)

where ~a(.) is Euler's ~o-function, and w(.) is defined in Proposition 5.14. Proof: Define 0(1) = 0, and let 0(m), m > 1, be the left side of (9.5). Lemma 6.3 of [V] implies n2

~o(m)_

-

-

1

3

rain

and MSbius inversion implies for m > 1

((-~)~-1) o(m) = ~ , ( a )

3

dim

~(d)m 2 _ ~(m)~(m)

= E

dim

3d 2

3

The lemma is proved. Next, we substitute the value (9.5) in the definition of rD(n) to find 1 Fo(n)\F(n)

Vol.2, 1992

T H E B I L L I A R D IN A R E G U L A R P O L Y G O N

375

Using (9.6) in (9.1) we have

~o)-- ~1 Z ~ (~)~ (~) DI,~ D
Z

/-o,~) ~.

(9.7)

Fo(n)\F(n)

The restriction D < n in (9.7) prevents T(n) from being a multiplicative number theoretic function. The addition of the value D = n in (9.7) would Na increase 7(n) by --~-[F(n): F0(n)], and we define T*(n) = T(n) +

1

: ~z

[F(n) : Fo(n)]N a a

(9.8)

~(~)~ (~)~o.~

~

Din Fo(n)\F(n)

Suppose now n = 2Rq, q odd. Express each D1 [q, and observe that if N is even,

(De, N) 3 =

Din in

(9.8) as D = 2CD1,

(2 c, 2R-1)3(Dla, q)3 ,

because a is odd when N is even (see Remark 4.12). Define F(0) -- 1, and ifR> 1 R

F(R) =

1E -~

~(2R-r

(2c' (RR-I))3

r

R-1

= 24 + -5 ~ 22"+~ c=O

= ~ 4 ( 7 . 2 3 a - 6.22n) .

H(R) denote the constant fiber size for the canonical map Fo(n)\F(n) -5. Fo(q)\F(q) (see (5.23)). We have in (9.8)

Let

T'(n)=F(R)H(R)E

E

~2(D) W(D)(Da'q)3"

(9.10)

DIq Fo(q)\F(q)

As [F(n) : F0(n)] = H(R)w(q) (see (5.17) and (5.22)), (9.8) and (9.10) imply for R > 0 (N = ~)

T(n) ~.(.~

F(R)

- ~1Z

Z

D]q Fo(q)\F(q)

~ (-~)~ (5)~~

n3 ~-~.

i~11~

376

W.A. VEECH

GAFA

while if n is odd r(n)

1

n3

w(n----~--3w(n------~Din

E ~(D)W(D)(Da'n)3 Fo(n)\F(n)

3

(9.12)

~(N) , (9.12), (8.20) and (9.1) yield Now since H(R)w(q) = ~(o(N))

Ress=l ((wn, 2 s ) = 4 8 ( n -

2) l[P( )ll

= 48e(n)(n -

2) ]lP(n)l I (n odd) (9.13)

where c(n) is defined for odd n by

1

c(n)--w(n)E

E

q~

(n o d d ) .

(9.14)

Din Fo(n)\F(n) Also, (9.11), (8.19) and (9.1) yield

Ress=l ~(COn,28) ---- 96(n - 2)TrllP(n)l ] = 48e(n)(n - 2)rtlP(n)l [ (n even) (9.15) where if n = 2Rq, q odd, c(n) is defined by

c(n) = (7.23R -6.22R)c(q)

(n = 2 n q , q odd)

(9.16)

using (9.14) for the value of c(q), q odd. In what follows we shall compute the value of c(n) for odd n. Recall from the discussion preceding Proposition 5.14 that m ~ K(m) is multiplicative, in the sense that K ( m l . . . i n k ) "~ K(ml) x ... x K(mk) when (mi, mj) = 1, i ~ j. It follows from this fact and (9.14) that n ~ c(n) (n odd) is also multiplicative. Therefore, to evaluate (9.14) it is sufficient to evaluate c(p ~) for p > 2 prime and ~ > 0. 9.17

LEMMA. With

the notation as above, let n = p~, p > 2 prime and

v > O. Then C(~t) -~ ~24

( 1 "~-p(p3r 1 -'-.A1 ) ]~

"

(9.18)

Vol.2, 1992

THE BILLIARD IN A REGULAR POLYGON

377

Proof:

Recall from the proof of Proposition 5.14 that the classes in K(n), n = p~ can be represented by pairs (n, 1),(1,b), 1 <_ b < n, and (pC, b), 0 < c < u, (b,n) = 1. Recall also that if c < u, then b --* [(pC, b)] is p~-to-1 from Z* to K(n). The contribution to (9.14) from terms with D = n is (of course) n 3. Therefore, we consider s u m m a n d s with D < n. In this range we have

~(~) P(~) (Da, n)3= c2(n) (a, D ) 3 D Using (9.19) the summands with a = 1 or a = n and contribute to (9.14) a total

(9.19)

1 <_D < n, Din

~(n)D (n +-~-5)

fixed,

(9.20)

Sum (9.20) over D = pa, 0 _< ,~ < v, and record the total T1 as

T1 = c2(n)n

I

n-1 2 p--if'i- + P --p2 1

"

(9.21)

Next, consider summands with a = pC, 0 < c < u, and a < 9 ' In this range (9.19) has the value over a:

~(n)a3D.

__ ~ - - ' 1

Multiply by ~

(see above) and sum

(1 _< D < n ,

_/)p2

.

O
Observe that (9.22) is valid (and zero) when D = np"

Sum (9.22) over

D = pX, 0 _< ,~ < v, to obtain a total T2:

~(n) 2 (pn(n-1)

-1 1

T2-p2_l 7:-i +p2; 1 rt 3

If n > pC = a _> 9 , then (9.19) is ~ ( n ) ~ . o v e r e l n ' ~ <_a < n:

D2

a-l E ~<~<~ aln

Multiply by ~ a

p(n)2n2p 2 ( ~ - ll

(9.23)

and sum

378

W.A. VEECH

GAFA

Sum this expression over D = p~, 0 < A < v, and record the total T3: p2 n 2 _ 1"[

~(n)2n2p { p (n - 1 )

T3=

(p-l)

p

1

n 2 p2

(9.24)

1 J" "

Finally, use the value T(n) = ,~(p-1) in (9.21), (9.23) and (9.24), and P compute c(n) = T1 "4-T2 -t- T3 + n3: n2{

c(n)=n 3 +~

p

-p%='~ + ~ n 3 .4-

=~4

(n-l)+

p2(n2 - 1) }

~- ~

+

n4{P(n_l)

-~-+-1

.

p

n2(n - 1)(n + 1)

p(p + 1)

p~(p+ 1)

{pn(n-1)

P2 n 2 - 1 }

1)} + -~

n2(n-1) (p+l

n2

~ y~: f

+ p(n + 1) + ~- - 1 + n(p + 1) - p(~ + 1) ~ )

P

+n3(n

- 1)

(l + p (-l:fi p+ l)]

Therefore, (9.18) is true for n = pp. We have proved 9.25 THEOREM. Let n > 4 have prime factorization n = 2a2 YI pap, and p•2

define c(n) be

c(n)=(7.2~o,-6.2~-o,)l-Ip",

1+ )-ClYN

"

(9.26/

p>2

Then

c(n) -- n 3

aes~=l ((Wn,2S)= 4 8 e ( n ) ( n - 2)zrHP(n)H "

(9.27)

The length spectra of w~ and the billiard in P(n) are equal and satisfy the asymptotic formula C(n)

- n3

N ( P ( n ) , t ) = g ( w ~ , t ) ,,, 48s(n)(n - 2)~

t2

IlP(,~)ll

(t ~

oo)

(9.us)

Vol.2, 1992

THE BILLIARD IN A REGULAR POLYGON

379

References [G]

[KI [M] IS1] [S2] IV]

A. GOOD,Local Analysis of Selberg's Trace Formula, Lecture Notes in Mathematics ~1040 Springer Verlag, Berlin 1983. W. KUBOTA, Elementary Theory of Eisenstein Series, Halsted Books, New York, 1973. H. MASUR, Lower bounds for the number of saddle connections and closed trajectories of a quadratic differential, preprint. P. SAaSAK, Prime Geodesic Theorems, Thesis, Stanford University, 1980. P. SAaNAK,Private communication. W.A. VEi~CH, Teichmfiller curves in moduli space, Eisenstein series and an application to triangular billiards, Inventiones Mathematicae 97 (1989), 553583.

[ZK] A.N. ZEMLYAEOV, A.B. KATOK,Topological transitivity of billiards in polygons, Mat. Zametki 18 (1975), 291-300.

William A. Veech Department of Mathematics Rice University Houston, TX 77251 USA e-mail: veech~rice.edu

Submitted: August 1991