SCIENCE CHINA Mathematics
CrossMark
. ARTICLES .
doi: 10.1007/s11425-015-0225-7
A boundary Schwarz lemma on the classical domain of type I LIU TaiShun∗ & TANG XiaoMin Department of Mathematics, Huzhou Teachers College, Huzhou 313000, China Email:
[email protected],
[email protected] Received April 13, 2016; accepted November 17, 2016
Abstract Let RI (m, n) be the classical domain of type I in Cm×n with 1 6 m 6 n. We obtain the optimal ˚ at a smooth boundary fixed point Z ˚ of RI (m, n) estimates of the eigenvalues of the Fr´ echet derivative Df (Z) for a holomorphic self-mapping f of RI (m, n). We provide a necessary and sufficient condition such that the boundary points of RI (m, n) are smooth, and give some properties of the smooth boundary points of RI (m, n). Our results extend the classical Schwarz lemma at the boundary of the unit disk ∆ to RI (m, n), which may be applied to get some optimal estimates in several complex variables. Keywords MSC(2010)
holomorphic mapping, Schwarz lemma at the boundary, the classical domain of type I 32H02, 32H99, 30C80
Citation: Liu T S, Tang X M. A boundary Schwarz lemma on the classical domain of type I. Sci China Math, 2017, 60, doi: 10.1007/s11425-015-0225-7
1
Introduction
The Schwarz lemma is one of the most influential results in the classical complex analysis, which is widely applied in many branches of mathematical research such as geometric function theory, hyperbolic geometry, complex dynamical systems and theory of quasi-conformal mappings. We refer to [1, 4, 5, 11] for a more complete insight on the Schwarz lemma. Cartan first studied the Schwarz lemma in several complex variables in [3]. In fact, this is the wellknown rigidity theorem of Cartan [3]. Theorem 1.1 (See [3]). Suppose that Ω is a bounded domain in Cn and f : Ω → Ω is a holomorphic mapping such that f (z) = z + o(∥z − z0 ∥) as z → z0 for some z0 ∈ Ω. Then f (z) ≡ z. Look discussed Schwarz lemma of several complex variables in [17], which extends the result of Cartan [3]. This result is stated as follows. Theorem 1.2 (See [17]). Let Ω be a bounded domain in Cn , and let f be a holomorphic self-mapping of Ω which fixes a point p ∈ Ω. Then (1) the eigenvalues of Jf (p) all have modulus not exceeding 1; (2) | det Jf (p)| 6 1; (3) if | det Jf (p)| = 1, then f is a biholomorphism of Ω. * Corresponding author c Science China Press and Springer-Verlag Berlin Heidelberg 2017 ⃝
math.scichina.com
link.springer.com
Liu T S et al.
2
Sci China Math
Naturally, higher-dimensional Schwarz lemma at the boundary has attracted the attention of many mathematicians as well. From Theorem 1.1, Burns and Krantz [2] first considered a new rigidity problem of holomorphic mappings at the boundary points. They obtained a rigidity theorem for holomorphic mappings on the bounded strongly pseudoconvex domain in Cn . Huang strengthened the Burns-Krantz result for holomorphic mappings with an interior fixed point in [9]. See [8, 10] for more on these matters. Here is one of typical results in these papers. Theorem 1.3 (See [9]). Let Ω ⊂⊂ Cn (n > 1) be a simply connected pseudoconvex domain with C ∞ boundary. Suppose that p ∈ ∂Ω is a strongly pseudoconvex point. If f : Ω → Ω is a holomorphic mapping such that f (z0 ) = z0 for some z0 ∈ Ω and f (z) = z + o(∥z − p∥2 ) as z → p, then f (z) ≡ z. It is natural to consider that what are the multidimensional generalizations of the boundary Schwarz lemma corresponding to Theorem 1.2. It is this problem which motivated our study in this paper. Recently, we first established the Schwarz lemma at the boundary of the unit ball in Cn , and gave some applications in the geometric function theory of several complex variables in [16]. We discussed the same problem on the unit polydisk and the strongly pseudoconvex domain, respectively in [15, 18]. Our main purpose here is to study the boundary Schwarz lemma on the classical domain of type I. We also study the properties of the smooth boundary points of the classical domain of type I. The rest of the article is organized as follows. In Section 2, we give some characterizations of the smooth boundary points of RI (m, n), which will be used in the subsequent section. In Section 3, we establish the Schwarz lemma at the boundary on the classical domain of type I, i.e., we obtain the optimal estimates of the eigenvalues of the Fr´echet derivative of holomorphic self-mappings of RI (m, n) at a smooth boundary fixed point of RI (m, n). In Section 4, we present some conclusions of the paper.
2
Preliminaries
2.1
Notation
Let Cm×n be the set of all m × n complex matrices with 1 6 m 6 n. For any Z, W ∈ Cm×n , the inner product and the corresponding norm are given by ⟨Z, W ⟩ =
m X n X
1
∥Z∥ = ⟨Z, Z⟩ 2 ,
zij wij , i=1 j=1
where Z = (zij )m×n and W = (wij )m×n . It is well known that as real vectors in R2mn , Z and W are orthogonal if and only if ℜ⟨Z, W ⟩ = 0. Throughout this paper, denote by Z ′ and Z, respectively, the transpose and the complex conjugate of Z. Let A be a square matrix of order m and let B be a square matrix of order n. Then for each Z, W ∈ Cm×n , ⟨AZ, W ⟩ =
m X n X m X
m X n X
(AZ)kl wkl = k=1 l=1 m X
m X n X
=
zil
aki zil wkl i=1 k=1 l=1 m X n X
aki wkl =
i=1 l=1
k=1
′
′
zil (A W )il = ⟨Z, A W ⟩,
i=1 l=1
where A = (aij )m×m . Similarly, we have ′
⟨ZB, W ⟩ = ⟨Z, W B ⟩,
′
⟨Z, AW ⟩ = ⟨A Z, W ⟩,
′
⟨Z, W B⟩ = ⟨ZB , W ⟩.
Moreover, if U is a unitary square matrix of order m and V is a unitary square matrix of order n, then ⟨U ZV, U W V ⟩ = ⟨Z, W ⟩. The classical domain of type I, denoted by RI (m, n), is defined as ′
RI (m, n) = {Z ∈ Cm×n : Im − ZZ > 0},
Liu T S et al.
Sci China Math
3
where Im is the unit square matrix of order m, and the inequality sign means that the left-hand side is positive definite. It is easy to check that RI (m, n) is a bounded convex circular domain in Cm×n . Let ∂RI (m, n) be the boundary of RI (m, n), and write C1×m as Cm . Let B m ⊂ Cm be the open unit ball under the Euclidean metric. The Minkowski functional ρ(Z) of RI (m, n) is defined by ρ(Z) = max{∥αZ∥ : α ∈ ∂B m },
Z ∈ Cm×n ,
where ∂B m is the boundary of B m . By [7], we know that ρ(Z) is a Banach norm of Cm×n , (ρ(Z))2 is ′ the largest eigenvalue of ZZ , and RI (m, n) = {Z ∈ Cm×n : ρ(Z) < 1}. In particular, RI (1, n) is just the open unit ball B n for which the Minkowski functional is ρ(Z) = ∥Z∥. For the unitary square matrix U of order m and the unitary square matrix V of order n, it is easy to see that Z ∈ RI (m, n) ⇔ U ZV ∈ RI (m, n) and Z ∈ ∂RI (m, n) ⇔ U ZV ∈ ∂RI (m, n). ′
Since Z ∈ RI (m, n) means that the elements in the principal diagonal of Im − ZZ are positive, we have |zij | < 1 for i = 1, . . . , m and j = 1, . . . , n. We also obtain ρ(U ZV ) = ρ(Z) for each Z ∈ Cm×n . Assume that f : RI (m, n) → Cm×n is a holomorphic mapping. The Fr´echet derivative of f at a ∈ RI (m, n) is defined as m X n X
(Df (a)(W ))ij = s=1 t=1
∂fij (a)wst , ∂zst
W ∈ Cm×n .
It is easy to see that Df (a) is a linear transformation from Cm×n to Cm×n and df (Z) |Z=a = Df (a)(dZ). The adjoint transformation of Df (a) is denoted by D∗ f (a), i.e., ⟨D∗ f (a)(Z), W ⟩ = ⟨Z, Df (a)(W )⟩,
Z ∈ Cm×n ,
W ∈ Cm×n .
Obviously, D∗ f (a) is also a linear transformation from Cm×n to Cm×n . Moreover, (D∗ f (a)(Z))ij =
m X n X s=1 t=1
∂f st (a)zst . ∂z ij
In fact, let eij ∈ Cm×n be a matrix which has 1 at i-th row and j-th column, and 0s elsewhere. Then
(D∗ f (a)(Z))ij = ⟨D∗ f (a)(Z), eij ⟩ = ⟨Z, Df (a)(eij )⟩ = Z,
·
m
n
X X ∂f ∂f st (a) = (a)zst . ∂zij ∂z ij s=1 t=1
It is clear that λ is an eigenvalue of Df (a) if and only if λ is an eigenvalue of D∗ f (a). 2.2
Smooth boundary points of RI (m, n)
˚ ∈ Cm×n , by [13] we know that Z ˚ has the following polar decomposition, i.e., For Z
˚= U Z
r1 0 · · · 0 0 · · · 0 0 .. .
r2 · · · .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
V,
0 0 · · · rm 0 · · · 0 where r1 > r2 > · · · > rm > 0, U is a unitary square matrix of order m and V is a unitary square matrix of order n. We give the properties of the smooth boundary points of RI (m, n).
Liu T S et al.
4
Sci China Math
˚ ∈ Cm×n has the polar decomposition above. Then Z ˚ is a smooth Proposition 2.1. Suppose that Z boundary point of RI (m, n) if and only if 1 = r1 > r2 > · · · > rm > 0. Furthermore, ρ(Z) is holomorphic ˚ and the gradient of ρ at Z, ˚ i.e., about Z and Z near Z,
1 0 ··· 0 0 ··· 0 0 ··· .. . . . .
0 .. .
˚ =U ∇ρ(Z)
0 .. .
0 ··· .. . . . .
0 .. .
V
0 0 ··· 0 0 ··· 0 ˚ with ⟨Z, ˚ ∇ρ(Z)⟩ ˚ = 1. is the unit outward normal vector to ∂RI (m, n) at Z ˚ ∈ ∂RI (m, n) if and only if r1 = 1. Suppose that 1 = r1 > r2 > · · · > Proof. It is easy to verify that Z ′ rm > 0. Assume that the characteristic polynomial of ZZ is ′
′
′
Φ(x, Z) = det(xIm − ZZ ) = xm − tr(ZZ )xm−1 + · · · + (−1)m det(ZZ ). ˚ Notice that Then (ρ(Z))2 is a root of Φ(x, Z) near Z. ∂Φ ˚ = [(x − 1)(x − r2 ) · · · (x − r2 )]′ |x=1 = (1, Z) m 2 ∂x
˚ = 0, Φ(1, Z)
m Y
(1 − rk2 ) > 0.
(2.1)
k=2
It follows from the implicit function existence theorem that (ρ(Z))2 is holomorphic about Z and Z ˚ and satisfies (ρ(Z)) ˚ 2 = 1. Hence, ρ(Z) is also holomorphic about Z and Z near Z. ˚ Now, we near Z, ′ 2 2 ˚ Since Φ((ρ(Z)) , Z) = det((ρ(Z)) Im − ZZ ) ≡ 0 near Z, ˚ we have compute ∇ρ(Z). ∂Φ ˚ ˚ ∂ρ (Z) ˚ + ∂Φ (1, Z) ˚ = 0. (1, Z)2ρ( Z) ∂x ∂z ij ∂z ij This, together with (2.1), implies m Y
2
(1 − rk2 )
k=2
∂ρ ˚ ∂Φ ˚ = 0. (Z) + (1, Z) ∂z ij ∂z ij
(2.2)
On the other hand, ′
′
′
Φ(x, Z) |(x,Z)=(1,Z) ˚ = det(xIm − ZZ ) |(x,Z)=(1,Z) ˚ = det(Im − U ZZ U ) |Z=Z, ˚
′
′ ˚Z ˚U = Im − U Z
···
0
0 1 − r22 · · · .. .. .. . . .
0 .. .
0
0
0
2 · · · 1 − rm
0
′
′ ˚Z ˚ U ) is and the algebraic cofactor of the element at s-th row and t-th column for det(Im − U Z 8 <
Jst =
:
m Q
(1 − rk2 ),
(s, t) = (1, 1),
k=2
0,
(s, t) ̸= (1, 1).
Thus, we obtain m
X ∂ ∂Φ ′ ′ ˚ =− (1, Z) (U ZZ U )st |Z=Z˚Jst ∂z ij ∂z ij s,t=1
=−
m Y k=2
(1 − rk2 )
∂ ′ ′ (U ZZ U )11 |Z=Z˚ ∂z ij
Liu T S et al.
=− =− =− =−
m Y k=2 m Y k=2 m Y k=2 m Y
(1 − rk2 ) (1 − rk2 ) (1 − rk2 )
m Y
5
m n X m XX
∂ ∂z ij
ul1 zls z ts ut1
l=1 s=1 t=1
˚ Z=Z
m X
ul1˚ zlj ui1 l=1 m X
m X
ul1 ui1
uls rs vsj s=1
l=1
(1 − rk2 )ui1 v1j
k=2
=−
Sci China Math
(1 − rk2 )
0 .. .
U
k=2
1 0 ··· 0 0 ··· 0 0 ··· .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
V
,
0 0 ··· 0 0 ··· 0
ij
˚ = (˚ where U = (uij )m×m , V = (vij )n×n and Z zij )m×n . It follows from this and (2.2) that
˚ ij = 2 (∇ρ(Z))
∂ρ ˚ (Z) = ∂z ij
0 .. .
U
1 0 ··· 0 0 ··· 0 0 ··· .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
0 0 ··· 0 0 ··· 0 That means
.
V ij
1 0 ··· 0 0 ··· 0 0 .. .
˚ =U ∇ρ(Z)
0 ··· .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
V = ̸ 0.
0 0 ··· 0 0 ··· 0 ˚ Moreover, since ⟨U ZV, U W V ⟩ = ⟨Z, W ⟩ for any Z, W ∈ Cm×n , we Hence, ∂RI (m, n) is smooth near Z. ˚ ∇ρ(Z)⟩ ˚ = 1. get ⟨Z, ˚ is a smooth boundary point of RI (m, n). Assume Conversely, suppose that Z 1 = r1 = r2 > · · · > rm > 0. ˚ have the same direction. Now, we Then any two nonzero outward normal vectors to ∂RI (m, n) at Z ˚ in Cm×n : consider the following two different (2mn − 1)-dimensional real affine spaces through Z ˚ + U αV : α ∈ Cm×n , ℜα11 = 0} and Σ1 = {Z
˚ + U αV : α ∈ Cm×n , ℜα22 = 0}. Σ2 = {Z
To simplify our notation, set 0
1
0
C
B B0 B B B0 B B .. @.
1 0 0 ··· 0 0 ··· 0
T1 =
B B0 B B B0 B B .. @.
0 0 · · · 0 0 · · · 0C 0 .. .
0 ··· .. . . . .
0 .. .
0 ··· .. . . . .
C 0C C .. C C .A
0 0 0 ··· 0 0 ··· 0
and
T2 =
1
0 0 0 ··· 0 0 ··· 0
C
1 0 · · · 0 0 · · · 0C 0 .. .
0 ··· .. . . . .
0 .. .
0 ··· .. . . . .
C
0C C. .. C C .A
0 0 0 ··· 0 0 ··· 0
˚ + U αV ∈ Σ1 , we have ℜ⟨U αV, U T1 V ⟩ = ℜα11 = 0. This shows that U T1 V is a normal vector For any Z ˚ (see Figure 1). to Σ1 at Z
Liu T S et al.
6
Sci China Math
U T1V
Z
U WV
Σ1 RI (m, n)
Figure 1
˚ The affine tangent space and normal vector to ∂RI (m, n) at Z
˚ On the other hand, for each U W V ∈ RI (m, n) Similarly, U T2 V is also a normal vector to Σ2 at Z. we obtain µ
˚ − U W V, U T1 V ⟩ = ℜ ℜ⟨Z
U
0
1
1 0 0 ··· 0 0 ··· 0
B B0 B B B0 B B .. @.
1 0 ··· 0 0 ··· 0 r3 · · · 0 0 · · · .. .. . . .. .. . . . . . . . .
¿
C 0C C 0C CV .. C C .A
− U W V, U T1 V
0 0 0 · · · rm 0 · · · 0 = 1 − ℜw11 > 0. This implies that RI (m, n) is located on one side of Σ1 , i.e., Σ1 is an affine tangent space to ∂RI (m, n) ˚ Similar to the above proof, we know that Σ2 is an affine tangent space to ∂RI (m, n) at Z ˚ as well. at Z. ˚ Because Z is a smooth boundary point of RI (m, n), this contradicts with Σ1 ̸= Σ2 . Therefore, we have 1 = r1 > r2 > · · · > rm > 0. The proof is complete. 2.3
Some lemmas
Let ∆ be the open unit disk in the complex plane C. The following lemma is the classical Schwarz lemma at the boundary. Lemma 2.2 (See [5]). Let f : ∆ → ∆ be a holomorphic function. If f is holomorphic at z = 1 with f (0) = 0 and f (1) = 1, then f ′ (1) > 1. Moreover, the inequality is sharp. If the condition f (0) = 0 is removed, then one has the following estimate instead: f ′ (1) >
(2.3)
1−f (0) f (z)−f (0) 1−f (0) 1−f (0)f (z) .
by applying Lemma 2.2 to g(z) = Lemma 2.3 (See [13]).
|1 − f (0)|2 >0 1 − |f (0)|2
Let
l1 0 · · · 0 0 · · · 0 0 .. .
a=A
l2 · · · .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
B ∈ RI (m, n).
0 0 · · · lm 0 · · · 0 Write
0
√1 2 1−l1
0 ..
Q=A 0
′
A,
. √
1 2 1−lm
R=
B B ′B B B B B @
1
√1
0
1−l12
..
C C C C B. C C A
. √
1 2 1−lm
0
In−m
Liu T S et al.
Sci China Math
7
Here, 1 > l1 > · · · > lm > 0, A is a unitary square matrix of order m and B is a unitary square matrix of order n. Then for any Z ∈ RI (m, n), φa (Z) = Q−1 (Im − Za′ )−1 (a − Z)R is a holomorphic automorphism of RI (m, n), which interchanges 0 and a. Moreover, φa is biholomorphic in a neighborhood of RI (m, n), and φ−1 a = φa ,
dφa (Z) |Z=0 = −Q−1 dZR−1 .
dφa (Z) |Z=a = −QdZR,
From now on, we always denote by F (Z, ξ) the infinitesimal form of Carath´eodory metric or Kobayashi metric on RI (m, n), where Z ∈ RI (m, n) and ξ ∈ Cm×n (see [12] for details). Corollary 2.4. Let ρ(Z) be the Minkowski functional of RI (m, n). Under the notation of Lemma 2.3, for any ξ ∈ Cm×n , F (a, ξ) = ρ(QξR). Proof. Since φa (Z) is a holomorphic automorphism of RI (m, n) and F (Z, ξ) is a biholomorphically invariant metric on RI (m, n), we get F (a, ξ) = F (0, Dφa (a)(ξ)). This, together with Dφa (a)(dZ) = dφa (Z) |Z=a and Lemma 2.3, yields F (a, ξ) = F (0, Dφa (a)(ξ)) = F (0, −QξR) = F (0, QξR). Thus, by [6, Lemma 3.2], we have F (a, ξ) = F (0, QξR) = ρ(QξR). The proof is complete. ˚ be a smooth boundary point of RI (m, n). Then Lemma 2.5. Let Z ˚ 6 ρ(W ) |⟨W, ∇ρ(Z)⟩| for any W ∈ Cm×n . W Proof. Without loss of generality, we may assume W ̸= 0. Then ρ(W ) ∈ ∂RI (m, n). Notice that RI (m, n) is a bounded convex circular domain. Then for each θ ∈ R, we obtain
·
˚ − eiθ W , ∇ρ(Z) ˚ > 0. ℜ Z ρ(W ) It follows from this and Proposition 2.1 that ℜ
eiθ ˚ 6 ℜ⟨Z, ˚ ∇ρ(Z)⟩ ˚ = 1. ⟨W, ∇ρ(Z)⟩ ρ(W )
˚ 6 ρ(W ). The proof is complete. This implies |⟨W, ∇ρ(Z)⟩| Lemma 2.6 (See [14]). Let f : RI (m, n) → RI (m, n) be a holomorphic mapping and let f (0) = 0. Then for any Z ∈ RI (m, n), ρ(f (Z)) 6 ρ(Z).
3
Main result
Now, we establish the Schwarz lemma at the smooth boundary points for holomorphic self-mappings of RI (m, n). Let r1 0 · · · 0 0 · · · 0 ˚= U Z
0 .. .
r2 · · · .. . . . .
0 .. .
0 ··· 0 .. . . .. . . .
0 0 · · · rm 0 · · · 0
V
Liu T S et al.
8
Sci China Math
be a smooth boundary point of RI (m, n), where U is a unitary square matrix of order m, V is a unitary square matrix of order n and 1 = r1 > r2 > · · · > rm > 0. Then Proposition 2.1 implies that the tangent ˚ is space TZ˚(∂RI (m, n)) to ∂RI (m, n) at Z ˚ = 0} = {U αV : α ∈ Cm×n , ℜα11 = 0}, TZ˚(∂RI (m, n)) = {β ∈ Cm×n : ℜ⟨β, ∇ρ(Z)⟩ 1,0 ˚ is and the holomorphic tangent space TZ˚ (∂RI (m, n)) to ∂RI (m, n) at Z 1,0 ˚ = 0} = {U αV : α ∈ Cm×n , α11 = 0}. TZ˚ (∂RI (m, n)) = {β ∈ Cm×n : ⟨β, ∇ρ(Z)⟩
˚ with Suppose that f : RI (m, n) → RI (m, n) is a holomorphic mapping and f is holomorphic at Z 1,0 m×n ˚ ˚ f (Z) = Z. Then it is easy to check that the (mn − 1)-dimensional space TZ˚ (∂RI (m, n)) ⊂ C is an ˚ invariant subspace of Df (Z). Let f : RI (m, n) → RI (m, n) be a holomorphic mapping with f (0) = a, and let
Theorem 3.1.
˚= U Z
r1 0 · · · 0 0 · · · 0 0 .. .
r2 · · · .. . . . .
0 .. .
0 ··· 0 .. . . .. . . .
V
0 0 · · · rm 0 · · · 0 be a smooth boundary point of RI (m, n), where 1 = r1 > r2 > · · · > rm > 0, U is a unitary square ˚ and f (Z) ˚ = Z, ˚ matrix of order m and V is a unitary square matrix of order n. If f is holomorphic at Z ˚ then all the eigenvalues λ, µi (i = 1, . . . , m + n − 2) and νi (i = 1, . . . , (m − 1)(n − 1)) of Df (Z) have the following properties: ˚ to ∂RI (m, n) at Z ˚ is an eigenvector of D∗ f (Z) ˚ and the (1) The unit outward normal vector ∇ρ(Z) corresponding eigenvalue is a real number λ that we just mentioned above, i.e., ˚ ˚ = λ∇ρ(Z). ˚ D∗ f (Z)(∇ρ( Z)) (2) λ > (3)
1−ρ(a) 1+ρ(a)
> 0, and if m = 1 then λ >
1,0 TZ˚ (∂RI (m, n))
˚ ′ |2 |1−Za 1−∥a∥2
>
1−∥a∥ 1+∥a∥
> 0.
= M ⊕ N , where
N = {U αV : α ∈ Cm×n , α11 = 0, (α21 , . . . , αm1 )′ = 0, (α12 , . . . , α1n ) = 0} ˚ and M is an (m + n − 2)-dimensional is an (m − 1)(n − 1)-dimensional invariant subspace of Df (Z), ˚ ˚ which is a linear transformation invariant subspace of Df (Z). Moreover, the eigenvalues µi of Df (Z), on M , satisfy √ |µi | 6 λ, i = 1, . . . , m + n − 2; ˚ which is a linear transformation on N , satisfy and the eigenvalues νi of Df (Z), |νi | 6 1,
i = 1, . . . , (m − 1)(n − 1). √ ˚ 6 λ m+n ˚ 6 λ + λ(m + n − 2) + (m − 1)(n − 1). Then m = 1 shows 2 (4) | det Df (Z)| , |trDf (Z)| √ ˚ 6 λ n+1 ˚ 6 λ + λ(n − 1). 2 , |trDf (Z)| |det Df (Z)| The inequalities in (2)–(4) are sharp. ˚ Proof. (1) For any β ∈ T ˚(∂RI (m, n)), we have Df (Z)(β) ∈ T ˚(∂RI (m, n)). Then Z
Z
˚ ˚ = ℜ⟨Df (Z)(β), ˚ ˚ = 0. ℜ⟨β, D∗ f (Z)(∇ρ( Z))⟩ ∇ρ(Z)⟩ It follows that there exists λ ∈ R such that ˚ ˚ = λ∇ρ(Z). ˚ D∗ f (Z)(∇ρ( Z))
Liu T S et al.
Sci China Math
9
˚ and ∇ρ(Z) ˚ is an eigenvector of D∗ f (Z) ˚ with respect to λ. This means that λ is an eigenvalue of D∗ f (Z), ˚ Since λ ∈ R, we know that λ is also an eigenvalue of Df (Z). The proof of (1) is complete. (2) We divide the proof of (2) into two cases. Case 1.
f (0) = a = 0. For any t ∈ (0, 1), by Lemma 2.6 we get ˚ 6 ρ(tZ) ˚ = t. ρ(f (tZ))
This, together with Lemma 2.5, implies ˚ ∇ρ(Z)⟩ ˚ 6 ρ(f (tZ)) ˚ 6 t. ℜ⟨f (tZ),
(3.1)
˚ ∇ρ(Z)⟩ ˚ = 1. Hence, combine f (tZ) ˚ =Z ˚ − (1 − t)Df (Z)( ˚ Z) ˚ + O(|t − 1|2 )(t → 1− ) By Proposition 2.1, ⟨Z, and (3.1) to obtain ˚ Z), ˚ ∇ρ(Z)⟩ ˚ + O(|t − 1|2 ) 6 t, 1 − (1 − t)ℜ⟨Df (Z)( which gives ˚ D∗ f (Z)(∇ρ( ˚ ˚ + O(|t − 1|) > 1. ℜ⟨Z, Z))⟩
(3.2)
˚ ˚ = λ∇ρ(Z) ˚ and ⟨Z, ˚ ∇ρ(Z)⟩ ˚ = 1, (3.2) yields Since D∗ f (Z)(∇ρ( Z)) λ + O(|t − 1|) > 1. Taking t → 1− , we get λ > 1. Case 2.
f (0) = a ̸= 0. Suppose that
a=A
l1 0 · · · 0 0 · · · 0 0 .. .
l2 · · · .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
B ∈ RI (m, n)
0 0 · · · lm 0 · · · 0 is a polar decomposition of a. Then by Lemma 2.3, g = φa ◦ f : RI (m, n) → RI (m, n) is a holomorphic ˚ with g(0) = 0. Moreover, mapping, and g is holomorphic at Z ˚ = g(Z) ˚ = φa (Z) ˚ = Q−1 (Im − Za ˚ ′ )−1 (a − Z)R ˚ W ˚ is also a smooth boundary point of RI (m, n). Since Dφa (Z)(β) ∈ TW ˚ (∂RI (m, n)) for any β ∈ TZ ˚ (∂RI (m, n)), we get ˚ ˚ )⟩ = 0, ℜ⟨Dφa (Z)(β), ∇ρ(W
˚ ˚ ))⟩ = 0. ℜ⟨β, D∗ φa (Z)(∇ρ( W
It follows that there is µ ∈ R such that ˚ ˚ )) = µ∇ρ(Z). ˚ D∗ φa (Z)(∇ρ( W
(3.3)
Set ˚ ∇ρ(W ˚ )⟩, h1 (ζ) = ⟨φa (ζ Z),
ζ ∈ ∆.
˚ , ∇ρ(W ˚ )⟩ Then h1 : ∆ → ∆ is a holomorphic function, and h1 is holomorphic at ζ = 1 with h1 (1) = ⟨W = 1. This, together with (2.3) and (3.3), implies ˚ µ∇ρ(Z)⟩ ˚ = ⟨Z, ˚ D∗ φa (Z)(∇ρ( ˚ ˚ ))⟩ = ⟨Dφa (Z)( ˚ Z), ˚ ∇ρ(W ˚ )⟩ = h′1 (1) > 0. µ = ⟨Z, W Take ˚ ∇ρ(W ˚ )⟩, h2 (ζ) = ⟨g(ζ Z),
ζ ∈ ∆.
Liu T S et al.
10
Sci China Math
Then h2 : ∆ → ∆ is a holomorphic function, and h2 is holomorphic at ζ = 1 with h2 (0) = 0 and h2 (1) = 1. Thus, by Lemma 2.2, (3.3) and (1) we obtain ˚ Z), ˚ ∇ρ(W ˚ )⟩ 1 6 h′2 (1) = ⟨Dg(Z)( ˚ ˚ Z)), ˚ ∇ρ(W ˚ )⟩ = ⟨Df (Z)( ˚ Z), ˚ D∗ φa (Z)(∇ρ( ˚ ˚ ))⟩ = ⟨Dφa (Z)(Df (Z)( W ˚ Z), ˚ ∇ρ(Z)⟩ ˚ = µ⟨Z, ˚ D∗ f (Z)(∇ρ( ˚ ˚ = λµ. = µ⟨Df (Z)( Z))⟩ This shows λ >
1 µ.
˚ Z), ˚ ∇ρ(W ˚ )⟩. It is easy to check that for each Now, we estimate µ = ⟨Dφa (Z)( ′
X ∈ Cm×k , Y ∈ Ck×l and Z ∈ Cl×n , the maximum eigenvalue of (XY Z)(XY Z) 6 (the maximum ′ ′ ′ eigenvalue of XX ) × (the maximum eigenvalue of (Y Z)(Y Z) ) 6 (the maximum eigenvalue of XX ) × ′ ′ (the maximum eigenvalue of Y Y ) × (the maximum eigenvalue of ZZ ). This gives ρm×n (XY Z) 6 ρm×k (X)ρk×l (Y )ρl×n (Z). Here, ρm×n , ρm×k , ρk×l and ρl×n are the corresponding matrix norms. Notice that ˚ Z) ˚ Dφa (Z)( ˚ ′ (Im − Za ˚ ˚ ′ )−1 Za ˚ ′ )−1 (a − Z)R ˚ − Q−1 (Im − Za ˚ ′ )−1 ZR = Q−1 (Im − Za ˚ ′ QW ˚ ′ )−1 Za ˚ +W ˚ − Q−1 (Im − Za ˚ ′ )−1 aR = Q−1 (Im − Za ˚ ′ − Im )QW ˚ + Q−1 (Im − Za ˚ ′ )−1 QW ˚ +W ˚ − Q−1 (Im − Za ˚ ′ )−1 Qa ˚ ′ )−1 (Za = Q−1 (Im − Za ˚ ′ )−1 Q(W ˚ − a) = Q−1 (Im − Za and ′ ˚ ′ )−1 (a − Z)Ra ˚ ˚ a ′ − Im , Q−1 (Im − Za − Im = W ′ ˚ ˚ ′ )Q] = W ˚ a ′ − Im , ˚ ′ )−1 [(a − Z)Ra − (Im − Za Q−1 (Im − Za
˚ ′ )−1 (Q − aRa′ ) = Im − W ˚ a′ , Q−1 (Im − Za ˚ ′ )−1 Q−1 = Im − W ˚ a′ , Q−1 (Im − Za ˚ ′ )−1 = (Im − W ˚ a′ )Q. Q−1 (Im − Za Then we obtain ˚ Z) ˚ = (Im − W ˚ a′ )Q2 (W ˚ − a). Dφa (Z)( This, together with Lemma 2.5, yields ˚ Z), ˚ ∇ρ(W ˚ )⟩ 6 ρ(Dφa (Z)( ˚ Z)) ˚ µ = ⟨Dφa (Z)( ˚ )ρn×m (a′ )][ρm×m (Q)]2 [ρm×n (W ˚ ) + ρm×n (a)] 6 [ρm×m (Im ) + ρm×n (W = [1 + ρ(a)]2 [1 − (ρ(a))2 ]−1 = It follows that λ>
1 + ρ(a) . 1 − ρ(a)
1 − ρ(a) 1 . > µ 1 + ρ(a)
In particular, if m = 1 then ′
˚ Z), ˚ W ˚ ⟩ = (1 − W ˚ a′ )Q2 (1 − aW ˚ ) = |1 − W ˚ a′ |2 Q2 = µ = ⟨Dφa (Z)(
1 ˚ ′ |2 Q2 |1 − Za
This implies λ>
˚ ′ |2 (1 − ∥a∥)2 1 − ∥a∥ 1 |1 − Za > = . = 2 µ 1 − ∥a∥ 1 − ∥a∥2 1 + ∥a∥
=
1 − ∥a∥2 . ˚ ′ |2 |1 − Za
Liu T S et al.
Sci China Math
11
The proof of (2) is complete. 1,0 ˚ (3) It is clear that TZ˚ (∂RI (m, n)) = {U αV : α ∈ Cm×n , α11 = 0} is an invariant subspace of Df (Z), ′ ′ ˚ i.e., (U Df (Z)(β)V )11 = 0 for any β ∈ T 1,0 (∂RI (m, n)). Now, we claim that N = {U αV : α ∈ ˚ Z
˚ We only need Cm×n , α11 = 0, (α21 , . . . , αm1 )′ = 0, (α12 , . . . , α1n ) = 0} is an invariant subspace of Df (Z). to prove that for each 0 0 ··· 0 0 α22 · · · α2n .. .. . . . . .. . .
β=U
V ∈ N,
0 αm2 · · · αmn ′ ′ ˚ ∈ Cm×n , then ε11 = 0, (ε21 , . . . , εm1 )′ = 0 and (ε12 , . . . , ε1n ) = 0. if we set ε = U Df (Z)(β)V 1,0 ˚ ˚ Since Df (Z)(β) ∈ TZ˚ (∂RI (m, n)), we know ε11 = 0. For t ∈ (0, 1), the polar decompositions of tZ ˚ are and f (tZ)
t 0 ··· 0 0 ··· 0 0 .. .
˚= U tZ
tr2 · · · 0 0 · · · .. . . .. .. . . . . . . .
0 .. .
V
0 0 · · · trm 0 · · · 0 and
0 .. .
˚ = U (t) f (tZ)
···
0
0 ··· 0
r2 (t) · · · .. . . . .
0 .. .
0 ··· 0 .. . . .. . . .
r1 (t)
0
0
0
V (t),
· · · rm (t) 0 · · · 0
respectively, where 1 > r1 (t) > r2 (t) > · · · > rm (t) > 0, U (t) is a unitary square matrix of order m ˚ and and V (t) is a unitary square matrix of order n. Then by Lemma 2.3, corresponding to a = tZ ˚ a = f (tZ), we take 0
Q=U
B B B B B B @
√
0
1
1
0
1−t2
C C C ′ CU , C C A
√ 12 2 1−t r2 ..
. √
0
R=V
1
√1
0
1−t2
B B B ′B B B B B @
√
. √
2 1−t2 rm
1 2 1−t2 rm
0
In−m
and 0
Q(t) =
B B B B U (t) B B B @
1
√
1 1−r12 (t)
0 √
1 1−r22 (t)
..
. √
0 0
R(t) = V
B B B B ′B (t) B B B B @
1 2 (t) 1−rm
C C C ′ C C U (t) , C C A
√
1 1−r12 (t)
0 1 1−r22 (t)
. √
1 2 (t) 1−rm
0
1 C C C C C CV C C C A
√
..
C C C C CV C C C A
1 2 1−t2 r2
..
In−m
(t).
1
Liu T S et al.
12
Sci China Math
˚ = Z, ˚ we have Because limt→1− f (tZ) lim r1 (t) = 1,
lim r2 (t) = r2 ,
t→1−
...,
t→1−
lim rm (t) = rm .
t→1−
In addition, we also get U (t) = U + O(|t − 1|),
V (t) = V + O(|t − 1|) and
˚ ˚ Df (tZ)(β) = Df (Z)(β) + O(|t − 1|)
˚ =Z ˚ − (1 − t)Df (Z)( ˚ Z) ˚ + O(|t − 1|2 ) that as t → 1− . Moreover, it follows from f (tZ) ˚ r1 (t) = ρ(f (tZ)) = 1 − (1 − t)2ℜ
m X n X i=1 j=1
∂ρ ˚ ˚ Z) ˚ + O(|t − 1|2 ) (Z)Dfij (Z)( ∂zij
˚ Z), ˚ ∇ρ(Z)⟩ ˚ + O(|t − 1|2 ) = 1 − (1 − t)ℜ⟨Df (Z)( ˚ D∗ f (Z)(∇ρ( ˚ ˚ + O(|t − 1|2 ) = 1 − (1 − t)ℜ⟨Z, Z))⟩ = 1 − λ(1 − t) + O(|t − 1|2 ) as t → 1− . This implies È
1 − r12 (t) =
È
È
1 − [1 − λ(1 − t) + O(|t − 1|2 )]2 =
2λ(1 − t) + O(|t − 1|2 )
(3.4)
−
as t → 1 . By Corollary 2.4, we have ˚ β) F (tZ, 2
=
0
6 6 6 6 ρ6 U 6 6 6 4
B B B B B B @
√
0
1
1
0
1−t2
B C B C B C ′ ′B C U βV B C B C B B A @
√ 12 2 1−t r2 ..
. √
0
√1
1 2 1−t2 rm
0
1−t2
√
1 2 1−t2 r2
..
√
=
1 2 1−t2 rm
0
3
C C C C CV C C C A
7 7 7 7 7 7 7 7 5
In−m 0
2 6 6 6 6 ρ6 6 6 6 4
.
1
1
0 √ 12 2 1−t r2 ..
···
0
0
0 α22 · · · α2n .. .. . . . . .. . .
. √
0
0
0 αm2 · · · αmn
1 2 1−t2 rm
B B B B B B B B @
13
1
0 √
1 2 1−t2 r2
..
. √
1 2 1−t2 rm
0
In−m
√ ˚ β) = 0. Similarly, we obtain which gives limt→1− 1 − t2 F (tZ, ˚ Df (tZ)(β)] ˚ F [f (tZ), 0
2
=
6 B B 6 B 6 6 B ρ 6U (t) B 6 B B 6 @ 4
1
√
1 1−r12 (t)
0 √
1 1−r22 (t)
..
√
0 0
×
B B B B B B B B B @
. 1 2 (t) 1−rm
√
1 1−r12 (t)
C C C ′ C ˚ C U (t) Df (tZ)(β)V C C A
0 √
C C C C C CV C C C A
1 1−r22 (t)
..
. √
1 2 (t) 1−rm
0
1
In−m
3 7 7 7 7 7 (t)7 . 7 7 7 5
′
(t)
C7 C7 C7 C7 C 7, C7 C7 C7 A5
Liu T S et al.
Notice that
Sci China Math
13
′ ′ ′ ′ ˚ ˚ U (t) Df (tZ)(β)V (t) = U Df (Z)(β)V + O(|t − 1|) = ε + O(|t − 1|)
as t → 1− . This, together with (3.4), implies È
˚ Df (tZ)(β)] ˚ 1 − r12 (t)F [f (tZ),
lim−
t→1
1
20
1 È
= lim− t→1
0
6B 6B 6B 2 B r1 (t)ρ 6 6B 6B 4@
1−
√
1 1−r22 (t)
..
. √
1 2 (t) 1−rm
0 0
×
B B ε21 +O(|t−1|) B √ 1−r12 (t) B B .. B . B @ εm1 +O(|t−1|)
√
6B 6B 6B 6B ρ 6B 6B 6B 4@
.. .
1−r22
√ε21 2 1−r2
..
.. .
.
· · · √ε1m 2 ε1(m+1) · · · ε1n 1−rm
0
···
0
0
··· 0
.. .
.. .
..
.
.. .
.. .
..
√εm1 2
0
···
0
0
··· 0
1−rm
.. .
.
13
0
B CB CB 1|) C B CB CB CB CB AB @
εm2 + O(|t − 1|) · · · εmn + O(|t − 1|)
√ε12
0
···
1
ε1n +O(|t−1|) √ 2 1−r1 (t)
ε22 + O(|t − 1|) · · · ε2n + O(|t −
1−r12 (t)
20
=
ε12 +O(|t−1|) √ 2 1−r1 (t)
O(|t−1|) 1−r12 (t)
C C C C C C A
1
0 √
1 1−r22 (t)
..
. √
1 2 (t) 1−rm
0
C7 C7 C7 C7 C7 C7 C7 C7 C7 A5
In−m
13 C7 C7 C7 C7 C7 . C7 C7 A5
˚ Df (tZ)(β)] ˚ ˚ β). It follows By the contraction property of the Kobayashi metric, we have F [f (tZ), 6 F (tZ, from this and (3.4) that 20 6B 6B 6B 6B ρ 6B 6B 6B 4@
0
√ε12
√ε21
0
···
0
0
··· 0
.. .
.. .
..
.
.. .
.. .
..
√εm1 2
0
···
0
0
··· 0
1−r22
1−r22
1−rm È
= lim− t→1
· · · √ε1m 2 ε1(m+1) · · · ε1n 1−rm
.
.. .
13 C7 C7 C7 C7 C7 C7 C7 A5
˚ Df (tZ)(β)] ˚ 1 − r12 (t)F [f (tZ),
È
1 − r12 (t) p ˚ β) 6 lim− √ 1 − t2 F (tZ, t→1 1 − t2 p √ ˚ β) = 0. = λ lim 1 − t2 F (tZ, t→1−
That means (ε21 , . . . , εm1 )′ = 0,
(ε12 , . . . , ε1n ) = 0.
˚ Hence, there exists an This shows that N is an (m − 1)(n − 1)-dimensional invariant subspace of Df (Z). ˚ such that (m + n − 2)-dimensional invariant subspace M of Df (Z) 1,0 TZ˚ (∂RI (m, n)) = M ⊕ N.
Since M ∩ N = {0}, we have (α21 , . . . , αm1 )′ = ̸ 0 or (α12 , . . . , α1n ) ̸= 0 for any β = U αV ∈ M \ {0}. ˚ on M , suppose that β (i) = U α(i) V ∈ M \ {0} is a nonzero eigenvector For each eigenvalue µi of Df (Z) ′ (i) (i) (i) (i) (i) ˚ (i) )V ′ = with respect to µi . Here, α11 = 0, (α21 , . . . , αm1 )′ ̸= 0 or (α12 , . . . , α1n ) ̸= 0, U Df (Z)(β
Liu T S et al.
14
Sci China Math
µi α(i) (i = 1, . . . , m + n − 2). By Corollary 2.4, we get ˚ β (i) ) F (tZ, 2
=
0
6 6 6 6 ρ6 U 6 6 6 4
B B B B B B @
√
0
1−t2
√
..
. √
0
1 ..
.
0
×
0
0
. √
1 2 1−t2 rm
0
1 2 1−t2 r2
..
. √
1 2 1−t2 rm
···
1−t2 (i) α22
3
C C C C CV C C C A
7 7 7 7 7 7 7 7 5
In−m
(i) α √ 1n
1
1−t2 C (i) C α2n C C .. C . C A (i) αmn
··· .. .
.. .
1
(i)
αm2 · · ·
C7 C7 C7 C7 C7 C7 . C7 C7 C7 A5
√ 12 2 1−t r2 ..
√
0 (i) α √ 12
CB C B α(i) C B √ 21 2 C B 1−t CB .. CB . A@ (i) 1 α √ 2 2 √ m1 1−t rm 1−t2 13
1
0
1−t2
2 1−t2 rm
0 √ 12 2 1−t r2
0 B B B B B B B B B @
1
10
6B 6B 6B B ρ6 6B 6B 4@
√1
B B C B C C ′ ′B C U β (i) V B B C B C B A @
1 2 1−t2 r2
20
=
0
1
1
In−m
Thus, we obtain 20
lim−
p
t→1
˚ β (i) ) = 1 − t2 F (tZ,
(i)
(i)
√α12
0
6B 6B 6B 6B B ρ6 6B 6B 6B 4@
√
α (i) (i) · · · √ 1m 2 α1(m+1) · · · α1n
1−r22
(i)
α21
1−r22
.. .
1−rm
0
···
0
0
··· 0
.. .
..
.
.. .
.. .
..
0
···
0
0
··· 0
(i)
√αm1 2
1−rm
.
.. .
13 C7 C7 C7 C7 C7 C7 C7 C7 A5
On the other hand, ˚ Df (tZ)(β ˚ (i) )] F [f (tZ), 2
=
0
6 B 6 B 6 B 6 B ρ 6U (t) B 6 B 6 B 4 @
√
1
1 1−r12 (t)
0 √
1 1−r22 (t)
..
√
1 2 (t) 1−rm
0 0
×
B B B B B B B B B @
√
1 1−r12 (t)
0 √
..
1 C C C C C CV C C C A
1 1−r22 (t)
. √
1 2 (t) 1−rm
0 ′
.
C C C ′ C ˚ (i) )V C U (t) Df (tZ)(β C C A
(t)
3 7 7 7 7 7 (t)7 . 7 7 7 5
In−m ′
˚ (i) )V (t) = µi α(i) + O(|t − 1|)(t → 1− ) and α = 0. Then Notice that U (t) Df (tZ)(β 11 È
lim−
t→1
˚ Df (tZ)(β ˚ (i) )] 1 − r12 (t)F [f (tZ),
(i)
′
̸= 0.
Liu T S et al.
Sci China Math 1
20
1 È
= lim− t→1
6B 6B 6B B r12 (t)ρ 6 6B 6B 4@
1−
0 √
1 1−r22 (t)
..
. √
1 2 (t) 1−rm
0 0
(i)
µi α12 +O(|t−1|)
√
O(|t−1|) 2 (t) 1−r1
×
B B µ α(i) +O(|t−1|) 21 B i √ 2 (t) B 1−r1 B . B .. B @ (i)
(i) µi α22
√
20
=
6B 6B 6B 6B B |µi |ρ 6 6B 6B 6B 4@
(i) µi αm2
2 (t) 1−r1
√
0 √
1−r22
(i)
α21
.. .
√
2 1−rm
+ O(|t − .. .
. (i) µi αmn
13
1
0
CB CB B 1|) C CB CB CB CB B A@
+ O(|t − 1|)
√
1 2 (t) 1−r2
..
. √
1 2 (t) 1−rm
0
C7 C7 C7 C7 C7 C7 C7 C7 A5
In−m
13
··· √
(i)
2 1−rm
(i)
α1(m+1) · · · α1n
···
0
0
··· 0
..
.
.. .
.. .
..
.. .
0
···
0
0
··· 0
(i)
αm1
2 (t) 1−r1
.. .
0
1−r22
(i) µi α2n
+ O(|t − 1|) · · · (i) α1m
(i) α12
10
(i)
√
+ O(|t − 1|) · · · ..
C C C C C C A
µi α1n +O(|t−1|)
···
2 (t) 1−r1
.. .
µi αm1 +O(|t−1|)
15
.
C7 C7 C7 C7 C7 C7 C7 C7 A5
= |µi | lim−
p
˚ β (i) ). 1 − t2 F (tZ,
t→1
It follows from this and (3.4) that √
˚ Df (tZ)(β ˚ (i) )] F [f (tZ), 1 > lim− = lim− ˚ β (i) ) t→1 t→1 F (tZ, This implies |µi | 6
È
√ λ,
È
˚ Df (tZ)(β ˚ (i) )] 1 − r12 (t) F [f (tZ), |µi | √ = √ . 2 (i) ˚ 2 1−t λ F (tZ, β ) 1 − r1 (t) 1 − t2
i = 1, . . . , m + n − 2.
˚ on N , suppose that β (i) = U α(i) V ∈ N \ {0} is a nonzero eigenvector For any eigenvalue νi of Df (Z) (i) (i) (i) (i) (i) with respect to νi . Here, α11 = 0, (α21 , . . . , αm1 )′ = 0, (α12 , . . . , α1n ) = 0,
(i)
(i)
(i) αm2
(i) αmn
α22 · · · α2n .. . . . . .. . ···
̸ 0 =
′ ˚ (i) )V ′ = νi α(i) for i = 1, . . . , (m − 1)(n − 1). Then by Corollary 2.4, we have and U Df (Z)(β
˚ β (i) ) F (tZ, 2
=
0
6 6 6 6 U ρ6 6 6 6 4
B B B B B B @
√
0
1
1
0
1−t2
√
1 2 1−t2 r2
..
. √
0
1
√1
B C B B C C ′ ′B C U β (i) V B B C C B B A @
√
=
1 2 1−t2 r2
..
. √
2 1−t2 rm
1 2 1−t2 rm
0
2 6 6 6 6 ρ6 6 6 6 4
0
1−t2
1
3
C C C C CV C C C A
7 7 7 7 7 7 7 7 5
In−m 0
1
0 √
1 2 1−t2 r2
.. 0
. √
1 2 1−t2 rm
0
0
0 .. .
(i) α22
.. . (i)
···
0
··· .. .
(i) α2n
.. . (i)
0 αm2 · · · αmn
B B B B B B B B @
13
1
0 √
1 2 1−t2 r2
..
. √
0
1 2 1−t2 rm
In−m
C7 C7 C7 C7 C 7. C7 C7 C7 A5
Liu T S et al.
16
Sci China Math
Hence, we get 20
0
6B 6B 6B 0 6B ρ 6B . 6B . 6B . 4@
˚ β (i) ) = lim− F (tZ,
t→1
···
0 (i)
.. .
..
(i)
√
αm2
2 1−rm
1−r22
√
(i)
α2(m+1)
√
α2m
1−r22
.
1−r22
2 1−rm
.. .
(i)
α · · · √ 2n
..
(i)
αm(m+1)
√
13
0 1−r22
.. .
α(i) mm 2 1−rm
···
···
0
(i)
··· √
α22 1−r22
0 √
0
2 1−rm
.. .
.
(i)
α · · · √ mn 2
C7 C7 C7 C7 C7 C7 C7 A5
̸= 0.
1−rm
On the other hand, ˚ Df (tZ)(β ˚ (i) )] F [f (tZ), 2
=
0
6 B 6 B 6 B B 6 ρ 6U (t) B 6 B B 6 @ 4
√
1
1 1−r12 (t)
0 1 1−r22 (t)
..
. √
1 2 (t) 1−rm
0 0
×
B B B B B B B B B @
C C C ′ C ˚ (i) )V C U (t) Df (tZ)(β C C A
√
√
1 1−r12 (t)
1
0 √
..
. √
1 2 (t) 1−rm
0
′
3
C C C C C CV C C C A
1 1−r22 (t)
(t)
7 7 7 7 7 (t)7 . 7 7 7 5
In−m
′
′
˚ (i) )V (t) = νi α(i) + O(|t − 1|) and r1 (t) = 1 − λ(1 − t) + O(|t − 1|2 ) as t → 1− . Notice that U (t) Df (tZ)(β Then ˚ Df (tZ)(β ˚ (i) )] lim F [f (tZ),
t→1−
20
1
1 = lim
t→1−
6B 6B 6B B ρ6 6B 6B 4@
0 √
1 1−r22 (t)
..
. √
1 2 (t) 1−rm
0 0
×
√
O(|t−1|) 2 (t) 1−r1
(i) νi α22
√
···
O(|t−1|)
B O(|t−1|) B√ B 1−r2 (t) 1 B B .. B . @
C C C C C C A
2 (t) 1−r1
+ O(|t − 1|) · · · .. .
..
(i) νi α2n
0
1
O(|t−1|)
B CB
2 (t) 1−r1
1−r1 (t)
=
b
6B 6B 6B 0 6B ρ 6B . 6B . 6B . 4@
0 (i)
νi α22 1−r22
.. . (i) νi αm2 2 1−rm 1−r22
0 √
√
···
0
··· √
1−r22
..
.
···
0
(i)
√
νi α2m 2 1−rm
√
1−r22
.. .
νi α(i) mm 2 1−rm
···
(i)
νi α2(m+1)
.. .
(i) νi αm(m+1)
√
2 1−rm
1 2 (t) 1−r2
..
CB CB CB AB @
.. .
.
0 √
B + O(|t − 1|) C CB
(i) (i) O(|t−1|) √ νi αm2 + O(|t − 1|) · · · νi αmn + O(|t − 1|) 2
20
13
1
··· ..
.
. √
0
0
In−m 13
(i) C7 νi α2n C7 √ C7 1−r22 C7 7 .. C 7 . C C7 A5 (i) νi αmn
··· √
2 1−rm
1 2 (t) 1−rm
C7 C7 C7 C7 C7 C7 C7 C7 A5
˚ β (i) ), > |νi | lim− F (tZ, t→1
Liu T S et al. ′
17
′
(i) ˚ (U (t) Df (tZ)(β )V (t) )11 . 1−r12 (t)
where b = limt→1−
Sci China Math
1 > lim− t→1
It follows that
˚ Df (tZ)(β ˚ (i) )] F [f (tZ), > |νi |. ˚ β (i) ) F (tZ,
This shows |νi | 6 1,
i = 1, . . . , (m − 1)(n − 1).
The proof of (3) is complete. 1,0 (4) Since TZ˚ (∂RI (m, n)) = {U αV : α ∈ Cm×n , α11 = 0} = M ⊕ N is an (mn − 1)-dimensional ˚ we know that there is a one-dimensional invariant subspace L of Df (Z) ˚ invariant subspace of Df (Z), such that Cm×n = L ⊕ M ⊕ N. 1,0 It follows from L ∩ TZ˚ (∂RI (m, n)) = {0} that for any β = U αV ∈ L \ {0} we have α11 = ̸ 0. Now, we e is an eigenvalue of Df (Z) ˚ on L. Assume that λ ˚ on L, and claim that λ is just the eigenvalue of Df (Z) e ˚ β = U αV ∈ L \ {0} is a nonzero eigenvector of Df (Z) with respect to λ. Then by Proposition 2.1, we
obtain e e ˚ ˚ = λ⟨β, ˚ = λα ⟨Df (Z)(β), ∇ρ(Z)⟩ ∇ρ(Z)⟩ 11 .
On the other hand, ˚ ˚ = ⟨β, D∗ f (Z)(∇ρ( ˚ ˚ = λ⟨β, ∇ρ(Z)⟩ ˚ = λα11 . ⟨Df (Z)(β), ∇ρ(Z)⟩ Z))⟩ e = λ. Therefore, λ, µ (i = 1, . . . , m + n − 2) and ν (i = 1, . . . , This, together with α11 = ̸ 0, yields λ i i ˚ on Cm×n . This implies (m − 1)(n − 1)) are all the eigenvalues of Df (Z)
˚ 6 λ m+n 2 | det Df (Z)| ,
˚ 6λ+ |trDf (Z)|
√ λ(m + n − 2) + (m − 1)(n − 1).
The proof of (4) is complete. ˚ perhaps because the Remark 3.2. From the view of geometry, N is an invariant subspace of Df (Z) ˚ is positive semi-definite and not positive definite on N . We get the same conclusions Levi form√of ρ at Z ˚ is of |µi | 6 λ (i = 1, . . . , m + n − 2) with [15, Theorem 3.1] perhaps because the Levi form of ρ at Z positive definite on M . Remark 3.3. From the proof of Theorem 3.1, it is clear that we need only to assume that the mapping f ˚ is C 1 up to the boundary of RI (m, n) near Z. Remark 3.4. When m = 1, n = 1, f (0) = 0 and RI (1, 1) = ∆, Theorem 3.1 is just Lemma 2.2. When m = 1 and RI (1, n) = B n , Theorem 3.1 is just [16, Theorem 3.1]. Finally, we give the following example to show that the inequalities in (2)–(4) of Theorem 3.1 are sharp. Example 3.5.
Let
ε 0 ··· 0 0 ··· 0 0 .. .
a=
0 ··· .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
∈ RI (m, n)
0 0 ··· 0 0 ··· 0 and 0 < ε < 1. Write eij ∈ Cm×n as a matrix, which has 1 at i-th row and j-th column, and 0s elsewhere. By Lemma 2.3, take Q=
!
√ 1 1−ε2
0
0
Im−1
,
R=
!
√ 1 1−ε2
0
0
In−1
.
Liu T S et al.
18
Suppose that
1 0 ··· 0 0 ··· 0 0 .. .
˚= Z
Sci China Math
r2 · · · .. . . . .
0 .. .
0 ··· .. . . . .
0 .. .
0 0 · · · rm 0 · · · 0 is a smooth boundary point of RI (m, n) and f (Z) = −φ−a (Z) = Q−1 (Im + Za′ )−1 (a + Z)R, where 1 > r2 > · · · > rm > 0. Then f : RI (m, n) → RI (m, n) is a holomorphic mapping with f (0) = a, ˚ Moreover, f has the following properties. and f is holomorphic at Z. ˚ ˚ (1) f (Z) = Z. (2) For any β ∈ Cm×n , q
˚ Df (Z)(β) =
1−ρ(a) 1+ρ(a)
0
0
q
β
Im−1
1−ρ(a) 1+ρ(a)
0
0
.
In−1
1−ρ(a) 1−ρ(a) ˚ 1+ρ(a) e11 . This shows that one of eigenvalues of Df (Z) is 1+ρ(a) . q q ˚ i1 ) = 1−ρ(a) ei1 (i = 2, . . . , m). ˚ 1j ) = 1−ρ(a) e1j (j = 2, . . . , n) and Df (Z)(e (4) Df (Z)(e 1+ρ(a) 1+ρ(a) q 1−ρ(a) ˚ that the m + n − 2 eigenvalues of Df (Z) are all 1+ρ(a) .
˚ 11 ) = (3) Df (Z)(e
This shows
˚ ij ) = eij (i = 2, . . . , m; j = 2, . . . , n). This shows that the (m − 1)(n − 1) eigenvalues (5) Df (Z)(e ˚ of Df (Z) are all 1. Proof. By Lemma 2.3, it is clear that f : RI (m, n) → RI (m, n) is a holomorphic mapping with ˚ f (0) = a, and f is holomorphic at Z. (1) It is obvious that ρ(a) = ε. Since !
˚ ′= Za
ε 0 00
!
˚ ′= Im + Za
,
1+ε
0
0
Im−1
˚= and a + Z
1 + ε 0 ··· 0 0 ··· 0 0 .. .
r2 · · · .. . . . .
0
0 · · · rm 0 · · · 0
0 .. .
0 ··· 0 .. . . .. . . .
we have ˚ = Q−1 (Im + Za ˚ ′ )−1 (a + Z)R ˚ f (Z)
=
√ 1 − ε2
0
0
Im−1
=
!
1 1+ε
0 .. .
r2 · · · .. . . . .
0 .. .
!
0
0 Im−1
1 0 ··· 0 0 ··· 0 0 ··· .. . . . .
1 + ε 0 ··· 0 0 ··· 0 0 .. .
r2 · · · .. . . . .
0
0 · · · rm 0 · · · 0
0 .. .
0 ··· 0 .. . . .. . . .
!
√ 1 1−ε2
0
0
In−1
0 .. .
0 0 · · · rm 0 · · · 0 ˚ = Z. (2) By a straightforward calculation, we get ˚ ′ )−1 βa′ (Im + Za ˚ ′ )−1 (a + Z)R ˚ ˚ ˚ ′ )−1 βR − Q−1 (Im + Za Df (Z)(β) = Q−1 (Im + Za
,
Liu T S et al.
Sci China Math
19
˚ ′ )−1 βR − Q−1 (Im + Za ˚ ′ )−1 βa′ Qf (Z) ˚ = Q−1 (Im + Za È
=
0 È
=
=
˚ β(R − a′ QZ) !
1−ε 1+ε
0
0
0
β
1−ρ(a) 1+ρ(a)
0
0
In−1
0
1−ε 1+ε
0
q
β
Im−1
!
√ 1 1−ε2 È
!
Im−1
0
"
β
Im−1
1−ε 1+ε
q
=
0 Im−1
0 È
!
1−ε 1+ε
−
!#
√ ε 1−ε2
0
0
0
!
0 In−1 1−ρ(a) 1+ρ(a)
0
0
.
(3.5)
In−1
(3)–(5) Replacing β with eij (i = 1, . . . , m; j = 1, . . . , n) in (3.5), we can obtain ˚ 11 ) = Df (Z)(e
1 − ρ(a) e11 , 1 + ρ(a)
Ê
˚ 1j ) = Df (Z)(e Ê
˚ i1 ) = Df (Z)(e
1 − ρ(a) e1j , 1 + ρ(a)
j = 2, . . . , n,
1 − ρ(a) ei1 , 1 + ρ(a)
i = 2, . . . , m,
˚ ij ) = eij , Df (Z)(e
i = 2, . . . , m,
j = 2, . . . , n
at once. The proof is complete.
4
Conclusions
In this paper, we considered the Schwarz lemma at the smooth boundary points of RI (m, n). There are some interesting problems that deserve further investigation such as the Schwarz lemma at the nonsmooth boundary points and the boundary Schwarz lemma on the other classical domains. In addition, we can apply it to obtain some new results in the geometric function theory of several complex variables. Acknowledgements This work was supported by National Natural Science Foundation of China (Grant Nos. 11571105 and 11471111) and Natural Science Foundation of Zhejiang Province (Grant No. LY14A010017). The authors thank the referees for their comments and suggestions.
References 1 Abate M. Iteration Theory of Holomorphic Maps on Taut Manifolds. Rende: Mediterranean Press, 1989 2 Burns D M, Krantz S G. Rigidity of holomorphic mappings and a new Schwarz lemma at the boundary. J Amer Math Soc, 1994, 7: 661–676 3 Cartan H. Les fonction de deux variables complexs et le probl` eme de la repr` esentation analytique. J Math Pures Appl, 1931, 96: 1–114 4 Elin M, Jacobzon F, Levenshtein M, et al. The Schwarz lemma: Rigidity and dynamics. In: Harmonic and Complex Analysis and Its Applications. New York: Springer, 2014, 135–230 5 Garnett J B. Bounded Analytic Functions. New York: Academic Press, 1981 6 Gong S, Liu T S. Distortion therorems for biholomorphic convex mappings on bounded convex circular domains. Chin Ann Math Ser B, 1999, 20: 297–304 7 Hua L K. Harmonic Analysis of Functions of Several Complex Variables in the Classical Domains. Providence: Amer Math Soc, 1963 8 Huang X J. Some applications of Bell’s theorem to weakly pseudoconvex domains. Pacific J Math, 1993, 158: 305–315 9 Huang X J. A preservation principle of extremal mappings near a strongly pseudoconvex point and its applications. Illinois J Math, 1994, 38: 283–302
20
Liu T S et al.
Sci China Math
10 Huang X J. A boundary rigidity problem for holomorphic mappings on some weakly pseudoconvex domains. Canad J Math, 1995, 47: 405–420 11 Kim K T, Lee H. Schwarz’s Lemma from a Differential Geometric Viewpoint. Bangalore: IISc Press, 2011 12 Krantz S G. Function Theory of Several Complex Variables, 2rd ed. Providence: Amer Math Soc, 2001 13 Liu T S. The growth theorems, covering theorems and distortion theorems for biholomorphic mappings on classical domains. Doctoral Dissertion. Hefei: University of Science and Technology of China, 1989 14 Liu T S, Ren G B. Decomposition theorem of normalized biholomorphic convex mapping. J Reine Angew Math, 1998, 496: 1–13 15 Liu T S, Tang X M. Schwarz lemma at the boundary of strongly pseudoconvex domain in Cn . Math Ann, 2016, 366: 655–666 16 Liu T S, Wang J F, Tang X M. Schwarz lemma at the boundary of the unit ball in Cn and its applications. J Geom Anal, 2015, 25: 1890–1914 17 Look K H. Schwarz lemma and analytic invariants. Scientia Sinica, 1958, 7: 453–504 18 Tang X M, Liu T S, Lu J. Schwarz lemma at the boundary of the unit polydisk in Cn . Sci China Math, 2015, 58: 1639–1652
