Cycle Classes of the E-O Stratification of the Moduli of Abelian Varieties

Torsten Ekedahl^{1} and Gerard van der Geer^{2}

1 Matematiska institutionen, Stockholms universitet, SE-106 91 Stockholm,

Sweden teke@math.su.se 2 Korteweg-de Vries Instituut, Universiteit van Amsterdam, Plantage

Muidergracht 24, 1018 TV Amsterdam, The Netherlands geer@science.uva.nl

To Yuri Ivanovich Manin on the occasion of his 70th birthday

Summary. We introduce a stratification on the space of symplectic flags on the de Rham bundle of the universal principally polarised abelian variety in positive characteristic. We study its geometric properties like irreducibility of the strata and we calculate the cycle classes. When the characteristic p is treated as a formal variable these classes can be seen as a deformation of the classes of the Schubert varieties for the corresponding classical flag variety (the classical case is recovered by putting p equal to 0). We relate our stratification with the E-O stratification on the moduli space of principally polarised abelian varieties of a fixed dimension and derive properties of the latter. Our results are strongly linked with the combinatorics of the Weyl group of the symplectic group.

1 Introduction

The moduli space A_{g} of principally polarised abelian varieties of dimension g is defined over the integers. For the characteristic zero fibre A_{g} ⊗ ℂ we have an explicit description as an orbifold Sp_{2g}(ℤ)\H_{g} with H_{g} the Siegel upper half space of degree g. It is a recent insight, though, that perhaps the positive characteristic fibres A_{g}⊗𝔽_{p} are more accessible than the characteristic zero one. A good illustration of this is provided by the E-O stratification of A_{g} ⊗ 𝔽_{p}, a stratification consisting of 2^{g} quasi-affine strata. It was originally defined by Ekedahl and Oort (see [Oo01]) by analyzing the structure of the kernel of multiplication by p of an abelian variety. It turns out that this group scheme can assume 2^{g} forms only, and this led to the strata. For g = 1 the two strata are the loci of ordinary and of supersingular elliptic curves. Some strata possess intriguing properties. For example, the stratum of abelian varieties of p-rank 0 is a complete subvariety of A_{g} ⊗ 𝔽_{p} of codimension g, the smallest codimension possible. No analogue in characteristic 0 of either this stratum or of the stratification is known. In fact, Keel and Sadun ([KS03]) proved that complete subvarieties of A_{g} ⊗ ℂ of codimension g do not exist for g ≥ 3.

While trying to find cycle classes for the E-O strata we realised that the strata could be described as degeneration loci for maps between vector bundles and as such loci are indexed by Young diagrams our attention was turned towards the combinatorics of the Weyl group. When considered in this light it is clear that much of the combinatorics of [Oo01] is closely related to the Weyl group W_{g} of Sp_{2g}, which is the semi-simple group relevant for the analytic description of A_{g} ⊗ ℂ. The main idea of this paper is to try to make this connection more explicit. More precisely, the combinatorics of the E-O strata is most closely related to the combinatorics associated to W_{g} and the Weyl subgroup corresponding to the maximal parabolic subgroup P of elements of Sp_{2g} stabilizing a maximal isotropic subspace in the 2g-dimensional symplectic vector space. Indeed, this sub-Weyl group is S_{g}, the group of permutations on g letters (embedded as a sub-Weyl group in W_{g}), and the E-O strata are in bijection with the cosets in W_{g}/S_{g}; we shall use the notation V_{ν} for the (open) stratum of A_{g} ⊗ 𝔽_{p} corresponding to ν ∈ W_{g}/S_{g} (and \overline{V}_{ν} for its closure). The coset space W_{g}/S_{g} is also in bijection with the set of Bruhat cells in the space of maximal totally isotropic flags Sp_{2g} /P and we believe this to be no accident. (The formal relation between A_{g} and Sp_{2g} /P is that Sp_{2g} /P is the compact dual of H_{g}.)

In order to push the analogy further we introduce a “flag space” F_{g} → A_{g} whose fibres are isomorphic to the fibres of the quotient morphism Sp_{2g} /B → Sp_{2g} /P, where B is a Borel subgroup of P. In positive characteristic we define (and this definition makes sense only in positive characteristic) a stratification of F_{g} whose open strata U_{w} and their corresponding closures \overline{U}_{w} are parametrised by the elements of W_{g}. This stratification is very similar to the stratification by Bruhat cells of Sp_{2g} /B and their closures, the Schubert strata, which are also parametrised by the elements of W_{g}. Our first main result is that this is more than a similarity when one works locally; we show (cf., Thm. 8.2) that for each point of F_{g} there is a stratum preserving local isomorphism (in the étale topology) taking the point to some point of Sp_{2g} /B. As a lot is known about the local structure of the Schubert varieties we immediately get a lot of information about the local structure of our strata. The first consequence is that \overline{U}_{w} is equidimensional of dimension equal to the length of w. A very important consequence is that the \overline{U}_{w} are all normal; this situation differs markedly from the case of the closed E-O strata which in general are not normal. Another consequence is that the inclusion relation between the strata is given exactly by the Bruhat-Chevalley order on W_{g}. (A much more sophisticated consequence is that the local structure of the ℓ-adic intersection complex for a closed stratum is the same as for the Schubert varieties and in particular that the dimensions of its fibres over the open strata of the closed stratum are given by the Kazhdan-Lusztig polynomials. We shall not however pursue that in this article.)

We give several applications of our results to the structure of the strata \overline{U}_{w}. The first, and most important, is that by construction the strata U_{w} are defined as the loci where two symplectic flags on the same vector bundle are in relative position given by w. After having shown that they have the expected codimension and are reduced we can use formulas of Fulton, as well as those of Pragacz and Ratajski, as crystallised in the formulas of Kresch and Tamvakis, to get formulas for the cycle classes of the strata. A result of Fulton gives such formulas (cf., Thm. 12.1) for all strata but in terms of a recursion formula that we have not been able to turn into a closed formula; however, these formulas should have independent interest and we use them to get formulas for the E-O strata as follows. If w ∈ W_{g} is minimal for the Bruhat-Chevalley order in its coset wS_{g}, then U_{w} maps by a finite étale map to the open E-O stratum V_{ν} corresponding to the coset ν := wS_{g}. We can compute the degree of this map in terms of the combinatorics of the element w and we then can push down our formulas for \overline{U}_{w} to obtain formulas for the cycle classes of the E-O strata. Also the formulas of Kresch and Tamvakis can be used to give the classes of E-O strata. One interesting general consequence (cf., Thm. 13.1) is that each class is a polynomial in the Chern classes λ_{i} of the Hodge bundle, the cotangent bundle of the zero-section of the universal abelian variety, and the coefficients are polynomials in p. This is a phenomenon already visible in the special cases of our formula that were known previously; the oldest such example being Deuring’s mass formula for the number of supersingular elliptic curves (weighted by one over the cardinalities of their groups of automorphisms) that says that this mass is (p−1)/12. This appears in our context as the combination of the formula (p − 1)λ_{1} for the class of the supersingular locus and the formula deg λ_{1} = 1/12. We interpret these results as giving rise to elements in the p-tautological ring; this is the ring obtained from the usual tautological ring, the ring generated by the Chern classes of the Hodge bundle, by extending the scalars to ℤ{p}, the localisation of the polynomial ring ℤ[p] at the polynomials with constant coefficient 1. Hence we get elements parametrised by W_{g}/S_{g} in the p-tautological ring and we show that they form a ℤ{p}-basis for the p-tautological ring. Putting p equal to 0 maps these elements to elements of the ordinary tautological ring which can be identified with the Chow ring of Sp_{2g} /P and these elements are the usual classes of the Schubert varieties. It seems that these results call for a p-Schubert calculus in the sense of a better understanding of these elements of the p-tautological ring and their behaviour under multiplication.

However, there seems to be a more intriguing problem. We have for each w ∈ W_{g} a stratum in our flag space and they push down to elements of the p-tautological ring under the projection map to Ã_{g} (a suitable toroidal compactification of A_{g}). When setting p to 0 these elements specialise to the classes of the images of the Schubert varieties of Sp_{2g} /B_{g} in Sp_{2g} /P and for them the situation is very simple; either w is minimal in its S_{g} coset and then the Schubert variety maps birationally to the corresponding Schubert variety of Sp_{2g} /P or it is not and then it maps to 0. When it comes to the elements of the p-tautological ring this only allows us to conclude – in the non-minimal case – that the coefficients are divisible by p and indeed in general they are not zero. We show that unless they map to 0 they will always map to a multiple of a class of an E-O stratum. When the element w is minimal in its S_{g}-coset, this stratum is indexed by the coset spanned by w. Our considerations give an extension of this map from elements minimal in their cosets to a larger class of elements. We give some examples of this extension but in general it seems a very mysterious construction.

Another application is to the irreducibility of our strata (and hence also to the strata of the E-O stratification as they are images of some of our strata). As the strata are normal this is equivalent to the connectedness of a stratum and this connectedness can sometimes (cf., Thm. 11.5) be proved via an arithmetic argument. It is natural to ask if this method produces all the irreducible strata and for the characteristic large enough (the size depending on g) we can show that indeed it does. This is done using a Pieri type formula for our strata obtained by applying a result of Pittie and Ram. A Pieri type formula for multiplying the class of a connected cycle by an ample line bundle has as a consequence that a part of boundary is supported by an ample line bundle and hence that this part of the boundary is connected. Applying this to λ_{1} which is an ample line bundle on A_{g} allows to show that our results are optimal, cf., [Ha07]. We are forced to assume that the characteristic is large (and are unable to specify how large) as we do not know by which power of λ_{1} one needs to twist the exterior powers of the dual of the Hodge bundle to make these generated by global sections.

There is a particular element of w_{∅} ∈ W_{g} which is the largest of the elements that are minimal in their right S_{g}-cosets and which has the property that \overline{U}_{w_{∅}} maps generically of finite degree onto A_{g}. It is really the strata that are contained in this stratum that seem geometrically related to A_{g} and indeed the elements w ∈ W_{g} lying below w_{∅} are the ones of most interest. (The rest of F_{g} appears mostly as a technical device for relating our strata to the Schubert varieties.) It should be of particular interest to understand the composite map \overline{U}_{w_{∅}} ⊂ F_{g} → A_{g}. It follows from a result of Oort on Dieudonné modules that the inverse image of an open E-O stratum under this map is a locally constant fibration. This focuses interest on its fibres, a fibre depending only on the element of ν ∈ W_{g}/S_{g} that specifies the E-O stratum. We call these fibres punctual flag spaces, (see Section 9 for details): we determine their connected components, showing in particular that two points in the same connected component can be connected by a sequence of quite simple rational curves. We also show that knowing which strata U_{w} have non-empty intersections with a given punctual flag space would determine the inclusion relations between the E-O strata.

A geometric point x of the stratum \overline{U}_{w_{∅}} corresponds to a symplectic flag of subgroup schemes of the kernel of multiplication by p on the principally polarised abelian variety which is the image of x in A_{g} under the map F_{g} → A_{g}. This is reminiscent of de Jong’s moduli stack S(g, p) of Γ_{0}(p)-structures, cf., [Jo93]. The major difference (apart from the fact that \overline{U}_{w_{∅}} only makes sense in positive characteristic) is that the g-dimensional element of the flag is determined by the abelian variety in our case. We shall indeed identify \overline{U}_{w_{∅}} with the component of the fibre at p of S(g, p) that is the closure of the ordinary abelian varieties provided with a flag on the local part of the kernel of multiplication by p. As a consequence we get that that component of S(g, p) is normal and Cohen-Macaulay.

This paper is clearly heavily inspired by [Oo01]. The attentive reader will notice that we reprove some of the results of that paper, sometimes with proofs that are very close to the proofs used by Oort. We justify such duplications by our desire to emphasise the relations with the combinatorics of W_{g} and the flag spaces. Hence, we start with (a rather long) combinatorial section where the combinatorial aspects have been separated from the geometric ones. We hope that this way of presenting the material will be as clarifying to the reader as it has been to us. We intend to continue to exploit the approach using the flag spaces in a future paper that will deal with K3-surfaces. Since its announcement in [Ge99], our idea of connecting the E-O stratification on A_{g} with the Weyl group and filtrations on the de Rham cohomology has been taken up in other work. In this connection we want to draw attention to papers by Moonen and Wedhorn, cf., [Mo01, MW04].

We would like to thank Piotr Pragacz for some useful comments.

Conventions. We shall exclusively work in positive characteristic p > 0 (note however that in Section 13 the symbol p will also be a polynomial variable). After having identified final types and final elements in Section 2 we shall often use the same notation for the final type (which is a function on {1, …, 2g}) and the corresponding final element (which is an element of the Weyl group W_{g}). In Sections 10 and 11 our strata will be considered in flag spaces over not just A_{g} and Ã_{g} but also over the corresponding moduli stacks with a level structure. We shall define several natural objects, such as the Hodge bundle, over several different spaces (such as the moduli space of abelian varieties as well as toroidal compactifications of it). In order not to make the notation overly heavy, the same notation (such as 𝔼) will normally be used for the objects on different spaces. As the objects in question will be compatible with pullback this should not cause confusion. Sometimes, when there might still be such a risk we shall use subscripts (such as 𝔼_{A_{g}} resp. 𝔼Ã_{g}) to distinguish between them. Also, when there is no risk of confusion we shall use the same name (like U_{w} and \overline{U}_{w}) for a stratum and its extension to the compactified moduli space.

Our moduli objects such as A_{g} are really Deligne-Mumford stacks. However, in order to avoid what we have found to be a sometimes awkward terminology (such as “flag stacks”), we shall usually speak of them as spaces rather than stacks. In a similar vein by for instance a “locally closed subset” of an algebraic stack we shall mean a reduced locally closed substack.

Contents

2 Combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565 2.1 Final Elements in The Weyl Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565 2.2 Final Types and Young Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 2.3 Canonical Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572 2.4 Admissible Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 2.5 Shuffles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576 3 The Flag Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578 3.1 The Flag space of the Hodge bundle . . . . . . . . . . . . . . . . . . . . . . . . . . . 578 3.2 The canonical flag of an abelian variety . . . . . . . . . . . . . . . . . . . . . . . . 580 4 Strata on the Flag Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582 4.1 The Stratification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582 4.2 Some Important Strata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 583 4.3 Shuffling flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588 4.4 The E-O strata on A_{g} ⊗ 𝔽_{p} . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 5 Extension to the boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 6 Existence of boundary components . . . . . . . . . . . . . . . . . . . . . . . . 593 7 Superspecial fibres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594 8 Local structure of strata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598 8.1 Stratified Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598 8.2 Height 1-Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598 9 Punctual flag spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601 10 Pieri formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 606 11 Irreducibility properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611 12 The Cycle Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 614 12.1 Fulton’s formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 12.2 The p-rank strata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 617 12.3 The a-number Strata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619 12.4 Positivity of tautological classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621 13 Tautological rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621 14 Comparison with S(g, p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625 15 Appendix g = 3, 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626 15.1 Admissible Strata for g = 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626 15.2 E-O Cycle Classes for g = 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626 15.3 E-O Cycle Classes for g = 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627

2 Combinatorics

This section is of a preparatory nature and deals with the combinatorial aspects of the E-O stratification. The combinatorics is determined by the Weyl group of the symplectic group of degree g. A general reference for the combinatorics of Weyl groups is [BL00]. We start by recalling some general notations and facts about W_{g} and its Bruhat-Chevalley order. We then go on to give various descriptions of the minimal elements in the S_{g} cosets (which we presume are well-known). The short subsection on shuffles will be used to understand the rôle that the multiplicative and étale parts of the Barsotti-Tate group play in our stratification in the case of positive p-rank.

2.1 Final Elements in The Weyl Group

The Weyl group W_{g} of type C_{g} in Cartan’s terminology is isomorphic to the semi-direct product S_{g} ⋉ (ℤ/2ℤ)^{g}, where the symmetric group S_{g} on g letters acts on (ℤ/2ℤ)^{g} by permuting the g factors. Another description of this group, and the one we shall use here, is as the subgroup of the symmetric group S_{2g} formed by elements which map any symmetric 2-element subset of {1, …, 2g} of the form {i, 2g + 1 − i} to a subset of the same type:

W_{g} = {σ ∈ S_{2g}: σ(i) + σ(2g + 1 − i) = 2g + 1 for i = 1, …, g}.

The function i ↦ 2g + 1 − i on the set {1, …, 2g} will occur frequently. We shall sometimes use the notation \overline{ı} for 2g + 1 − i. Using it we can say that σ ∈ S_{2g} is an element of W_{g} precisely when σ(\overline{ı}) = \overline{σ(i)} for all i. This makes the connection with another standard description of W_{g}, namely as a group of signed permutations. An element w in this Weyl group has a length and a codimension defined by:

ℓ(w) = #{i < j ≤ g : w(i) > w(j)} + #{i ≤ j ≤ g : w(i) + w(j) > 2g + 1}

and

codim(w) = #{i < j ≤ g: w(i) < w(j)} + #{i ≤ j ≤ g: w(i) + w(j) < 2g + 1}

and these satisfy the equality

ℓ(w) + codim(w) = g^{2}.

We shall use the following notation for elements in W_{g}. By [a_{1}, a_{2}, …, a_{2g}] we mean the permutation of {1, 2, …, 2g} with σ(i) = a_{i}. Since σ(i) determines σ(2g + 1 − i) for 1 ≤ i ≤ g, sometimes we use the notation [a_{1}, …, a_{g}] instead (when the a_{i} are single digits we shall often dispense with the commas and write [a_{1} … a_{g}] which should cause no confusion). We shall also use cycle notation for permutations. In particular, for 1 ≤ i < g we let s_{i} ∈ S_{2g} be the permutation (i, i + 1)(2g − i, 2g + 1 − i) in W_{g} which interchanges i and i + 1 (and then also 2g − i and 2g + 1 − i) and we let s_{g} = (g, g + 1) ∈ S_{2g}. Then the pair (W = W_{g}, S = {s_{1}, …, s_{g}}) is a Coxeter system.

Let (W, S) be a Coxeter system and a ∈ W. If X is a subset of S we denote by W_{X} the subgroup of W generated by X. It is well-known that for any subset X of S there exists precisely one element w of minimal length in aW_{X} and it has the property that every element w′ ∈ aW_{X} can be written in the form w′ = wx with x ∈ W_{X} and ℓ(w′) = ℓ(w) + ℓ(x). Such an element w is called an X-reduced element, cf., [GrLie4-6, Ch. IV, Excercises §1].

Let W = W_{g} be the Weyl group and S be the set of simple reflections. If we take X = S\{s_{g}}, then we find

W_{X} = {σ ∈ W_{g}: σ{1, 2, …, g} = {1, 2, …, g}} ≅ S_{g}.

There is a natural partial order on W_{g} with respect W_{X}, the Bruhat-Chevalley order. It is defined in terms of Schubert cells X(w_{i}) by:

w_{1} ≤ w_{2} ⟺ X(w_{1}) ⊆ X(w_{2}).

Equivalently, if for w ∈ W_{g} we define

r_{w}(i, j) := #{a ≤ i: w(a) ≤ j},  (1)

then we have the combinatorial characterization

w_{1} ≤ w_{2} ⟺ r_{w_{1}}(i, j) ≥ r_{w_{2}}(i, j) for all 1 ≤ i, j ≤ 2g.

(Indeed it is easy to see that it is enough to check this for all 1 ≤ i ≤ g and 1 ≤ j ≤ 2g.) Chevalley has shown that w_{1} ≥ w_{2} if and only if any (hence every) X-reduced expression for w_{1} contains a sub-expression (obtained by just deleting elements) that is a reduced expression for w_{2}; here reduced means that w_{2} is written as a product of ℓ(w_{2}) elements of S. Again, a reference for these facts is [BL00].

We now restrict to the following case. Let V be a symplectic vector space over ℚ and consider the associated algebraic group G = Sp(V). If E ⊂ V is a maximal isotropic subspace, then the stabiliser of the flag (0) ⊂ E ⊂ V is a parabolic subgroup conjugate to the standard parabolic subgroup corresponding to X. Since X is a heavily used letter we shall use H := S\{s_{g}} ⊂ S instead. Hence W_{H} will denote the subgroup of W_{g} generated by the elements of H and we will also use the notation P_{H} for the parabolic subgroup corresponding to W_{H}, i.e., the subgroup of the symplectic group stabilizing a maximal totally isotropic subspace. Note that W_{H} consists of the permutations of W_{g} that stabilise the subsets {1, …, g} and {g + 1, …, 2g} and that the restriction of the action of an element of W_{H} to {1, …, g} determines the full permutation. Therefore, we may identify W_{H} with S_{g}, the group of permutations of {1, …, g}, and we shall do so without further notice. (This is of course compatible with the fact that H spans an A_{g−1}-subdiagram of the Dynkin diagram of G.) There are 2^{g} = |W_{g}|/|W_{H}| elements in W_{g} which are H-reduced elements. These 2^{g} elements will be called final elements of W_{g}. The Bruhat-Chevalley order between elements in W_{g} as well as the condition for being H-reduced can be conveniently expressed in terms of the concrete representation of elements of W_{g} as permutations in the following way.

Let A, B two finite subsets of {1, 2, …, g} of the same cardinality. We shall write A ≺ B if for all 1 ≤ i ≤ |A| the i’th largest element of A is at most equal to the i’th largest element of B.

Lemma 2.1. i) If w = [a_{1}a_{2} … a_{g}] and w′ = [b_{1}b_{2} … b_{g}] are two elements of W_{g}, then w ≤ w′ in the Bruhat-Chevalley order precisely when for all 1 ≤ d ≤ g we have {a_{1}, a_{2}, …, a_{d}} ≺ {b_{1}, b_{2}, …, b_{d}}.

ii) Let w = [a_{1}a_{2} … a_{g}] ∈ W_{g}. Denote the final element of wW_{H} by w^{f}. For w′ = [b_{1}b_{2} … b_{g}] ∈ W_{g} we have w^{f} ≤ w′ in the Bruhat-Chevalley order precisely when {a_{1}, a_{2}, …, a_{g}} ≺ {b_{1}, b_{2}, …, b_{g}}.

iii) An element σ ∈ W_{g} is H-reduced (or final) if and only if σ(i) < σ(j) for all 1 ≤ i < j ≤ g. Also, σ is H-reduced if and only if σ sends the first g − 1 simple roots into positive roots.

Proof. See for instance [BL00, p. 30]. ⊓ ⊔

2.2 Final Types and Young Diagrams

There are other descriptions of final elements that are sometimes equally useful. They involve maps of {1, 2, …, 2g} to {1, 2, …, g} and certain Young diagrams. We begin with the maps.

Definition 2.2. A final type (of degree g) is an a non-decreasing surjective map

ν: {0, 1, 2, …, 2g} → {0, 1, 2, …, g}

satisfying

ν(2g − i) = ν(i) − i + g for 0 ≤ i ≤ g.

We always have ν(0) = 0 and ν(2g) = g. Note that we either have ν(i + 1) = ν(i) and then ν(2g − i) = ν(2g − i − 1) + 1 or ν(i + 1) = ν(i) + 1 and then ν(2g − i) = ν(2g − i − 1). A final type is determined by its values on {0, 1, …, g}. There are 2^{g} final types of degree g corresponding to the vectors (ν(i + 1) − ν(i))_{i=0}^{g−1} ∈ {0, 1}^{g}. The notion of a final type was introduced by Oort (cf. [Oo01]).

To an element w ∈ W_{g} we can associate the final type ν_{w} defined by

ν_{w}(i) := i − r_{w}(g, i).

This is a final type because of the rule r_{w}(g, 2g − i) − r_{w}(g, i) = g − i which follows by induction on i ∈ {1, …, g} from the fact that w(2g + 1 − a) = 2g + 1 − w(a). It depends only on the coset wW_{H} of w since a permutation of {a : 1 ≤ a ≤ g} does not change the definition of r_{w}(g, i) = #{a ≤ g: w(a) ≤ i}.

Conversely, to a final type ν we now associate an element w_{ν} of the Weyl group, a permutation of {1, 2, …, 2g} as follows. Let

β = {i_{1}, i_{2}, …, i_{k}} = {1 ≤ i ≤ g: ν(i) = ν(i − 1)}

with i_{1} < i_{2} < … given in increasing order and let

β^{c} = {j_{1}, j_{2}, …, j_{g−k}}

be the elements of {1, 2, …, g} not in ξ, in decreasing order. We then define a permutation w_{ν} by mapping 1 ≤ s ≤ k to i_{s} and k+1 ≤ s ≤ g to 2g+1−j_{s−k}. The requirement that w_{ν} belong to W_{g} now completely specifies w_{ν} and by construction w_{ν}(i) < w_{ν}(j) if 1 ≤ i < j ≤ g. Thus w_{ν} is a final element of W_{g}. It is clear from 2.1 that we get in this way all final elements of W_{g}. The Bruhat-Chevalley order for final elements can also be read off from the final type ν. We have w ≥ w′ if and only if ν_{w} ≥ ν_{w′}. This follows from 2.1, ii).

We summarise:

Lemma 2.3. By associating to a final type ν the element w_{ν} and to a final element w ∈ W_{g} the final type ν_{w} we get an order preserving bijection between the set of 2^{g} final types and the set of final elements of W_{g}.

The final types are in bijection with certain Young diagrams. Our Young diagrams will be put in a position that is opposite to the usual positioning, i.e., larger rows will be below smaller ones and the rows will be lined up to the right (see next example). Furthermore, we shall make Young diagrams correspond to partitions by associating to a diagram the parts that are the lengths of its rows. Given g we shall say that a Young diagram is final of degree g if its parts are ≤ g and no two parts are equal. They therefore correspond to subsets ξ of {1, 2, …, g}. We write such a ξ as {ξ_{1}, …, ξ_{r}} with g ≥ ξ_{1} > ⋯ > ξ_{r}.

To a final type ν we now associate the Young diagram Y_{ν} whose associated subset ξ is defined by

ξ_{j} = #{i: 1 ≤ i ≤ g, ν(i) ≤ i − j}.

A pictorial way of describing the Young diagram is by putting a stack of i − ν(i) squares in vertical position i for 1 ≤ i ≤ g.

Example 2.4. This example corresponds to

{ν(i): i = 1, ..., g} = {1, 2, …, g − 5, g − 5, g − 4, g − 4, g − 3, g − 3}

and hence ξ = {5, 3, 1} (see Fig. 1).

123 … g − 1 g

Fig. 1. Young diagram with ξ = {5, 3, 1}.

The final elements w in W_{g} are in 1-1-correspondence with the elements of W_{g}/W_{H}. The group W_{g} acts on W_{g}/W_{H} by multiplication on the left, i.e., by the permutation representation. Therefore W_{g} also acts on the set of final types and the set of final Young diagrams. To describe these actions we need the notion of a break point.

By a break point of a final type ν we mean an integer i with 1 ≤ i ≤ g such that either

1. ν(i − 1) = ν(i) ≠ ν(i + 1), or

2. ν(i − 1) ≠ ν(i) = ν(i + 1).

If i is not a break point of ν = ν_{w} then either ν(i + 1) = ν(i − 1) or ν(i + 1) = ν(i − 1) + 2 and therefore ν_{s_{i}w} = ν_{w}. In particular, g is always a break point. The set of break points of ν equals

{1 ≤ i ≤ g: ν_{s_{i}w} ≠ ν_{w}}.

Since ν = ν_{w} determines a coset wW_{H}, we have that i is not a break point of ν if and only if w^{−1}s_{i}w ∈ W_{H}, i.e., if and only if wW_{H} is a fixed point of s_{i} acting on W_{g}/W_{H}. The action of s_{i} on a final type ν is as follows: if i is not a break point then ν is fixed, otherwise replace the value of ν at i by ν(i) + 1 if ν(i − 1) = ν(i) and ν(i) − 1 otherwise.

If w is a final element given by the permutation [a_{1}, a_{2}, …, a_{g}], then it defines a second final element, called the complementary permutation, defined by the permutation [b_{1}, b_{2}, …, b_{g}], where b_{1} < b_{2} < … < b_{g} are the elements of the complement {1, 2, …, 2g}\{a_{1}, …, a_{g}}. If ξ is the partition defining the Young diagram of w then ξ^{c} defines the Young diagram of the complementary permutation. The set of break points of w (that is, of the corresponding ν) and its complementary element are the same.

Lemma 2.5. Let w ∈ W_{g} be a final element with associated final type ν and complementary element v.

i) We have that v = σ_{1}wσ_{0} = wσ_{1}σ_{0}, where σ_{0} (resp. σ_{1}) is the element of S_{g} (resp. W_{g}) that maps i with 1 ≤ i ≤ g to g + 1 − i (resp. to 2g + 1 − i).

ii) Let i ∈ {1, …, g}. If ν(i−1) ≠ ν(i) then v^{−1}(i) = ν(i) and if ν(i−1) = ν(i) then v^{−1}(i) = 2g + 1 − ν(2g + 1 − i).

Proof. As w maps i to a_{i} we have that 2g + 1 − a_{i} is not among the a_{j} and hence b_{i} = 2g + 1 − a_{g+1−i} (using that both the a_{i} and b_{i} are increasing sequences). This gives σ_{1}wσ_{0}(i) = σ_{1}(a_{g+1−i}) = 2g + 1 − a_{g+1−i} = b_{i} but we note that as w ∈ W_{g} it commutes with σ_{1}. Thus i) holds.

If ν(i − 1) ≠ ν(i) and, say, ν(i) = i − k then we have k natural numbers 1 ≤ i_{1} < i_{2} < ⋯ < i_{k} < i such that ν(i_{j} − 1) = ν(i_{j}). By the definition of v we then have v(i − k) = (i − k) + k since the k values i_{j} (j = 1, …, k) are values for v, hence not of w. The second part is checked similarly. ⊓ ⊔

Remark 2.6. The elements σ_{1} and σ_{0} of course have clear root theoretic relevance, they are respectively the longest element of W_{g} and S_{g}. Multiplication by σ_{1}σ_{0} reverses the Bruhat-Chevalley order. Similarly it is clear that going from a final element to its complementary element also reverses the Bruhat-Chevalley order among the final elements and the first part of our statement says that that operation is obtained by multiplying by σ_{1} and σ_{0}. Somewhat curiously our use of the complementary permutation seems unrelated to these facts.

In terms of Young diagrams the description is analogous and gives us a way to write the element w_{ν} as a reduced product of simple reflections. To each s_{i} we can associate an operator on final Young diagrams. If Y is a final diagram, s_{i} is defined on Y by adding or deleting a box in the i’th column if this gives a final diagram (only one of the two can give a final diagram) and then s_{i}Y will be that new diagram; if neither adding nor deleting such a box gives a final Young diagram we do nothing. In terms of the description as subsets ξ adding a box corresponds to g + 1 − i ∈ ξ and g + 2 − i ∉ ξ and then s_{i}ξ = (ξ \ {g + 1 − i}) ∪ {g + 2 − i}. It is then clear that for any final Young diagram Y there is a word s_{i_{1}}s_{i_{2}}s_{i_{3}} ⋯ s_{i_{k}} such that Y = s_{i_{1}}s_{i_{2}}s_{i_{3}} ⋯ s_{i_{k}}∅, where ∅ denotes the empty Young diagram. Comparison with the action of s_{i} on final types and the correspondence between final types and Young diagrams shows that the action of s_{i} on diagrams is indeed obtained from that on final types. If we now have a word t = s_{i_{1}} … s_{i_{k}} in the s_{i} we can make it act on Young diagram by letting each individual s_{i} act as specified. Note that this action only depends on the image of t in W_{g} but for the moment we want to consider the action by words. We define the area of a Young diagram Y to be the number of boxes it contains. We shall say that the word t is building if the area of t∅ is equal to k, the length of the word (not the resulting element). This is equivalent to the action of s_{i_{r}} adding a box to s_{i_{r+1}} … s_{i_{k}}∅ for all r.

Lemma 2.7. i) If ν is a final type and t is a word in the s_{i} such that Y_{ν}^{c} = t∅ then w_{ν} = w, where w is the image of t in W_{g} and ℓ(w_{ν}) = g(g + 1)/2 − area(Y_{ν}).

ii) t is H-reduced if and only if t is building.

Proof. To prove i) we begin by noting that t∅ depends only on the image of t in W_{g} so that i) is independent of the choice of t. Hence we may prove it by choosing a particular t using induction on the area of Y_{ν}^{c}. Note that g(g+1)/2− area(Y_{ν}) = area(Y_{ν}^{c}) so that the last part of i) says that ℓ(w_{ν}) = area(Y_{ν}^{c}). The final type ν with ν(i) = 0 for i ≤ g corresponds to final diagram Y_{ν} with empty complementary diagram. We have w_{ν} = 1 ∈ W_{g}, the empty product and it has length 0. This proves the base case of the induction. Suppose we proved the statement for diagram Y_{ν} with area(Y_{ν}^{c}) ≤ a. Adding one block to Y_{ν}^{c} to obtain Y_{ν′}^{c} means that for some i we have g + 1 − i ∈ ξ^{c} and g + 2 − i ∉ ξ^{c}, where ξ^{c} is the subset corresponding to Y_{ν}^{c}, the new subset is (ξ′)^{c} = (ξ \ {g − i}) ∪ {g − i + 1}. This means that if i < g there are b < a ≤ g such that w_{ν}(b) = i and w_{ν}(a) = 2g − i and w_{ν′}(b) = i + 1 and w_{ν′}(a) = 2g + 1 − i and the rest of the integers between 1 and g remain unchanged. (The case when i = g is similar and left to the reader.) This makes it clear that we have w_{ν′} = s_{i}w_{ν} so by the induction t maps to w_{ν}. It remains to show the formula for ℓ(w_{ν}). In the definition of ℓ(w_{ν}) only the second term contributes as w_{ν}(i) < w_{ν}(j) if i < j ≤ g. Now, the only difference in the collections of sums w(i) + w(j) for i ≤ j and w equal to w_{ν} and w_{ν′} appears for (i, j) = (b, a) and we have w_{ν}(b)+w_{ν}(a) = 2g and w_{ν′}(b)+w_{ν′}(a) = 2g +2 so that the length of w_{ν′} is indeed one larger than that of w_{ν}.

As for ii), we have that t∅ = Y_{ν}^{c}, where ν is the final type of w and then ii) is equivalent to t being H-reduced if and only area(Y_{ν}^{c}) is equal to the length of t. However by i) we know that area(Y_{ν}^{c}) is equal to ℓ(w_{ν}) and t is indeed H-reduced precisely when its length is equal to ℓ(w_{ν}). ⊓ ⊔

Example 2.8. Consider again the Young diagram of the previous example but now for g = 5 (see Fig. 2). We have ξ = {5, 3, 1} and thus ξ^{c} = {2, 4},

1 2 3 4 5

Fig. 2. Young diagram with g = 3 and ξ = {5, 3, 1}

so w_{ν} = [13579] and w_{ν} can be written as s_{4}s_{5}s_{2}s_{3}s_{4}s_{5} (we emphasize that permutations act from the left on diagrams).

We now characterise final types. Besides the function ν = ν_{w} defined by

ν(i) = i − #{a ≤ g: w(a) ≤ i} = i − r_{w}(g, i)

and extended by ν(2g − i) = ν(i) − i + g for i = 0, …, g, we define a function μ = μ_{w} on the integers 1 ≤ i ≤ 2g by

μ(i) := (max{w^{−1}(a): 1 ≤ a ≤ i} − g)^{+} ,

where (x)^{+} := max(x, 0). Alternatively, we have

μ(i) = min {0 ≤ j ≤ g : r_{w}(g + j, i) = i }.

Note that both μ and ν are non-decreasing functions taking values between 0 and g. Also for i = 1, …, 2g − 1 we have ν(i + 1) = ν(i) or ν(i + 1) = ν(i) + 1 and ν(2g) = μ(2g) = g. If w is a final element then ν_{w} is the final type associated to w. For an arbitrary w the function ν is the final type of the final element in the coset wS_{g}.

Lemma 2.9. We have μ_{w}(i) ≥ ν_{w}(i) for 1 ≤ i ≤ 2g with equality precisely when w is a final element and then ν_{w} is the final type of w.

Proof. We suppress the index w and first prove the inequality μ ≥ ν. Let 1 ≤ i ≤ g. Suppose that μ(i) = m, i.e., the maximal j with w(j) in [1, i] is g + m. Then there are at most m elements from [g + 1, 2g] which map under w into [1, i] and there are at least i − m elements from [1, g] with their image under w in [1, i], so i − ν(i) ≥ i − m, in other words ν(i) ≤ μ(i). For i in the interval [g + 1, 2g] we consider ν(2g − i) = #{a ≤ g: w(a) > i}. If μ(2g − i) = m then there are at least g − m elements from [1, g] mapping into [1, 2g − i] and thus ν(2g − i) is at most equal to m.

If w is final then w respects the order on [1, g] and this implies that if #{a ≤ g: w(a) ≤ t} = n then t − n elements from [g + 1, 2g] map under w to [1, t], so the maximum element from [g + 1, 2g] mapping into [1, t] is g + t − n. Hence μ(t) = t − n = ν(t).

Conversely, if μ(i) = ν(i) then this guarantees that w(i) < w(j) for all pairs 1 ≤ i < j ≤ g. Thus w is a final element. ⊓ ⊔

Corollary 2.10. Let w ∈ W_{g}. Then w is a final element if and only if r_{w}(g + ν_{w}(i), i) = i for all 1 ≤ i ≤ g.

Proof. Lemma 2.9 says that if w is final then we have

ν(i) = μ(i) = min {0 ≤ j ≤ g : r_{w}(g + j, i) = i }

and in particular that ν(i) ∈ {0 ≤ j ≤ g : r_{w}(g + j, i) = i } which gives one direction.

Conversely, if we have r_{w}(g + ν_{w}(i), i) = i, then ν_{w}(i) ≥ μ_{w}(i) and then Lemma 2.9 gives that w is final. ⊓ ⊔

2.3 Canonical Types

We now deal with an iterative way of constructing the function ν starting from its values on the endpoints and applying it repeatedly.

A final type ν is given by specifying ν(j) for j = 1, …, 2g. But it suffices to specify the values of ν for the break points of ν. Under ν an interval [i_{1}, i_{2}] between two consecutive break points of ν is either mapped to an interval of length i_{2} − i_{1} or to one point. However, the image points ν(i_{1}) and ν(i_{2}) need not be break points of ν. Therefore we enlarge the set of break points to a larger set C_{ν}, called the canonical domain. We define C_{ν} to be the smallest subset of {0, 1, …, 2g} containing 0 and 2g such that if j ∈ C_{ν} then also 2g − j ∈ C_{ν} and if j ∈ C_{ν} then ν(j) ∈ C_{ν}. It is obtained by starting from R = {0, 2g} and adding the values ν(k) and ν(2g−k) for k ∈ R and continuing till this stabilises. The restriction of ν to C_{ν} is called a canonical type. We wish to see that the canonical domain C_{ν} contains the break points of ν and hence that we can retrieve ν from the canonical type of ν. To see this we need a technical lemma (its formulation is somewhat obscured by the fact that we also want to use it in another slightly different context).

Definition-Lemma 2.11. We shall say that a subset S ⊆ {0, 1, …, 2g} is stable if it has the property that it contains 0 and is stable under i ↦ i^{⊥} := 2g −i. For a stable subset S a map f: S → S ∩ {0, 1, …, g} is adapted to S if f(0) = 0 and f(2g) = g, if it is contracting, i.e., it is increasing and we have f(j) − f(i) ≤ j − i for i < j and if it fulfills the following complementarity condition: For any two consecutive i, j ∈ S (i.e., i < j and there are no k ∈ S with i < k < j) we have f(j) − f(i) = j − i ⇒ f(j^{⊥}) = f(i^{⊥}).

i) If S is stable and f is a non-surjective function adapted to S then there is a proper subset T ⊂ S such that f_{|T} is adapted to T.

ii) If S is stable and f is a surjective function adapted to S then for any two consecutive i, j ∈ S we have either f(i) = f(j) or f(j) − f(i) = j − i.

iii) We say that (S, f) is minimally stable if S is stable and f is adapted to S and furthermore there is no proper stable subset T ⊂ S for which f_{|T} is adapted to it, then the function ν: {1, 2, …, 2g} → {1, 2, …, g} obtained from f by extending it linearly between any two consecutive i, j ∈ S is a final type, S = C_{ν} and ν is the unique final extension of f. Conversely, if f is the canonical type of a final type ν then (C_{ν}, f) is minimally stable and in particular ν is the linear extension of its canonical type.

Proof. For i) consider T = f(S) ∪ (f(S))^{⊥}. It is clearly stable under f and ⊥ and contains 0. If f is not surjective then T is a proper subset of S.

Assume now that we are in the situation of ii). We show that if i < j ∈ S are consecutive then either f(j) − f(i) = j − i or f(i) = f(j) by descending induction on j − i.

By induction we are going to construct a sequence i_{k} < j_{k} ∈ S k = 1, 2, … of consecutive elements such that either (i_{k−1}, j_{k−1}) = (j_{k}^{⊥}, i_{k}^{⊥}) or (f(i_{k}), f(j_{k})) = (i_{k−1}, j_{k−1}) but not both (i_{k−1}, j_{k−1}) = (j_{k}^{⊥}, i_{k}^{⊥}) and (i_{k−2}, j_{k−2}) = (j_{k−1}^{⊥}, i_{k−1}^{⊥}) and in any case j_{k} − i_{k} = j − i. We start by putting i_{1} := i, j_{1} := j. Assume now that i_{k} < j_{k} have been constructed. If we do not have i_{k}, j_{k} ≤ g, then as g ∈ S, we must have j_{k}^{⊥}, i_{k}^{⊥} ≤ g and then we put (i_{k+1}, j_{k+1}) = (j_{k}^{⊥}, i_{k}^{⊥}). If we do have i_{k}, j_{k} ≤ g then by the surjectivity of f there are i_{k+1}, j_{k+1} ∈ S such that f(i_{k+1}) = i_{k} and f(j_{k+1}) = j_{k}. As f is increasing i_{k+1} < j_{k+1} and by choosing i_{k+1} to be maximal and j_{k+1} to be minimal we may assume that they are neighbours. We must have that j_{k+1} − i_{k+1} = j_{1} − i_{1}. Indeed, we have f(j_{k+1}) − f(i_{k+1}) ≤ j_{k} − i_{k} as f is contracting. If we have strict inequality we have j − i = j_{k} − i_{k} < j_{k+1} − i_{k+1} and hence by the induction assumption we have either that j_{k} − i_{k} = f(j_{k+1}) − f(i_{k+1}) = j_{k+1} − i_{k+1} which is a contradiction or j_{k} = f(j_{k+1}) = f(i_{k+1}) = i_{k} which is also a contradiction. Hence we have j_{k+1} − i_{k+1} = j_{k} − i_{k} = j − i and we have verified the required properties of (i_{k+1}, j_{k+1}).

There must now exist 1 ≤ k < ℓ such that (i_{k}, j_{k}) = (i_{ℓ}, j_{ℓ}) and we pick k minimal for this property. If k = 1 we have either j −i = j_{ℓ−1} −i_{ℓ−1} = f(j_{ℓ})− f(i_{ℓ}) = f(j) − f(i) or j − i = j_{ℓ−2} − i_{ℓ−2} = f(j_{ℓ−1}) − f(i_{ℓ−1}) = f(i^{⊥}) − f(j^{⊥}) which implies that f(i) = f(j) by assumptions on f. We may hence assume that k > 1. We can not have both (i_{k−1}, j_{k−1}) = (j_{k}^{⊥}, i_{k}^{⊥}) and (i_{ℓ−1}, j_{ℓ−1}) = (j_{ℓ}^{⊥}, i_{ℓ}^{⊥}) as that would contradict the minimality of k. If (i_{k−1}, j_{k−1}) = (j_{k}^{⊥}, i_{k}^{⊥}) and (i_{ℓ−1}, j_{ℓ−1}) = (f(i_{ℓ}), f(j_{ℓ})) then we get j_{k−1}−i_{k−1} = j−i = j_{ℓ−1}−i_{ℓ−1} = f(i_{k−1}^{⊥}) − f(j_{k−1}^{⊥}) which implies f(j_{k−1}) = f(i_{k−1}) which is either what we want in case k = 2 or a contradiction. Similarly the case (i_{ℓ−1}, j_{ℓ−1}) = (j_{ℓ}^{⊥}, i_{ℓ}^{⊥}) and (i_{k−1}, j_{k−1}) = (f(i_{k}), f(j_{k})) leads to a contradiction as does the case (i_{ℓ−1}, j_{ℓ−1}) = (f(i_{ℓ}), f(j_{ℓ})) and (i_{k−1}, j_{k−1}) = (f(i_{k}), f(j_{k})).

Finally, to prove the first part of iii) we note that by ii) for i < j ∈ S consecutive we either have f(i) = f(j) or f(j) − f(i) = j − i. This means that the linear extension ν of f has the property that for 1 ≤ i ≤ 2g we have either ν(i) = ν(i − 1) or ν(i) = ν(i − 1) + 1 and if ν(i) = ν(i − 1) + 1 we get by the conditions on f that ν(2g − i + 1) = ν(2g − i). If for some i we have ν(i) = ν(i−1) and ν(2g −i+1) = ν(2g −i) we get that g = f(2g) = ν(2g) < g which is impossible by assumption and hence ν is indeed a final type. It is clear that S fulfills the defining property of C_{ν} so that S = C_{ν}. The conditions on a final element implies that ν(j) − ν(i) ≤ j − i for j < i and thus that f has a unique final extension.

Conversely, if ν is a final type then C_{ν} clearly fulfills the required conditions and we have just noted that ν(j)−ν(i) ≤ j −i for j < i. The complementarity condition for f|C_{ν} follows from the equivalence ν(i) = ν(i − 1) ⟺ ν(2g − i + 1) = ν(2g − i) + 1. ⊓ ⊔

We now give an interpretation of the canonical domain in terms of the Weyl group. Let v ∈ W_{g} be a final element. A canonical fragment of v is an interval ]i, j] := [i + 1, …, g] ⊆ {1, 2, …, 2g} which is maximal with respect to the requirement that for all k ≥ 1 the set v^{k}(]i, j]) is an interval.

Proposition 2.12. Let v ∈ W_{g} be a final element, w its the complementary element, and ν the final type of w.

i) The set {1, 2, …, 2g} is the disjoint union of the canonical fragments of v. Moreover, the canonical fragments of v are permuted by v.

ii) If ]i, j] is a canonical fragment of v, and if ν(j) ≠ ν(j −1), then ν maps ]i, j] bijectively to ]ν(i), ν(j)].

iii) If ]i, j] is a canonical fragment of v, then so is ]\overline{ȷ}, \overline{ı}].

iv) The upper endpoints of the canonical fragments of v together with 0 comprise exactly the canonical domain for w.

Proof. If two canonical fragments I and J meet, their union K will be an interval and as v^{k}(K) = v^{k}(I) ∪ v^{k}(J), we see that v^{k}(K) will be an interval for all k. By the maximality we get that I = J. On the other hand ]i − 1, i] fulfills the stability condition so that i lies in a fragment. Hence {1, 2, …, 2g} is the disjoint union of fragments.

Let now R be the set of upper endpoints of fragments together with 0. As {1, 2, …, 2g} is the disjoint union of the fragments of v it follows that if ]i, j] is a fragment, then i is also the upper end point of a fragment. Thus it follows from iii) that R is stable under i ↦ \overline{ı}. Let now i be an upper endpoint of a fragment. We want to show that ν(i) ∈ R and we may certainly assume that ν(i) ≠ 0 and we may also, by way of contradiction assume that i is a minimal upper endpoint for which ν(i) is not an upper endpoint. If ν(i) ≠ ν(i − 1), then v^{−1}(i) = ν(i) and hence ν(i) is an upper endpoint of a fragment. Hence we may pick j < i such that ν(i) = ν(i − 1) = ⋯ = ν(j) ≠ ν(j − 1). Then j can not belong to the same fragment as i and thus there must be an upper endpoint j ≤ k < i. Then ν(k) = ν(i) and by minimality of i we see that ν(k) is an upper endpoint which is a contradiction.

We therefore have shown that R contains 0 and is stable under i ↦ \overline{ı} and ν. Hence it contains the canonical domain. Let now j ∈ C_{ν} \ {0} and let i be the largest j ∈ C_{ν} such that i < j. We now want show by induction on k that v^{−k}(I), I :=]i, j], remains an interval for all k and that also v^{−k}(j) is one of its endpoints. Now, it follows from Lemma 2.5 that C_{ν} \ {0} is stable under v and hence v^{−k}(j) will be the only element of C_{ν} in v^{−k}(I). Under the induction assumption v^{−k}(I) is an interval with v^{−k}(j) as one of its endpoints and hence ν is constant on v^{−k}(I) by Lemma 2.11. By Lemma 2.5, an interval v^{−1} maps v^{−k}(I) to the interval with v^{−k−1}(j) as one of its endpoints. This means that I is contained in a fragment and R ∩ I = {j}. This means that there are no elements of R between consecutive elements of C_{ν} and hence R ⊆ C_{ν}. ⊓ ⊔

Corollary 2.13. By associating to a final type ν its canonical type, its Young diagram and the element w_{ν} we obtain a bijection between the following sets of cardinality 2^{g}: the set of final types, set of canonical types, the set of final Young diagrams and the set of final elements of W_{g}.

2.4 Admissible Elements

The longest final element of W_{g} is the element

w_{∅} := s_{g}s_{g−1}s_{g}s_{g−2}s_{g−1}s_{g} … s_{g}s_{1}s_{2}s_{3} … s_{g},

which as a permutation equals [g+1, g+2, …, 2g]. Elements of W_{g} that satisfy w ≤ w_{∅} are called admissible. We now characterise these.

Lemma 2.14. i) An element w ∈ W_{g} fulfills w ≤ w_{∅} if and only if we have w(i) ≤ g + i for all 1 ≤ i ≤ g.

ii) The condition that w ≤ w_{∅} is equivalent to r_{w}(i, g + i) = i for all 1 ≤ i ≤ g.

Proof. The first part follows immediately from the description of the Bruhat-Chevalley order (2.1) and the presentation of w_{∅}.

For the second part one easily shows that w(i) ≤ w_{∅}(i) = g + i for all 1 ≤ i ≤ g is equivalent to r_{w}(i, g + i) = i for all 1 ≤ i ≤ g which gives the first equivalence. ⊓ ⊔ Remark 2.15. The number of elements w ∈ W_{g} with w ≤ w_{∅} and of given length has recently been determined by J. Sjöstrand (cf., [Sj07]). This implies in particular that the number of elements w ∈ W_{g} with w ≤ w_{∅} equals

(x\frac{d}{dx})^{g} (\frac{1}{1 − x})_{|x=1/2} ,

a fact that we originally guessed from a computation for small g and a search in [S] leading to sequence A000629.

We give an illustration of the various notions for the case g = 2.

Example 2.16. g = 2. The Weyl group W_{2} consists of 8 elements. We list (see Fig. 3) the element, a reduced expression as a word (i.e. a decomposition w = s_{i_{1}} ⋯ s_{i_{k}} with k = ℓ(w)), its length, the functions ν and μ and for final elements we also give the partition defining the Young diagram.

w  s  ℓ ν  μ  Y
[4321] s_{1}s_{2}s_{1}s_{2} 4 {1, 2} {2, 2}
[4231] s_{1}s_{2}s_{1} 3 {1, 1} {2, 2}
[3412] s_{2}s_{1}s_{2} 3 {1, 2} {1, 2} ∅
[2413] s_{1}s_{2} 2 {1, 1} {1, 1} {1}
[3142] s_{2}s_{1} 2 {0, 1} {0, 2}
[2143] s_{1} 1 {0, 0} {0, 0}
[1324] s_{2} 1 {0, 1} {0, 1} {2}
[1234] 1 0 {0, 0} {0, 0} {1, 2}

Fig. 3. The g = 2 case.

The orbits of the complementary element will play an important rôle in our discussion of the canonical flag. Here we introduce some definitions pertaining to them.

Definition 2.17. Let w ∈ W_{g} be a final and let v be its complementary element. Assume that S is an orbit of the action of v on its fragments. As v commutes with i ↦ \overline{ı} we have that S is either invariant under i ↦ \overline{ı} in which case we say that it is an odd orbit or that \overline{S} is another orbit in which case we say \{S, \overline{S}\} is an even orbit pair.

2.5 Shuffles

Recall that a (p, q)-shuffle is a permutation σ of {1, 2, …, p + q} for which we have σ(i) < σ(j) whenever i < j ≤ p or p < i < j. It is clear that for each subset I of {1, 2, …, g} there is a unique (|I|, g − |I|)-shuffle σ^{I} such that I = {σ^{I}(1), σ^{I}(2), …, σ^{I}(|I|)} and we will call it the shuffle associated to I. We will use the same notation for the corresponding element in W_{g} (i.e., fulfilling σ^{I}(2g + 1 − i) = 2g + 1 − σ^{I}(i) for 1 ≤ i ≤ g). By doing the shuffling from above instead of from below we get another shuffle σ_{I} given by σ_{I}(i) = g + 1 − σ^{I}(g + 1 − i). We will use the same notation for its extension to W_{g}. Note that σ_{I} will shuffle the elements {g + 1, g + 2, …, 2g} in the same way that σ^{I} shuffles {1, 2, …, g}, i.e., σ^{I}(g + i) = g + σ_{I}(i), which is the relation with σ^{I} that motivates the definition. Note that if I = {i_{1} < ⋯ < i_{r}} and if we assume that i_{r} > r (if this does not hold not then σ^{I} and σ_{I} are the identity elements) and we let k be the smallest index such that i_{k} > k, then σ^{I} = s_{i_{k}−1}σ^{I′} and σ_{I} = s_{g+1−(i_{k}−1)}σ_{I′}, where I′ = {i_{1}, …, i_{k} − 1, …, i_{r}}. We call the element s_{i−1}ws_{g+1−(i−1)} for w ∈ W_{g} the i’th elementary shuffle of w, say that I′ is the elementary reduction of I whose reduction index is i_{k}.

We define the height of a shuffle associated to a subset {i_{1}, i_{2}, …, i_{k}} ⊆ {1, 2, …, g} to be ∑_{s}(i_{s} − s). Using w′ = s_{i}ws_{g+1−t} ⟺ s_{i}w′s_{g+1−t} = w, we see that starting with an element w obtained by applying a shuffle to a final element we arrive at a final element after ht(w) elementary shuffles.

Definition 2.18. Let Y be a final Young diagram of degree g. The shuffles of Y are the elements of W_{g} of the form σ^{I}w_{Y} σ_{I}^{−1} for I ⊆ {1, 2, …, g}.

If w ≤ w_{∅} we say that i with 1 ≤ i ≤ g is a semi-simple index for w if w(i) = g + i (note that as w ≤ w_{∅} we always have w(i) ≤ g + i). The set of semi-simple indices will be called the semi-simple index set and its cardinality the semi-simple rank. We say that w is semi-simply final if the semi-simple index set has the form [g − f + 1, g] (where then f is the semi-simple rank). This is equivalent to w having the form […, 2g − f + 1, 2g − f + 2, …, 2g]. If w = w_{Y}, Y a final Young diagram, then w is semi-simply final and the semisimple rank is equal to g minus the length of the largest row of Y (defined to be zero if Y is empty).

Proposition 2.19. Let w ≤ w_{∅} be a semi-simply final element of semisimple rank f and let I ⊆ {1, 2, …, 2g} be a subset with #I = f. Put Ĩ := {g + 1 − i : i ∈ I }. Then w′ := σ^{I}wσ_{I}^{−1} is an element with w′ ≤ w_{∅} of semi-simple rank f and semi-simple index set Ĩ. Conversely, all w′ ≤ w_{∅} whose semi-simple index set is equal to I are of this form.

Proof. Put j := σ_{I}^{−1}(i). Note that j > g − f ⟺ i ∈ Ĩ. If j > g − f we have w(j) = g + j and hence σ^{I}w_{Y} σ_{I}^{−1}(i) = σ^{I}(g + j) = g + σ_{I}(j) = g + i. If on the other hand j ≤ g − f, then if w_{Y} (j) ≤ g holds there is nothing to prove, and if it does not hold we may write w_{Y} (j) = g + k and as the semi-simple rank of Y is f we have k < j. Then σ^{I}w_{Y} σ_{I}^{−1}(i) = σ^{I}(g + j) = g + σ_{I}(k) and as k < j ≤ g − f we have σ_{I}(k) < σ_{I}(j) = i, which gives σ^{I}w_{Y} σ_{I}^{−1}(i) < g + i.

The converse is easy and left to the reader. ⊓ ⊔

Finally, we define the a-number of an element w ∈ W_{g} by the rule

a(w) := r_{w}(g, g).

If w is final with associated Young diagram Y then its a-number, also denoted a_{Y}, is the largest integer a with 0 ≤ a ≤ g such that Y contains the diagram that corresponds to the set ξ = {a, a − 1, a − 2, …, 1}.

3 The Flag Space

3.1 The Flag space of the Hodge bundle

In this section we introduce the flag space of a principally polarised abelian scheme over a base scheme of characteristic p. We use the Frobenius morphism to produce from a chosen flag on the de Rham cohomology a second flag, the position of which with respect to the first flag will be the object of study.

We let S be a scheme (or Deligne-Mumford stack) in characteristic p and let X → S be an abelian scheme over S with principal polarization (everything would go through using a polarization of degree prime to p but we shall stick to the principally polarised case). We consider the de Rham cohomology sheaf H_{dR}^{1}(X /S). It is defined as the hyper-direct image R^{1}π_{∗}(O_{X} → Ω_{X /S}^{1}) and is a locally free sheaf of rank 2g on S. The polarization (locally in the étale topology given by a relatively ample line bundle on X /S) provides us with a symmetric homomorphism ρ: X → X̂ and the Poincaré bundle defines a perfect pairing between H_{dR}^{1}(X /S) and H_{dR}^{1}(X̂/S) and thus H_{dR}(X /S) comes equipped with a non-degenerate alternating form (cf. [Oo95])

⟨ , ⟩: H_{dR}^{1}(X /S) × H_{dR}^{1}(X /S) → O_{S}.

Moreover, we have an exact sequence of locally free sheaves on S

0 → π_{∗}(Ω_{X /S}^{1}) → H_{dR}^{1}(X /S) → R^{1}π_{∗}O_{X} → 0.

We shall write ℍ for the sheaf H_{dR}^{1}(X /S) and 𝔼 for the Hodge bundle π_{∗}(Ω_{X /S}^{1}). We thus have an exact sequence

0 → 𝔼 → ℍ → 𝔼^{∨} → 0

of locally free sheaves on S. The relative Frobenius F: X → X^{(p)} and the Verschiebung V : X^{(p)} → X satisfy F · V = p · id_{X^{(p)}} and V · F = p · id_{X} and they induce maps, also denoted F resp. V, in cohomology:

F: ℍ^{(p)} → ℍ and V : ℍ → ℍ^{(p)}.

Of course, we have F V = 0 and V F = 0 and F and V are adjoints (with respect to the alternating form). This implies that Im(F) = ker(V) and Im(V) = ker(F) are maximally isotropic subbundles of ℍ and ℍ^{(p)} respectively. Moreover, since dF = 0 on Lie(X) it follows that F = 0 on 𝔼^{(p)} and thus Im(V) = ker(F) = 𝔼^{(p)}. Verschiebung thus provides us with a bundle map (again denoted by V): V : ℍ → 𝔼^{(p)}.

Consider the space F = Flag(ℍ) of symplectic flags on the bundle ℍ consisting of flags of subbundles {𝔼_{i}}_{i=1}^{2g} satisfying rk(𝔼_{i}) = i, 𝔼_{g+i} = 𝔼_{g−i}^{⊥}, and 𝔼_{g} = 𝔼. This space is a scheme over S and it is fibred by the spaces F^{(i)} of partial flags

𝔼_{i} ⊊ 𝔼_{i+1} ⊊ … ⊊ 𝔼_{g}.

So F = F^{(1)} = Flag(ℍ) and F^{(g)} = S and there are natural maps

π_{i,i+1} : F^{(i)} → F^{(i+1)},

the fibres of which are Grassmann varieties of dimension i. So the relative dimension of F is g(g − 1)/2. The space F^{(i)} is equipped with a universal partial flag. On F the Chern classes of the bundle 𝔼 decompose into their roots:

λ_{i} = σ_{i}(ℓ_{1}, …, ℓ_{g})  with  ℓ_{i} = c_{1}(𝔼_{i}/𝔼_{i−1}),

where σ_{i} is the i’th elementary symmetric function.

On F^{(i)} we have the Chern classes ℓ_{i+1}, …, ℓ_{g} and

λ_{j}(i) := c_{j}(𝔼_{i}) j = 0, 1, …, i.

Its Chow ring is generated over that of A_{g} by the monomials ℓ_{1}^{m_{1}} ⋯ ℓ_{g}^{m_{g}} with 0 ≤ m_{j} ≤ j − 1. For later use we record the following Gysin formula.

Formula 3.1. We have (π_{i,i+1})_{∗}ℓ_{i+1}^{k} = s_{k−i}(i + 1), where s_{j}(i + 1) denotes the j’th Segre class of 𝔼_{i+1} (j’th complete symmetric function in the Chern roots ℓ_{1}, …, ℓ_{i+1}).

Given an arbitrary flag of subbundles

0 = 𝔼_{0} ⊊ 𝔼_{1} ⊊ … ⊊ 𝔼_{g} = 𝔼

with rank(𝔼_{i}) = i we can extend this uniquely to a symplectic filtration on ℍ by putting

𝔼_{g+i} = (𝔼_{g−i})^{⊥}.

By base change we can transport this filtration to ℍ^{(p)}.

We introduce a second filtration by starting with the isotropic subbundle

𝔻_{g} := ker(V) = V^{−1}(0) ⊂ ℍ

and continuing with

𝔻_{g+i} = V^{−1}(𝔼_{i}^{(p)}).

We extend it to a symplectic filtration by setting 𝔻_{g−i} = (𝔻_{g+i})^{⊥}. We thus have two filtrations 𝔼_{•} and 𝔻_{•} on the pullback of ℍ to F.

We shall use the following notation

L_{i} = 𝔼_{i}/𝔼_{i−1} and M_{i} = 𝔻_{i}/𝔻_{i−1} for 1 ≤ i ≤ 2g.

For ease of reference we formulate a lemma which follows immediately from definitions.

Lemma 3.2. We have M_{g+i} ≅ L_{i}^{p}, L_{2g+1−i} ≅ L_{i}^{∨} and M_{2g+1−i} ≅ M_{i}^{∨}.

More generally, for a family X → S of principally polarised abelian varieties we shall say that a Hodge flag for the family is a complete symplectic flag {𝔼_{i}} of ℍ for which 𝔼_{g} is equal to the Hodge bundle. By construction this is the same thing as a section of F_{g} → S. We shall also call the associated flag {𝔻_{i}} the conjugate flag of the Hodge flag.

3.2 The canonical flag of an abelian variety

In this section we shall confirm that the canonical filtration of X[p] (kernel of multiplication by p) by subgroup schemes of a principally polarised abelian variety X as defined by Ekedahl and Oort [Oo01] has an analogue for de Rham cohomology. Just as in [Oo01] we do this in a family X → S. It is the coarsest flag which is isotropic (i.e., if 𝔻 is a member of the flag then so is 𝔻^{⊥}) and stable under F (i.e., if 𝔻 is a member of the flag then so is F (𝔻^{(p)})). The existence of such a minimal flag is proven by adding elements F^{⊥} and F (𝔻^{(p)}) for 𝔻 already in the flag in a controlled fashion. We start by adding 0 to the flag. We then insist on three rules:

1. If we added 𝔻 ⊆ 𝔻_{g}, then we immediately add 𝔻^{⊥} (unless it is already in the flag constructed so far).

2. If we added 𝔻_{g} ⊆ 𝔻, then we immediately add F (𝔻^{(p)}) (unless it is already in the flag constructed so far).

3. If neither rule (1) nor rule (2) applies, then we add F (𝔻^{(p)}) for the largest element 𝔻 of the flag for which F (𝔻^{(p)}) is not already in the flag.

We should not however do this construction on S; we want to insure that we get a filtration by vector bundles: At each stage when we want to add the image F (𝔻^{(p)}), we have maps F: 𝔻^{(p)} → ℍ of vector bundles and we then have a unique minimal decomposition of the base as a disjoint union of locally closed subschemes such that on each subscheme this map has constant rank; these are subschemes because they are given by degeneracy conditions. At the same time as we add F (𝔻^{(p)}) to the flag we replace the base by this disjoint union. Over this disjoint union F (𝔻^{(p)}) then becomes a subbundle of ℍ and whether or not it is equal to the one of the previously defined subbundles is a locally constant condition. A simple induction then shows that we get a flag, i.e., for any two elements constructed one is included in the other, on a disjoint union of subschemes of S. As each element added is either the image under F of an element previously constructed or the orthogonal of such an element, it is clear that this flag is the coarsening of any isotropic flag stable under F and it is equally clear that the decomposition of S is the coarsest possible decomposition. We shall call the (partial) flag obtained in this way the canonical flag of X /S and the decomposition of S the canonical decomposition of the base.

To each stratum S′ of the canonical decomposition of S we associate a canonical type as follows: Let T ⊆ {1, 2, …, 2g} be the set of ranks of the elements of the canonical flag and let f: T → T ∩ {1, …, g} be the function which to t associates rk(F (𝔻^{(p)})), where 𝔻 is the element of the canonical flag of rank t. We now claim that T and f fulfills the conditions of Lemma 2.11. Clearly T contains 0 and by construction it is invariant under i ↦ 2g − i. Again by construction f is increasing and has f(0) = 0 and f(2g) = g. Furthermore, if i, j ∈ T with i < j then F induces a surjective map (𝔻/𝔻′)^{(p)} → F (𝔻)/F (𝔻′), where 𝔻 resp. 𝔻′ are the elements of the canonical flag for which the rank is j resp. i and hence f(j) − f(i) = rk(F (𝔻)/F (𝔻′)) ≤ rk(𝔻/𝔻′) = j − i. Finally, assume that f(j) − f(i) = j − i and let 𝔻 and 𝔻′ be as before. Putting 𝔻_{1} := F (𝔻^{(p)}) and 𝔻′_{1} := F (𝔻^{′(p)}) these are also elements of the canonical filtration and we by assumption F induces an isomorphism F: (𝔻/𝔻′)^{(p)} → 𝔻_{1}/𝔻′_{1}. The fact that it is injective means that 𝔻 ∩ ker F = 𝔻′ ∩ ker F which by taking annihilators, and using that ker F is its own annihilator, gives 𝔻^{⊥} +ker F = 𝔻^{′⊥} +ker F. In turn, this implies F (𝔻^{⊥}) = (𝔻^{′⊥}) and hence that f(2g−i) = f(2g−j). Now, if f is not surjective then by Lemma 2.11 there is a proper subset of T fulfilling the conditions of the Lemma 2.11. This is not possible as T by construction is a minimal subset with these conditions. Hence T and f fulfils the conditions of Lemma 2.11 and hence by them we get that (f, T) is a canonical type. Let ν be its associated final type. If 0 = 𝔻_{0} ⊂ 𝔻_{1} ⊂ ⋯ ⊂ 𝔻_{2g} = ℍ is the canonical flag with rk 𝔻_{i} = i we have also proved that F induces an isomorphism (𝔻_{j}/𝔻_{i})^{(p)} → 𝔻_{ν(j)}/𝔻_{ν(i)} which can be rephrased as an isomorphism

F: 𝔻_{v(I)}^{(p)} ⟶ 𝔻_{I},

where we have used the notation 𝔻_{J} := 𝔻_{i}/𝔻_{j} for an interval J =]j, i] and v ∈ W_{g} is the complementary element of (the final element of) ν. We shall say that ν (or more properly f) is the canonical type of the principally polarised abelian variety X_{S′} → S′. (We could consider the canonical type as a locally constant function on the canonical decomposition to the set of canonical (final) types.)

Remark 3.3. Note that the canonical flag is a flag containing 𝔻_{g} and is not defined in terms of 𝔼_{g}. That will later mean that the canonical flag will be a coarsening of a conjugate flag which is derived from a Hodge flag. On the one hand this is to be expected. As the canonical flag is just that; it will be constructed in a canonical fashion from the family of principally polarised abelian varieties. Hence it is to be expected, and it is clearly true, that the canonical flag is horizontal with respect to the Gauss-Manin connection. On the other hand we do not want to just consider conjugate flags (or make constructions starting only with conjugate flags). The reason is essentially the same; as 𝔻_{g} (or more generally the elements of the canonical flag) is horizontal it will not reflect first order deformations whereas 𝔼_{g} isn’t and does. This will turn out to be of crucial importance to us and is the reason why the Hodge flags will be the primary objects while the conjugate flags are secondary. On the other hand, when working pointwise, over an algebraically closed field say, we may recover the Hodge flag from the conjugate flag and then it is usually more convenient to work with the conjugate flag.

Example 3.4. Let X be an abelian variety with p-rank f and a(X) = 1 (equivalently, on 𝔼_{g} the operator V has rank g − 1 and semi-simple rank g − f). Then the canonical type is given by the numbers {rk(𝔻_{i})}, i.e.,

{0, f, f + 1, …, 2g − f − 1, 2g − f, 2g}

and ν is given by ν(f) = f, ν(f+1) = f, ν(f+2) = f+1, …, ν(g) = g−1, …, ν(2g−f −1) = g−1, ν(2g−f) = g and ν(2g) = g. The corresponding element w ∈ W_{g} is [f + 1, g + 1, …, 2g − f − 1, 2g − f + 1, …, 2g].

4 Strata on the Flag Space

4.1 The Stratification

The respective position of two symplectic flags are encoded by a combinatorial datum, an element of a Weyl group. We shall now define strata on the flag space F over the base S of a principally polarised abelian scheme X → S that mark the respective position of the two filtrations 𝔼_{•} and 𝔻_{•} that we have on the de Rham bundle over F.

Intuitively, the stratum \overline{U}_{w} is defined as the locus of points x such that at x we have

dim(𝔼_{i} ∩ 𝔻_{j}) ≥ r_{w}(i, j) = #{a ≤ i: w(a) ≤ j} for all 1 ≤ i, j ≤ 2g.

A more precise definition would be as degeneracy loci for some appropriate bundle maps. While this definition would work fine in our situation where we are dealing with flag spaces for the symplectic group it would not quite work when the symplectic group is replaced by the orthogonal group on an even-dimensional space (cf., [FP98]). With a view on future extensions of the ideas of this paper to other situations we therefore adopt the definition that would work in general. Hence assume that we have a semi-simple group G, a Borel group B of it, a G/B-bundle T → Y (with G as structure group) over some scheme Y, and two sections s, t: Y → T of it. Then for any element w of the Weyl group of G we define a (locally) closed subscheme U_{w} resp. \overline{U}_{w} of Y in the following way. We choose locally (possibly in the étale topology) a trivialization of T for which t is a constant section. Then s corresponds to a map Y → G/B and we let U_{w} (resp. \overline{U}_{w}) be the inverse image of the B-orbit BwB (resp. of its closure in G/B). Another trivialization will differ by a map Y → B and as BwB and its closure are B-invariant these definitions are independent of the chosen trivializations and hence give global subschemes on Y. If s and t have the property that Y = U_{w}, then we shall say that s and t are in relative position w and if Y = \overline{U}_{w} we shall say that s and t are in relative position ≤ w.

Remark 4.1. The notation is somewhat misleading as it suggests that \overline{U}_{w} is the closure of U_{w} which may not be the case in general. In the situation that we shall meet it will however be the case (cf., Corollary 8.4).

The situation to which we will apply this construction is when the base scheme is the space F of symplectic flags 𝔼_{•} as above, s is the tautological section of the flag space of ℍ over F, and t is the section given by the conjugate flag 𝔻_{•}. From now on we shall, unless otherwise mentioned, let U_{w} and \overline{U}_{w} denote the subschemes of F coming from the given s and t and w ∈ W_{g}. In this case it is actually often more convenient to use the language of flags rather than sections of G/B-bundles and we shall do so without further mention. We shall also say that a Hodge flag 𝔼_{•} is of stamp w resp. stamp ≤ w if 𝔼_{•} and its conjugate flag 𝔻_{•} are in relative position w resp. ≤ w.

Lemma 4.2. Over U_{w} we have an isomorphism L_{i} ≅ M_{w(i)} for all 1 ≤ i ≤ 2g.

Proof. By the definition of the strata we have that the image of 𝔼_{i} ∩ 𝔻_{w(i)} has rank one greater than the ranks of 𝔼_{i−1} ∩ 𝔻_{w(i)}, 𝔼_{i} ∩ 𝔻_{w(i)−1}, and 𝔼_{i−1} ∩ 𝔻_{w(i)−1}. So the maps 𝔼_{i}/𝔼_{i−1} ← (𝔼_{i} ∩ 𝔻_{w(i)})/(𝔼_{i−1} ∩ 𝔻_{w(i)−1}) → 𝔻_{w(i)}/𝔻_{w(i)−1} give the isomorphism. ⊓ ⊔

When the base of the principally polarised abelian scheme is A_{g} we shall use the notation F_{g} for the space of Hodge flags. Note that a Hodge flag with respect to X → S is the same thing as a lifting over F_{g} → A_{g} of the classifying map S → A_{g}. The conjugate flag as well as the strata U_{g} and \overline{U}_{g} on S are then the pullbacks of the conjugate flag resp. the strata on F_{g}.

4.2 Some Important Strata

We now give an interpretation for some of the most important strata. To begin with, if one thinks in terms instead of filtrations of X[p] by subgroup schemes it becomes clear that the condition F (𝔻_{i}^{(p)}) ⊆ 𝔻_{i} should be of interest. It can almost be characterised in terms of the strata \overline{U}_{w}.

Proposition 4.3. Let X → S be a family of principally polarised abelian varieties and 𝔼_{•} a Hodge flag such that the flag is of stamp ≤ w and w is the smallest element with that property.

i) For j ≤ g and for all i ∈ [1, …, g] we have r_{w}(i, g + j) = i if and only if V (𝔼_{i}) ⊆ 𝔼_{j}^{(p)}.

ii) For j ≤ g we have that r_{w}(g + j, i) = i implies that F (𝔻_{i}^{(p)}) ⊆ 𝔻_{j} and the converse is true if S is reduced.

iii) We have that V (𝔼_{i}) ⊆ 𝔼_{i}^{(p)} for all i precisely when w ≤ w_{∅}. If S is reduced F (𝔻_{i}^{(p)}) ⊆ 𝔻_{i} for all i precisely when w ≤ w_{∅}.

Proof. We have that V (𝔼_{i}) ⊆ 𝔼_{j}^{(p)} if and only if 𝔼_{i} ⊆ V^{−1}(𝔼_{j}^{(p)}) = 𝔻_{g+j}. On the other hand, by definition rk 𝔼_{i} ∩ 𝔻_{g+j} ≤ r_{w}(i, g + j) with equality for at least one point of S. As rk 𝔼_{i} ∩ 𝔻_{g+j} = i ⟺ 𝔼_{i} ⊆ 𝔻_{g+j} we get the first part.

For the second part we start by claiming that 𝔼_{i}^{(p)} ⊆ 𝔻_{j} is implied by F (𝔻_{i}^{(p)}) ⊆ 𝔻_{j}. Indeed, F (𝔻_{i}^{(p)}) ⊆ 𝔻_{j} is equivalent to F (𝔻_{i}^{(p)}) being orthogonal to 𝔻_{2g−j}, i.e., to the condition that for u ∈ 𝔻_{i}^{(p)}) and v ∈ 𝔻_{2g−j} we have ⟨F u, v⟩ = 0. This implies that 0 = ⟨F u, v⟩ = ⟨u, V v⟩^{p} and hence ⟨u, V v⟩ = 0 as S is reduced which means that 𝔻_{i}^{(p)} ⊆ (V (𝔻_{2g−j}))^{⊥} = (𝔼_{g−j}^{(p)})^{⊥} = 𝔼_{g+j}^{(p)}. As S is reduced this implies that 𝔻_{i} ⊆ 𝔼_{g+j} and this in turn is equivalent to r_{w}(g + j, i) = i. The argument can be reversed (and then it does not require S to be reduced).

Finally, we have from the first part that V (𝔼_{i}) ⊆ 𝔼_{i}^{(p)} for all i ≤ g precisely when r_{w}(i, g + i) = i for all i ≤ g but by induction on i that is easily seen to be equivalent to w(i) ≤ g + i for all i ≤ g which by definition means that w ≤ w_{∅}. As 𝔼_{g}^{(p)} = V (ℍ) the condition V (𝔼_{i}) ⊆ 𝔼_{i}^{(p)} for i > g is trivially fulfilled.

The proof of the second equivalence is analogous in that using ii), the condition that F (𝔻_{i}^{(p)}) ⊆ 𝔻_{i} is equivalent to r_{w}(g + i, i) = i. In general, r_{u}(i, j) = r_{u^{−1}}(j, i) so that this condition is equivalent to r_{w^{−1}}(i, g+i) = i and hence by the same argument as before this condition for all i is equivalent to w^{−1} ≤ w_{∅}. Chevalley’s characterisation of the Bruhat-Chevalley order makes it clear that u ≤ v ⟺ u^{−1} ≤ v^{−1} and hence we get w^{−1} ≤ w_{∅} ⟺ w ≤ w_{∅}^{−1}. However, w_{∅} is an involution. ⊓ ⊔

Remark 4.4. i) As we shall see (cf., Corollary 8.4) the strata \overline{U}_{w} in the universal case of F_{g} are reduced.

ii) Flags of stamp w ≤ w_{∅} are called admissible.

We can also show that the relations between final and canonical types are reflected in relations for flags. We say that a Hodge flag is a final flag if it is of stamp w for a final element w. Also if I =]i, j] ⊆ {1, 2, …, 2g} is an interval and 𝔽_{•} is a complete flag of a vector bundle of rank 2g then we define 𝔽_{I} to be 𝔽_{j}/𝔽_{i}.

Proposition 4.5. Let X → S be a principally polarised abelian scheme over S and let 𝔼_{•} be a final flag for it of stamp w.

i) The conjugate flag 𝔻_{•} is a refinement of the canonical flag. In particular w is determined by X → S. More directly, we have that the final type ν associated to w is given by

rk(𝔼_{g} ∩ 𝔻_{i}) = i − ν(i)

for all i. In particular, the canonical decomposition of S with respect to X → S consists of a single stratum and its canonical type is the canonical type associated to w.

ii) Conversely, assume that S is reduced and that the canonical decomposition of S consists of a single stratum, and let ν be the final type associated to the canonical type of the canonical flag. Then any Hodge flag 𝔼_{•} whose conjugate flag 𝔻_{•} is a refinement of the canonical flag and for which we have F (𝔻_{i}^{(p)}) ⊆ 𝔻_{ν(i)} for all i, is a final flag.

ii) If I is a canonical fragment for v, the complementary element to w, then F induces a bijection (𝔻_{v(I)})^{(p)} ⟶ 𝔻_{I}.

Proof. We start by showing that F (𝔻_{i}) = 𝔻_{ν(i)} for all i. Indeed, this is equivalent to F (𝔻_{i}) ⊆ 𝔻_{ν(i)} and rk((ker(F) = 𝔼_{g}^{(p)}) ∩ 𝔻_{i}^{(p)})= i − ν(i) as the second condition says that F (𝔻_{i}) has rank ν(i). Now, the condition F (𝔻_{i}) ⊆ 𝔻_{ν(i)} is by Proposition 4.3 implied by r_{w}(g + ν(i), i) = ν(i) which is true for a final element by Corollary 2.10. On the other hand, the condition rk(𝔼_{g}^{(p)} ∩ 𝔻_{i}^{(p)}) = i − ν(i) is implied by r_{w}(g, i) = rk(𝔼_{g} ∩ 𝔻_{i}) = i − ν(i) which is true by the very definition of ν. Now, the fact that F (𝔻_{i}) = 𝔻_{j} for all i and some j (depending on i) implies by induction on the steps of the construction of the canonical flag that 𝔻 is a refinement of the canonical flag. The rest of the first part then follows from what we have proved.

As for ii), assume first that 𝔼_{•} has a fixed stamp w′ and let ν′ be its final type. As 𝔻_{•} is an extension of the Hodge flag we get that when i is in the canonical domain of ν that i − ν′(i) = rk(𝔼_{g} ∩ 𝔻_{i}) = i − ν(i) so that ν and ν′ coincide on the canonical domain of ν and hence they coincide by Lemma 2.11. The assumption that F (𝔻_{i}^{(p)}) ⊆ 𝔻_{ν(i)} = 𝔻_{ν′(i)} for all i is by Proposition 4.3 equivalent to r_{w}(g + ν′(i), i) = i for all i and hence Corollary 2.10 gives that w is final of type ν′ = ν.

Finally, assume that I =]i, j]. The induced map 𝔻_{j}^{(p)}/𝔻_{i}^{(p)} → 𝔻_{ν(j)}/𝔻_{ν(i)} is always surjective but it follows from Lemma 2.11 that either the right hand side of this map has rank 0 or it has the same rank as the right hand side. If they have the same rank it induces an isomorphism 𝔻_{I}^{(p)} ⟶ 𝔻_{v^{−1}(I)}. If the right hand side has rank zero, then again from Lemma 2.11 the two sides of 𝔻(p) ; \overline{ȷ}/𝔻(p) ; \overline{ı} → 𝔻_{ν(\overline{ȷ})}/𝔻_{ν(\overline{ı})} have the same rank and hence this map is an

isomorphism and again gives an isomorphism 𝔻_{I}^{(p)} ⟶ 𝔻_{v^{−1}(I)}. As I is an arbitrary fragment we conclude the proof of ii). ⊓ ⊔

The number of final extensions of a canonical flag will now be expressed in the familiar terms of the number of flags (resp. self-dual flags) in a vector space over a finite field (resp. a vector space with a unitary form). Hence we let γ_{n}^{e}(m) be the number of complete 𝔽_{p^{m}}-flags in 𝔽_{p^{m}}^{n} and let γ_{n}^{o}(m) be the number of complete 𝔽_{p^{2m}}-flags self-dual under the unitary form ⟨(u_{1}, …, u_{n}), (v_{1}, …, v_{n})⟩ := u_{1}v_{1}^{p^{m}} + ⋯ + u_{n}v_{n}^{p^{m}}. Recall now the definition of even and odd orbits of fragments given in Definition 2.17.

Lemma 4.6. Let X be a principally polarised abelian variety over an algebraically closed field and w ∈ W_{g} the element whose canonical type is the canonical type of X. Put

γ(w) = γ_{g}(w) := ∏_{S=\overline{S}} γ_{#I}^{o}(#S/2) ∏_{\{S,\overline{S}\}} γ_{#I}^{e}(#S),

where the first product runs over the odd orbits and the second over the even orbit pairs and in both cases I indicates a member of S.

The number of final flags for X is then equal to γ(w).

Proof. As we are over a perfect field, any symplectic flag extending 𝔻_{g} is the conjugate flag of a unique Hodge flag. Hence we get from Proposition 4.5 that a final flag is the same thing as a flag 𝔻_{•} extending the canonical flag and for which we have F (𝔻_{i}^{(p)}) ⊆ 𝔻_{ν(i)} for all i. The condition that dim(𝔼_{g} ∩ 𝔻_{i}) = i − v(i) then gives that we actually have F (𝔻_{i}^{(p)}) = 𝔻_{ν(i)}. However, as 𝔻_{•} refines the canonical flag it is determined by the induced flags of the 𝔻_{I} for fragments I of v and the stability condition F (𝔻_{i}^{(p)}) = 𝔻_{ν(i)} transfers into a stability condition under the isomorphisms F: 𝔻_{v(I)}^{(p)} ⟶ 𝔻_{I} of Subsection 3.2.

Hence the problem splits up into separate problems for each orbit under v and i ↦ \overline{ı} (the map i ↦ \overline{ı} transfers into the isomorphism 𝔻\overline{I} ≅ 𝔻_{I}^{∨} induced by the symplectic form). Given any fragment I ∈ S, S a v-orbit of fragments of v, the flag of 𝔻_{v^{k}(I)} for any k is then determined by the flag corresponding to I by the condition of F^{k}-stability takes the k’th Frobenius pullback of the flag on 𝔻_{v^{k}(I)} to the one of 𝔻_{I}. Furthermore, the flag on 𝔻_{I} then has to satisfy the consistency condition of being stable under F_{S} := F^{#S}.

For an even orbit pair \{S, \overline{S}\} the self-duality requirement for the flag means that the flags for the elements of \overline{S} are determined by those for the elements of S. Moreover, given I ∈ S there is no other constraint on the flag on 𝔻_{I} than that it is stable under F_{S}. Hence we have the situation of a vector bundle D over our base field k and an isomorphism F_{S}: D^{(p^{m})} ⟶ D, where m = #S, and we want to count the number of flags stable under F_{S}. Now, as k is algebraically closed, D_{p} := {v ∈ D : F_{S}(v) = v } is an 𝔽_{p^{m}}-vector space for which the inclusion map induces an isomorphism k ⨂_{𝔽_{p^{m}}} D_{p} ⟶ D. It then follows that F_{S}-stable flags correspond to 𝔽_{p^{m}}-flags of D_{p}.

If instead S is an odd orbit we will together with I also have \overline{I} in S. If we denote the cardinality of S by 2m, the flag on 𝔻_{I} must be mapped to the dual flag on 𝔻\overline{I} by F_{S} := F^{m}. The situation is similar to the even orbit pair situation but with a “unitary twist” as in Proposition 7.2 and we get instead a correspondence with self-dual flags. We leave the details to the reader. ⊓ ⊔

Example 4.7. For the canonical type associated to the final type of an element w of example 3.4 we have

γ_{g}(w) = (p + 1)(p^{2} + p + 1) ⋯ (p^{f−1} + p^{f−2} + … + 1).

Definition 4.8. For 1 ≤ f < g we let w = u_{f} be the final element

u_{f} = s_{g}s_{g−1}s_{g} ⋯ s_{g−f+1} ⋯ s_{g}s_{g−f−1} ⋯ s_{g} ⋯ s_{1} ⋯ s_{g},

i.e., if we introduce τ_{j} = s_{j}s_{j+1} ⋯ s_{g} then we have u_{f} = τ_{g}τ_{g−1} ⋯ τ̂_{f} ⋯ τ_{1}. It corresponds to the Young diagram consisting of one row with g − f blocks and equals the element w given in 3.4.

Recall the notion of a-number a(X) for a g-dimensional abelian variety X of an (algebraically closed) field k of characteristic p. It equals the dimension over k of the vector space Hom(α_{p}, X) of maps of the group scheme α_{p} to X. Equivalently, a(X) equals the dimension of the kernel of V on H^{0}(X, Ω_{X}^{1}). In our terms H^{0}(X, Ω_{X}^{1}) = 𝔼_{g} and ker V = 𝔻_{g} so that a(X) = dim 𝔼_{g} ∩ 𝔻_{g}. The p-rank or semi-simple rank f on the other hand can be characterised by the condition that the dim_{k} ∩_{i≤g}(V^{i}ℍ)^{(p^{g−i})} is equal to f.

Lemma 4.9. i) Let Y be a final Young diagram and w ∈ W_{g} its final element and assume that x = (X, 𝔼_{•}, 𝔻_{•}) ∈ U_{w}(k), k a field. Then the p-rank of X equals the p-rank of Y.

ii) Let w = u_{f} (so that f < g) and x = (X, 𝔼_{•}, 𝔻_{•}) ∈ F_{g}(k). Then we have x ∈ U_{w} (resp. \overline{U}_{w}) if and only if the filtration 𝔼_{•} is V -stable and the p-rank of X is f and the a-number of X is 1 (resp. the p-rank of X is ≤ f). The image of U_{w} in A_{g} is the locus of abelian varieties of p-rank f and a-number 1.

Proof. By definition we have r_{w}(i, g + i) = i for all 1 ≤ i ≤ g. Moreover, r_{w}(i, g + i − 1) = i precisely when i ≤ g − f. Hence by Proposition 4.3 we see V (𝔼_{i}) ⊆ 𝔼_{i}^{(p)} for all 1 ≤ i ≤ g, V (𝔼_{i}) ⊆ 𝔼_{i−1}^{(p)} for i ≤ g − f and V (𝔼_{i}) ⊊ 𝔼_{i−1}^{(p)} for i > g −f. The first and last conditions mean that V induces an isomorphism V : 𝔼_{{i}} ⟶ 𝔼_{{i}}. On the other hand, the second condition gives V^{g−f}(𝔼_{g−f}) = 0. Together this gives that the semi-simple rank of X is f.

For the second part, since we have w ≤ w_{∅} we must check the condition on the p-rank and the a-number. By the definition on U_{w} we have x ∈ U_{w} if and only if rk(𝔼_{g} ∩ 𝔻_{j}) = 1 for f + 1 ≤ j ≤ g and rk(𝔼_{g} ∩ 𝔻_{j}) = 0 for 1 ≤ j ≤ f. This implies that the kernel of V (which equals 𝔻_{g} ∩ 𝔼_{g}) has rank 1 and the semi-simple rank of V on 𝔼_{g} is f. For x ∈ \overline{U}_{w} we get instead rk(𝔼_{g} ∩ 𝔻_{j}) ≤ 1 for f + 1 ≤ j ≤ g and rk(𝔼_{g} ∩ 𝔻_{j}) = 0 for 1 ≤ j ≤ f. ⊓ ⊔

Also the strata U_{w} with w ∈ S_{g} admit a relatively simple interpretation. Recall that an abelian variety is called superspecial if its a-number is equal to its dimension. This happens if and only if the abelian variety (without polarization) is geometrically isomorphic to a product of supersingular elliptic curves.

Lemma 4.10. Let x be a geometric point of F_{g} lying over [X] ∈ A_{g}. The following four statements are equivalent:

1. x ∈ ∪_{w∈S_{g}}U_{w}.

2. dim(𝔼_{g} ∩ 𝔻_{g}) ≥ g.

3. ker(V) = 𝔼_{g}.

4. The underlying abelian variety X is superspecial.

Proof. An abelian variety X is superspecial if and only if X is a product of supersingular elliptic curves and this condition is equivalent to dim(𝔼_{g}∩𝔻_{g}) ≥ g. This explains the equivalences of (2), (3) and (4). If x ∈ U_{w} with w ∈ S_{g} then r_{w}(g, g) = g, hence (2) holds. Conversely, if X is superspecial then any filtration 𝔼_{•} on 𝔼_{g} is V -stable and can be extended to a symplectic filtration. The lemma now follows from the observation that the degeneracy strata for w ∈ S_{g}, the Weyl group of GL_{g}, cover the flag space of flags on 𝔼_{g}. ⊓ ⊔

Lemma 4.11. Let x be a point of U_{w} with underlying abelian variety X. Then the a-number of X equals a(w). Moreover, if Y = {1, 2, …, a} with corresponding final element w_{Y} ∈ W_{g} then the image of U_{w_{Y}} in A_{g} is the locus T_{a} of abelian varieties with a-number a.

Proof. The a-number of an abelian variety is by definition the dimension of the kernel of V on H^{0}(X, Ω_{X}^{1}). But this is equal to r_{w}(g, g) = a(w). The condition that a(X) = a implies that r_{w}(g, g) = a, hence ν(g) = g − a. This implies that ν(g − a + i) ≥ i for i = 1, …, a. Therefore the ‘smallest’ ν satisfying these conditions is ν_{w_{Y}}. ⊓ ⊔

4.3 Shuffling flags

Our first result on the stratification will concern the case when the p-rank is positive. All in all the étale and multiplicative parts of the kernel of multiplication by p on the abelian variety have very little effect on the space of flags on its de Rham cohomology. There is however one exception to this. The most natural thing to do is to put the multiplicative part at the bottom (i.e., the first steps of the flag, and thus, by self-duality, the étale part at the top), which is what automatically happens for a final filtration (on the conjugate filtration, that is). We may however start with a final filtration and then “move” the μ_{p}-factors up-wards. Note, that over a perfect field the kernel of multiplication by p is the direct sum of its multiplicative, local-local, and étale parts so that this is always possible. In general however it is possible only after a purely inseparable extension. This means that we get an inseparable map from a stratum where not all the μ_{p}-factors are at the bottom to a stratum where they all are. We intend to first give a combinatorial description of the strata that can be obtained in this way from a final stratum and then to compute the degrees of the inseparable maps involved. However, as we have to compute an inseparable degree we should work with Hodge filtrations instead of conjugate filtrations as conjugate filtrations kill some infinitesimal information. This causes a slight conceptual problem as the V -simple parts in a final filtration are to be found “in the middle” rather than at the top and bottom (recall that V maps the top part of the conjugate filtration to the bottom of the Hodge filtration). This will not be a technical problem but the reader will probably be helped by keeping it in mind.

It turns out that the arguments used do not change if instead of considering shuffles of final elements we consider shuffles of semi-simply final elements. We shall treat the more general case as we shall need it later.

Hence we pick a subset Ĩ ⊆ \{1, 2, …, g\} and let \overline{U} ss ; Ĩ be the closed subscheme of F_{g} defined by the conditions that V maps 𝔼_{i}^{(p)} to 𝔼_{i}^{(p)} for all 1 ≤ i ≤ g and to 𝔼_{i−1}^{(p)} for i ∉ Ĩ. Hence \overline{U}_{w} ⊆ \overline{U} ss ; Ĩ precisely when w ≤ w_{∅} and the semi-simple index set of w is a subset of Ĩ. We also put

U ss ; Ĩ := \overline{U} ss ; Ĩ \ ∪_{Ĩ′⊂Ĩ}\overline{U} ss ; Ĩ′

so that U_{w} ⊆ U ss ; Ĩ precisely when w ≤ w_{∅} and its semi-simple index set is equal to Ĩ. If I := {g + 1 − i : i ∈ Ĩ} we get from Proposition 2.19 that these w are precisely those of the form σ^{I}w′σ_{I}^{−1} for the semi-simply final w′.

We are now going to construct, for every I ⊆ {1, 2, …, g}, a morphism

S_{I}: U ss ; Ĩ → U_{{g−f+1,...,g}}^{ss}, where Ĩ:= {g + 1 − i : i ∈ I } and #Ĩ = f.

Let ĩ be the reduction index of the elementary reduction I′ of I and put i := g + 1 − ĩ. By Proposition 2.19 we have that r_{w}(i + 1, g + i) = i + 1 and w(i) = g + i. This means that if 𝔼_{•} is the (tautological) Hodge flag on U ss ; Ĩ,

then V (𝔼_{i+1}) ⊆ 𝔼_{i}^{(p)} and V (𝔼_{i}) ⊊ 𝔼_{i−1}^{(p)} everywhere on U ss ; Ĩ. This means that V induces a bijection on 𝔼_{{i}} and is zero on 𝔼_{{i+1}}. Let Id denote the map s ↦ 1 ⊗ s from T to T^{(p)} for the sheaves involved. Then the map induced by the quotient map

Ker(Id −V)_{𝔼_{{i+1,i}}} → Ker(Id −V)_{𝔼_{{i}}},

where the kernel is computed in the étale topology on U_{w}, is an isomorphism. Also for a sheaf T equipped with a linear isomorphism V : T → T^{(p)}, we have an isomorphism Ker(V − I) ⨂ O → T with O the structure sheaf. It follows that the short exact sequence

0 → 𝔼_{{i}} ⟶ 𝔼_{{i+1,i}} ⟶ 𝔼_{{i+1}} → 0

splits uniquely in a way compatible with V. This means that we may define a new flag where 𝔼′_{j} = 𝔼_{j} for j ≠ i and 𝔼′_{i}/𝔼′_{i−1} = 𝔼_{i+1}/𝔼_{i}. Then the classifying map U ss ; Ĩ → F_{g} for this new flag will have its image in U ss ; Ĩ′. Repeating this process we end up with a flag whose classifying map will have its image in U_{{g−f+1,...,g}} which by definition is our map S_{I}.

Proposition 4.12. If I ⊆ {1, …, g} then the map S_{I}: U ss ; Ĩ → U_{{g−f+1,...,g}}^{ss}

is finite, radicial, and surjective.

Proof. To get from a point of U_{{g−f+1,...,g}}^{ss} to one of U ss ; Ĩ one has to find a V -invariant complement to some 𝔼_{i}/𝔼_{i−1} in 𝔼_{i+1}/𝔼_{i−1}. As V will be zero on 𝔼_{i+1}/𝔼_{i} and bijective on 𝔼_{i}/𝔼_{i−1}, a complement over the fraction field of a discrete valuation ring will extend to a complement over the discrete valuation ring (as the complement can not meet 𝔼_{i}/𝔼_{i−1} over the special fibre) so that the map is proper. It then remains to show that the map is a bijection over an algebraically closed field. In that case 𝔼_{g} splits canonically as a sum of a V -nilpotent part and a V -semisimple part and the bijectivity is clear. ⊓ ⊔

In order to determine the degree (necessarily of inseparability) we shall do the same factorisation as in the definition of S_{I} so that we may consider the situation of Ĩ with ĩ and I′ being the reduction index resp. elementary reduction of I. For the tautological flag 𝔼_{•} on U ss ; Ĩ we have that V is an isomorphism on 𝔼_{{i}} and zero on 𝔼_{{i+1}} while the opposite is true on U ss ; Ĩ′.

Lemma 4.13. The map U ss ; Ĩ → U ss ; Ĩ′ is flat of degree p.

Proof. We consider the partial symplectic flag space F_{g}(i) consisting of the flags of F_{g} by removing the i’th member 𝔻_{i} and its annihilator. This means that we have a ℙ^{1}-bundle F_{g} → F_{g}(i). Now, under this map U ss ; Ĩ and U ss ; Ĩ′ map to the same subscheme U ⊆ F_{g}(i) and the map U ss ; Ĩ → U ss ; Ĩ′ is compatible with these projections. Over U put E := 𝔼_{{i,i+1}}, M := ker(V : E → E^{(p)}), and L := Im(V : E → E^{(p)}). Then on the ℙ^{1}-bundle π: F_{g} → F_{g}(i), the subscheme U ss ; Ĩ′ is defined by the vanishing of the composite map O(−1) → π^{∗}E → π^{∗}(E/M) and in fact gives a section of F_{g} over U given by the line sub-bundle M ⊂ E. Hence it is enough to show that the projection map U ss ; Ĩ → F_{g}(i) is flat of degree p. We have that U ss ; Ĩ′ ⊆ F_{g} is defined by the vanishing of the composite O(−1)^{(p)} → π^{∗}E^{(p)} → π^{∗}(E^{(p)}/M). It is then enough to show that U ss ; Ĩ′ ⊆ F_{g} is a relative Cartier divisor and for that it is enough to show that it is a proper subset in each fibre of F_{g} → F_{g}(i). This however is clear as for a geometric point of F_{g}(i) there are just two points that lie in \overline{U}_{∅}, given by M and L^{p^{−1}}.

⊓ ⊔

Composing these maps we get the following proposition.

Proposition 4.14. Let I ⊆ {1, …, g} and Ĩ:= {g + 1 − i : i ∈ I }. Then the map S_{I}: U_{I}^{ss} → U_{{1,...,g}}^{ss} is a finite purely inseparable map of degree p^{ht(I)}.

Proof. The flatness and the degree of S_{I} follows by factoring it by maps as in Lemma 4.13 and noting that the number of maps is ht(I). The rest then follows from Proposition 4.12. ⊓ ⊔

Remark 4.15. The result implies in particular that if w′ is a shuffle of w by I, then S_{I}: U_{w′} → U_{w} is flat and purely inseparable of degree p^{ht(I)}. We shall later (see Corollary 8.4) show that U_{w′} and U_{w} are reduced. This shows that over a generic point of U_{w} each simple shuffle towards w′ really requires a finite inseparable extension of degree p. This is a kind of non-degeneracy statement which is the inseparable analogue of maximal monodromy (of which we shall see some examples later on). It can also be seen as saying that a certain Kodaira-Spencer map is injective.

4.4 The E-O strata on A_{g} ⊗ 𝔽_{p}

Definition 4.16. Let w ∈ W_{g} be a final type. Then the E-O stratum V_{w} associated to w is the locally closed subset of A_{g} of points x for which the canonical type of the underlying abelian variety is equal to the canonical type of w. We let \overline{V}_{w} be the closure of V_{w}.

It is known that the dimension of V_{w} is equal to dim(w), [Oo01]. This and the fact that the E-O strata form a stratification will also follow from our results in 8 and 9.3.

5 Extension to the boundary

The moduli space A_{g} admits several compactifications. The Satake or Baily-Borel compactification A_{g}^{∗} is in some sense minimal, cf. [FC90], Ch. V. It is a stratified space

A_{g}^{∗} = ⨆_{i=0}^{g} A_{i}.

Chai and Faltings define in [FC90] a class of smooth toroidal compactifications of A_{g}. If Ã_{g} is such a toroidal compactification then there is a natural map q: Ã_{g} → A_{g}^{∗} extending the identity on A_{g}. This induces a stratification of Ã_{g}:

Ã_{g} = ⨆_{i=0}^{g} q^{−1}(A_{g−i}) = ⨆_{i=0}^{g} A_{g}^{⟨i⟩}.

The stratum A_{g}^{⟨i⟩} parametrises the semi-abelian varieties of torus rank i.

The Hodge bundle 𝔼 on A_{g} can be extended to a rank g vector bundle, again denoted by 𝔼, on Ã_{g}. On A_{g}^{⟨i⟩} the Hodge bundle fits into a short exact sequence

0 → 𝔼′ → 𝔼 → 𝔼′′ → 0,

where the 𝔼′ is a rank g − i bundle and 𝔼′′ can be identified with cotangent bundle to the toric part of the universal semi-abelian variety along the identity section over A_{g}^{⟨i⟩}. The bundle 𝔼′ is the pull back under q: A_{g}^{⟨i⟩} → A_{g}^{∗} of the Hodge bundle on A_{g−i}.

The Verschiebung V acts in a natural way on the above short exact sequence and it preserves 𝔼′. It induces an action on 𝔼′′ with trivial kernel because 𝔼′′ comes from the toric part and is generated by logarithmic forms.

The de Rham bundle ℍ on A_{g} also admits an extension (denoted again by ℍ). This is the logarithmic de Rham sheaf R^{1}π_{∗}(Ω_{X̃_{g}/Ã_{g}}^{•}(log)), where the log refers to the acceptance of logarithmic singularities along the divisor at infinity, cf., [FC90, Thm. VI, 1.1]. We have a short exact sequence

0 → 𝔼 → ℍ → 𝔼^{∨} → 0

extending the earlier mentioned sequence on A_{g}. The symplectic form on ℍ extends as well.

We now want to compare strata on A_{g} and Ã_{g} and for this we introduce some notation. For a given integer 1 ≤ i ≤ g we can consider the Weyl group W_{g−i} as a subgroup of W_{g} by letting it act on the set {i+1, i+2, …, g, …, 2g− i} via the bijection j ⟷ i + j for 1 ≤ j ≤ g − i. More precisely, define ρ_{i}: W_{g−i} → W_{g} via

ρ_{i}(w)(l) = \begin{cases}i + w(l) & for 1 ≤ l ≤ g − i \\ g + l & for g − i + 1 ≤ l ≤ g. \end{cases}

This map respects the Bruhat-Chevalley order and final elements are mapped to final elements.

As symplectic flags on ℍ are determined by their restriction to 𝔼 and as we can extend 𝔼 to Ã_{g}, we can extend F_{g} to a flag space bundle F̃_{g} over Ã_{g}. Then we can also consider the degeneracy loci U_{w} and \overline{U}_{w} for F̃_{g}. We shall use the same notation for these extensions.

Similarly, we can define the notion of a canonical filtration for a semiabelian variety. If 1 → T → A → A′ → 0 is a semi-abelian variety with abelian part A′ and toric part T of rank t and if the function ν′ on {0, c_{1}, …, c_{r}, c_{r+1}, …, c_{2r} = 2 dim(A′)} is the canonical type of A′, then we define the canonical type of A to be the function ν on

{0, t, t + c_{1}, …, t + c_{r}, t + c_{r+1}, …, t + c_{2r}, 2g − t, 2g}

defined by ν(t + c_{i}) = t + ν′(c_{i}). Using this definition we can extend the E-O stratification to Ã_{g}.

The stratification of Ã_{g} by the strata A_{g}^{⟨i⟩} induces a stratification of F̃_{g} by flag spaces F̃_{g}^{⟨i⟩}. Recall that the stratum A_{g}^{⟨i⟩} admits a map q: A_{g}^{⟨i⟩} → A_{g−i} induced by the natural map Ã_{g} → A_{g}^{∗}. Similarly, we have a natural map π_{i} = π: F_{g}^{⟨i⟩} → F_{g−i} given by restricting the filtration on 𝔼 to 𝔼′.

We now describe the interplay between the two stratifications (F_{g}^{⟨i⟩})_{i=1}^{g} and (U_{w})_{w∈W_{g}}.

Lemma 5.1. Let w ∈ W_{g} be an element with w ≤ w_{∅}.

i) We have: U_{w} ∩ F_{g}^{⟨i⟩} ≠ ∅ if and only if w is a shuffle of an element in ρ_{i}(W_{g−i}).

ii) If w = ρ_{i}(w′) with associated degeneracy loci U_{w} ⊂ F_{g} and U_{w′} ⊂ F_{g−i} then we have U_{w} ∩ F_{g}^{⟨i⟩} = π_{i}^{−1}(U_{w′}).

iii) At a point x of Ã_{g} for which the torus part of the “universal” semiabelian scheme has rank r, there is a formally smooth map from the formal completion of Ã_{g} at x to the formal multiplicative group 𝔾̂_{m}^{r} with the following properties. The locus where the torus rank of the universal semi-abelian scheme is s ≤ r is the inverse image of the locus of points of 𝔾̂_{m}^{r} where r − s coordinates are 1. The restriction of this map to the formal completion of any \overline{U}_{w} containing x is formally smooth.

iv) In particular U_{w} is the closure of its intersection with F_{g}.

Proof. A V -stable filtration on 𝔼 restricts to a V -stable filtration on 𝔼′. If U_{w} ∩ F_{g}^{⟨i⟩}is not empty then it determines a w′ ∈ W_{g−i} such that U_{w} ∩ F_{g}^{⟨i⟩} ⊆ π_{i}^{−1}(U_{w′}). Since V is invertible on 𝔼′′ one sees that w is a shuffle of ρ_{i}(w′) and that U_{w} ∩ F_{g}^{⟨i⟩} = π_{i}^{−1}(U_{w′}).

The third part is a direct consequence of the local construction of Ã_{g} using toroidal compactifications and of the universal semi-abelian variety using Mumford’s construction where it is defined by taking the quotient of a semiabelian variety by a subgroup of the torus part, the subgroup being generated by the coordinate functions of 𝔾̂_{m}^{r} (see [FC90, Ch. III:§4] for details). As ℍ and 𝔼 only depend on that universal semi-abelian scheme it is clear that the restriction of the map to a \overline{U}_{w} is smooth.

The last part follows directly from the third. ⊓ ⊔

Note also that Lemma 5.1 is compatible with shuffling. It also results from this lemma that we can define the E-O stratification on the Satake compactification by either considering the closure of the stratum V_{w} on A_{g} or by considering the images of the final strata V_{w} on Ã_{g}.

6 Existence of boundary components

Our intent in this section is to show the existence in irreducible components of our strata of points in the smallest possible stratum \overline{U}_{1}, the stratum associated to the identity element of W_{g}.

Proposition 6.1. Let X be an irreducible component of any \overline{U}_{w} in F_{g}. Then X contains a point of \overline{U}_{1}.

Proof. We prove this by induction on g and on the Bruhat-Chevalley order of w. The statement is clear for g = 1. We start off by choosing a Chai-Faltings compactification Ã_{g} of A_{g} with a semi-abelian family over it (and a “principal” cubical structure so that we get a principal polarization on the semi-abelian variety modulo its toroidal part).

What we now actually want to prove is the same statement as in the proposition but for F̃_{g} instead. Since \overline{U}_{1} is contained in F_{g} the result will follow. We start off by considering Y := X ∩ (F̃_{g}\F_{g}). Assume that Y is non-empty and irreducible. If it is not then we replace it by an irreducible component of Y. Then Y is contained in π_{1}^{−1}(\overline{U}_{w′}) with ρ_{1}(w′) = w for some w′ ∈ W_{g−1}. We claim that Y is an irreducible component of π_{1}^{−1}(\overline{U}_{w′}). This follows from the fact that “we can freely move the toroidal part into an abelian variety” which is Lemma 5.1, part iv).

By induction on g we can assume that \overline{U}_{w′} in F_{g−1} contains \overline{U}_{1′}, where 1′ is the identity element of W_{g−1}. Any component Z of \overline{U}_{ρ_{1}(1′)} which lies in X and meets Y does not lie completely in the boundary F̃_{g} − F_{g}. By induction on the Bruhat-Chevalley order we can assume that w = ρ_{1}(1′) and X = Z. Note also that for any w′′ < w we have that \overline{U}_{w′′} does not meet the boundary and by induction on the Bruhat-Chevalley order we get that X = U_{w} ∩ X. On the other hand if X does not meet the boundary we immediately get the same conclusion.

Hence we may and shall assume that Y has the property that it lies completely inside U_{w} and that it is proper. Lemma 6.2 now shows that it has an ample line bundle of finite order which together with properness forces Y to be zero-dimensional. Now observe that dim Y ≥ ℓ(w) (the proof is analogous to [Ful, Thm 14.3]). This gives ℓ(w) = 0 and so w = 1 and we are reduced to a trivial case. ⊓ ⊔

The following is a version of the so-called Raynaud trick.

Lemma 6.2. Suppose that X is a proper irreducible component of \overline{U}_{w} inside F_{g} such that X ∩ U_{w} = X. Then X is 0-dimensional.

Proof. We have the variety X and two symplectic flags 𝔼_{•} and 𝔻_{•} which at all points of X are in the same relative position w. It follows from 4.2 that we have an isomorphism between L_{i} := 𝔼_{i}/𝔼_{i−1} and M_{w(i)} := 𝔻_{w(i)}/𝔻_{w(i)−1} over X and then as we also have isomorphisms between L_{i}^{p} and M_{g+i} and L_{i} and L_{2g+1−i}^{−1} we conclude that all the L_{i} have finite order. On the other hand we know from the theory of flag spaces that L_{2g} ⨂ L_{2g−1} ⨂ ⋯ ⨂ L_{g+1} is relatively ample and as L_{g} ⨂ L_{g−1} ⨂ ⋯ ⨂ L_{1} is the pull back of an ample line bundle over the base A_{g} we conclude. ⊓ ⊔

7 Superspecial fibres

We shall now discuss the fibre of F_{g} → A_{g} over superspecial points. The superspecial abelian varieties are characterised by the condition that 𝔼_{g} = 𝔻_{g}, i.e., the strata U_{w} for which w ∈ S_{g}. Furthermore, V induces an isomorphism 𝔼/𝔼_{g} ⟶ 𝔼_{g}^{(p)}. On the other hand the polarization gives an isomorphism (𝔼_{g})^{∗} ⟶ 𝔼/𝔼_{g}. This leads to the following definition.

Definition 7.1. i) Let S be an 𝔽_{p}-scheme. A p-unitary vector bundle is a vector bundle E over S together with an isomorphism F^{∗}E ⟶ E^{∗}, where F: S → S is the (absolute) Frobenius map.

ii) Let E be a p-unitary vector bundle over S and let P → S be the bundle of complete flags on E. The p-unitary Schubert stratification of P is the stratification given by letting U_{w}, w ∈ S_{g}, consist of the points for which the universal flag F and the dual of the Frobenius pullback (F^{∗}F)^{∗} are in position corresponding to w.

A map F^{∗}E → E^{∗} of vector bundles is the same thing as a map F^{∗}E ⨂_{O_{S}} E → O_{S}, which in turns corresponds to a bi-additive map ⟨−, −⟩: E × E → O_{S} fulfilling ⟨fa, b⟩ = f^{p}⟨a, b⟩ and ⟨a, fb⟩ = f⟨a, b⟩. We shall normally use this latter description.

All p-unitary vector bundles are trivial in the étale topology as the following proposition shows.

Proposition 7.2. If ⟨−, −⟩ is a p-unitary structure on the vector bundle E then

E := {a ∈ E : ∀b ∈ E: ⟨b, a⟩ = ⟨a, b⟩^{p} }

is a local (in the étale topology) system of 𝔽_{p^{2}}-vector spaces. Furthermore, ⟨−, −⟩ induces a unitary (with respect to the involution (−)^{p} on 𝔽_{p^{2}}) structure on E. Conversely, if E is a local system of 𝔽_{p^{2}}-vector spaces with a unitary structure, then E := E ⨂_{𝔽_{p^{2}}} O_{S} is a p-unitary vector bundle.

These two constructions establish an equivalence between the categories of p-unitary vector bundles and that of local systems of unitary 𝔽_{p^{2}}-vector spaces. In particular all p-unitary vector bundles (of the same rank) are locally isomorphic in the étale topology.

Proof. The pairing ⟨−, −⟩ gives rise to an isomorphism ψ′: E ⟶ (F^{∗}E)^{∗} by a ↦ b ↦ ⟨b, a⟩ and an isomorphism ψ: (F^{2})^{∗}: E ⟶ (F^{∗}E)^{∗} by a ↦ b ↦ ⟨a, b⟩^{p}. The composite ρ := ψ^{−1} ∘ψ′ thus gives an isomorphism E ⟶ (F^{2})^{∗}E. Then E is simply the kernel of ρ − 1 ⊗ id and the fact that E = E ⨂_{𝔽_{p^{2}}} O_{S} follows from [HW61]. The pairing ⟨−, −⟩ then induces a unitary pairing on E which is perfect as ⟨−, −⟩ is. Conversely, it is clear that a unitary pairing on E translates to one on E.

Finally, as all (perfect) unitary pairings on 𝔽_{p^{2}}-vector spaces of fixed dimension are isomorphic we get the local isomorphism. ⊓ ⊔

Proposition 7.2 has the following immediate corollary.

Corollary 7.3. The flag variety fibrations of two p-unitary vector bundles of the same rank on the same base are locally isomorphic by an isomorphism preserving the p-unitary Schubert strata.

Let us now consider the situation where the base scheme is Spec(𝔽_{p^{2}}) and E is a 𝔽_{p^{2}}-vector space given a unitary perfect pairing. The smallest p-unitary Schubert stratum consists of flags that coincide with their unitary dual. Taking duals once more we see that they are taken to themselves after pullback by the square of the Frobenius, hence are defined over 𝔽_{p^{2}}. Furthermore, they are selfdual with respect to the unitary pairing. This should come as no surprise as that stratum corresponds to final filtrations on superspecial abelian varieties. The next to lowest strata are somewhat more interesting.

Lemma 7.4. Let V be a g-dimensional 𝔽_{p^{2}} unitary vector space and let ℙ be projective space based on V. If s = (i, i + 1) ∈ S_{g} for some 1 ≤ i < g then the closed Schubert stratum U_{s} ⊆ ℙ consists of the flags 0 = 𝔼_{0} ⊂ 𝔼_{1} ⊂ ⋯ ⊂ 𝔼_{g} where the 𝔼_{j} for j ≠ i, g − i are 𝔽_{p^{2}}-rational with 𝔼_{j}^{⊥} = 𝔼_{g−j}, 𝔼_{g−i} = (𝔼_{i}^{(p)})^{⊥} unless i = 2g, and 𝔼_{i} ≠ (𝔼_{g−i}^{(p)})^{⊥}.

Proof. If 𝔼_{•} and 𝔻_{•} are two flags in V ⨂ R (R some 𝔽_{p^{2}}-algebra) and they are in position s, then dim(𝔼_{j} ∩ 𝔻_{j}) = r_{s}(j, j) = j for i ≠ j, i.e., 𝔼_{j} = 𝔻_{j} and for i the conditions give us 𝔼_{i} ∩ 𝔻_{i} = 𝔼_{i−1}. In our case, where 𝔻_{j} = F^{∗}𝔼_{g−j}^{⊥}, this means 𝔼_{j} = F^{∗}𝔼_{g−j}^{⊥} for j ≠ i and 𝔼_{i} ≠ (𝔼_{g−i}^{(p)})^{⊥}. If also j ≠ g − i we can

use this twice and get that 𝔼_{j} = 𝔼_{j}^{(p^{2})}, i.e., 𝔼_{j} is 𝔽_{p^{2}}-rational. ⊓ ⊔

As a result we get the following connectedness result, analogous to [Oo01, Prop. 7.3].

Theorem 7.5. Let V be a g-dimensional 𝔽_{p^{2}} unitary vector space and let ℙ be projective space based on V. Let S ⊆ \{1, …, g − 1\}. Let \overline{U} be the union of the \overline{U}_{s_{i}} for i ∈ S. Then two flags 0 = A_{0} ⊂ A_{1} ⊂ ⋯ ⊂ A_{g−1} ⊂ A_{g} and 0 = B_{0} ⊂ B_{1} ⊂ ⋯ ⊂ B_{g−1} ⊂ B_{g} in \overline{U}_{1} lie in the same component of \overline{U} precisely when B_{i} = A_{i} for all i ∉ S. Furthermore, every connected component of \overline{U} contains an element of \overline{U}_{1}.

Proof. The last statement is clear as every irreducible component of any \overline{U}_{s_{i}} contains a point of \overline{U}_{1}. This follows from Proposition (6.1) but can also easily be seen directly.

We start by looking at the locus \overline{U}_{F}^{i} of a \overline{U}_{s_{i}} with 1 ≤ i < g of flags for which all the components of the flag except the dimension i and dimension g − i parts are equal to a fixed (partial) 𝔽_{p^{2}}-rational self-dual flag F_{•}, the following claims are easily proved using Lemma 7.4.

i) For any 1 ≤ i ≤ (g − 2)/2 or (g + 2)/2 ≤ i < g we get an element in \overline{U}_{F}^{i} by picking any 𝔼_{i−1} ⊂ 𝔼_{i} ⊂ 𝔼_{i+1} and then letting 𝔼_{g−i} be determined by Lemma 7.4. Hence the locus is isomorphic to ℙ^{1} and the intersection with \overline{U}_{1} consists of the points for which 𝔼_{i} and 𝔼_{g−i} are 𝔽_{p^{2}}-rational.

ii) When g is even we get an element in \overline{U}_{F}^{i} by picking 𝔼_{g/2−1} ⊂ 𝔼_{g/2} ⊂ 𝔼_{g/2+1}. Hence the locus is isomorphic to ℙ^{1} and the intersection with \overline{U}_{1} consists of the points for which 𝔼_{g/2} is 𝔽_{p^{2}}-rational.

iii) When g is odd we get an element in \overline{U}_{F}^{i} by picking 𝔼_{(g−3)/2} ⊂ 𝔼_{(g−1)/2} ⊂

𝔼_{(g+3)/2} for which \overline{𝔼}_{(g−1)/2} ⊂ F^{∗}\overline{𝔼}_{(g−1)/2}^{⊥}, where \overline{𝔼}_{(g−1)/2} = 𝔼_{(g−1)/2}/𝔼_{(g−3)/2} and F^{∗} comes from the 𝔽_{p^{2}}-rational structure on 𝔼_{(g+3)/2}/𝔼_{(g−3)/2} and the scalar product is induced from that on 𝔼_{g}, and then define 𝔼_{(g+1)/2} by

the condition that 𝔼_{(g+1)/2}/𝔼_{(g−3)/2} = F^{∗}\overline{𝔼}_{(g−1)/2}^{⊥}. As all non-degenerate unitary forms are equivalent, choosing a basis of 𝔼_{(g+3)/2}/𝔼_{(g−3)/2} for which the form has the standard form ⟨(x, y, z), (x, y, z)⟩ = x^{p+1} + y^{p+1} + z^{p+1} and hence \overline{U}_{F}^{i} is isomorphic to the Fermat curve of degree p + 1 and hence is irreducible. The intersection with \overline{U}_{1} consists of the points for which 𝔼_{(g−1)/2} is 𝔽_{p^{2}}-rational and then 𝔼_{(g+1)/2} = 𝔼_{(g−1)/2}^{⊥}. iv) When g is odd we get an element in \overline{U}_{F}^{i} by picking 𝔼_{(g+3)/2} fulfilling

conditions dual to those of iii). Hence again \overline{U}_{F}^{i} is irreducible and the intersection with \overline{U}_{1} consists of the points for which 𝔼_{(g+1)/2} is 𝔽_{p^{2}}-rational and then 𝔼_{(g−1)/2} = 𝔼_{(g+1)/2}^{⊥}.

It follows from this description that two flags in \overline{U}_{1} lie in the same component of \overline{U} if and only if they are equivalent under the equivalence relation generated by the relations that for any unitary 𝔽_{p^{2}}-flag 0 = A_{0} ⊂ A_{1} ⊂ ⋯ ⊂ A_{g−1} ⊂ A_{g} we may replace it by any flag that is the same except for A_{i} and A_{g−i} for i ∈ S. The theorem then follows from the following Lemma 7.6. ⊓ ⊔

Lemma 7.6. i) Let k be a field and Fℓ_{n} be the set of complete flags of vector spaces in a finite dimensional vector space. The equivalence relation generated by the operations of modifying a flag E_{•} by, for any i, replacing E_{i} by any i-dimensional subspace of E_{i+1} containing E_{i−1} contains just one equivalence class.

ii) Let Fℓ_{n} the set of complete flags of vector spaces in an n-dimensional 𝔽_{p^{2}}-vector space, self-dual with respect to a perfect unitary pairing. An elementary modification of such a flag E_{•} is obtained by either, for any 1 ≤ i ≤ (n − 1)/2, replacing E_{i} by any isotropic i-dimensional subspace of E_{i+1} containing E_{i−1} and E_{n−i} by its annihilator or, when n is even, replacing E_{n/2} by any maximal totally isotropic subspace contained in E_{n/2+1} and containing E_{n/2−1}. Then the equivalence relation generated by all elementary modifications contains just one equivalence class.

Proof. Starting with i), we prove it by induction on n, the dimension of the vector space V. Given two flags E_{•} and F_{•}, if E_{1} and F_{1} are equal we may use induction applied to E_{•}/E_{1} and F_{•}/E_{1}. We now use induction on the smallest j such that E_{1} ⊆ F_{j}. The case j = 1 has already been taken care of. We now get a new flag F′_{•} by replacing F_{j−1} by F_{j−2} ⨁ E_{1}, which works as E_{1} ⊊ F_{j−1} and we then have E_{1} ⊆ F′_{j−1}.

Continuing with ii) we again use induction on n and start with two selfdual flags E_{•} and F_{•}. Let us first assume that n is even, n = 2k. Then E_{k} and F_{k} are isotropic subspaces. If they have non-trivial intersection then we may pick a 1-dimensional subspace contained in it and then use i) to replace E_{•} and F_{•} by flags for which E_{k} and F_{k} are the same and E_{1} = F_{1}. This implies that also E_{n−1} = F_{n−1} and we may consider E_{n−1}/E_{1} with its two flags induced from E_{•} and F_{•} and use induction to conclude. Assuming F_{k} ∩ E_{k} = {0} we may again use i) to modify F_{•}, keeping F_{k} fixed, so that E_{1} ⊆ F_{k+1}. This means that E_{1} ⨁ F_{k−1} is totally isotropic and we may replace F_{k} by it to obtain a new flag F′_{•} for which F′_{k} and E_{k} intersect non-trivially.

When n is odd, n = 2k + 1, we may again use induction on n to finish if E_{k} and F_{k} intersect non-trivially. If not we may again use i) to reduce to the case when E_{1} ⊆ F_{k+2} and then we may replace F_{k} by E_{1} ⨁ F_{k−1} and F_{k+1} by its annihilator. ⊓ ⊔

8 Local structure of strata

8.1 Stratified Spaces

We now want to show that F_{g} looks locally like the space of complete symplectic flags (in 2g-dimensional space). More precisely we shall get an isomorphism between étale neighbourhoods of points that preserves the degeneration strata. This is proved by showing a result on suitable infinitesimal neighbourhoods which involves not just the complete flag spaces but also partial ones. In order to have a convenient way of formulating such a result we introduce the two following notions:

By a stratified space we shall mean a scheme together with a collection of closed subschemes, called strata. A map between stratified spaces is said to be stratified if it maps strata into strata.

If P is a partially ordered set then a diagram X_{•} of spaces over P associates to each element q of P a scheme X_{q} and to each relation q > q′ a map X_{q} → X_{q′} fulfilling the condition that the composite X_{q} → X_{q′} → X_{q′′} equals the map X_{q} → X_{q′′} for any q > q′ > q′′. We shall also similarly speak about a diagram of stratified spaces where both the schemes and the maps are assumed to be stratified. Given a field k and a k-point x of a diagram X_{•} we may speak of its (strict) Henselisation at x, which at each q ∈ P is the Henselisation at x of X_{q}.

For a positive integer g we now consider the partially ordered set P_{g} whose elements are the subsets of {1, 2, …, g−1} and with ordering that of inclusion. We have two diagrams of stratified spaces over this set: The first, Fℓ_{g}^{•}, associates to the subset S the flag space of a maximal totally isotropic subspace E of a symplectic 2g-dimensional vector space and partial flags of subspaces of E whose dimensions form the set S. The map associated to an inclusion S ⊂ S′ is simply the map forgetting some of the elements of the flag. Similarly, we let F_{g}^{•} be the diagram that to a subset S associates the space of flags over the moduli space A_{g} of principally polarised abelian varieties that associates to a principally polarized abelian variety the space of flags on its Hodge bundle whose dimensions form the subset S.

The diagram Fℓ_{g}^{•} becomes a stratified diagram by considering the stratifications given by the (closed) Schubert cells with respect to some fixed complete flag. In positive characteristic p the diagram F_{g}^{•} becomes a stratified diagram by considering the degeneracy loci given by the relative positions of the Hodge flag 𝔼_{•} and the conjugate flag 𝔻.

8.2 Height 1-Maps

For schemes in a fixed positive characteristic p we shall say that a closed immersion S ↪ S′ defined by the ideal sheaf I on S′ is a height 1-map if I_{S}^{(p)} = 0, where for an ideal I, we let I^{(p)} be the ideal generated by the p’th powers of elements of I. If R is a local ring in characteristic p with maximal ideal m_{R}, the height 1-hull of R is the quotient R/m_{R}^{(p)}. It has the property that its spectrum is the largest closed subscheme of Spec R for which the map from Spec R/m_{R} to Spec R/m_{R}^{(p)} is a height 1-map. If k is a field of characteristic p and x: Spec k → S a k-map to a k-scheme S of characteristic p, then by the height 1-neighbourhood of x we will mean the spectrum of the height 1-hull of the local ring of S at x. It is clear that taking height 1-neighbourhoods of k-points is functorial under maps between pointed kschemes. Finally, we shall say that two local rings are height 1-isomorphic if their respective height 1-hulls are isomorphic and that the height 1-hull of a k-point is height 1-smooth if it is isomorphic to the height 1-hull of a smooth k-point (i.e., is of the form k[[t_{1}, …, t_{n}]]/m^{(p)}) with m = (t_{1}, …, t_{n}).

Theorem 8.1. For each perfect field k of positive characteristic p and each kpoint x of F_{g}^{•} there is a k-point y of Fℓ_{g}^{•} such that the height 1-neighbourhood of x is isomorphic to the height 1-neighbourhood of y by a stratified isomorphism of diagrams.

Proof. Denote also by x the point of F_{g}, the space of complete flags of the Hodge bundle, associated to x as a point of the diagram F_{g}^{•}. Let X^{•} be the height 1-neighbourhood of x in F_{g}^{•} and X the height 1-neighbourhood of x in F_{g}. Now the ideal of the closed point of x in X has a divided power structure for which all the divided powers of order ≥ p are zero. This allows us get a trivialization of the restriction of the de Rham cohomology ℍ_{X} ⟶ X × W which is horizontal (i.e., compatible with the Gauss-Manin connection on the left and the trivial connection on the right). Now, as the absolute Frobenius map on X factors through the closed point we get that the pullback 𝔼_{•}^{(p)} is a horizontal flag and then so is 𝔻_{•}, its elements being either inverse images of horizontal subbundles by the horizontal map V or duals of horizontal subbundles. We now get a map from X to the space Fℓ_{g} of complete symplectic flags on W such that the pullback of the universal flag equals 𝔼_{•}. We may, furthermore, choose a symplectic isomorphism of W and the standard symplectic space such that 𝔻_{•} is taken to the fixed complete flag. We can extend this map in a compatible fashion for all partial flag spaces giving a map from the diagram X^{•} to Fℓ_{g}^{•} and we will denote by y the k-point which is the composite of x and this map. This map is clearly a stratified map and by the infinitesimal Torelli theorem (cf., [FC90, pp. 14-15]) it induces an isomorphism from X^{•} to Y^{•}, the first height 1-neighbourhood of y in Fℓ_{g}^{•}. ⊓ ⊔

Theorem 8.2. For each perfect field k of positive characteristic p and each k-point x of F_{g} there is a k-point y of Fℓ_{g} such that the Henselisation of x is isomorphic to the Henselisation of y by a stratified isomorphism.

Proof. The theorem provides such an isomorphism over the height 1-hull X of x. Now, over O_{F_{g},x} we may extend the trivialization of ℍ_{X} to a trivialization of ℍ_{F_{g},x} which also extends the trivialization of 𝔻 (making, of course, no requirements of horizontality). This gives a map from the localization, X̃, of F_{g} at x to Fℓ_{g} that extends the map from X to Fℓ_{g}. It thus induces a map from X̃ to Ỹ, the localization Fℓ_{g}^{•} at y. Now, this map induces an isomorphism on tangent spaces and F_{g} is smooth. This implies that we get an induced isomorphism on Henselisation and proves the theorem. ⊓ ⊔

Lemma 8.3. Let A be a principally polarised abelian variety over an algebraically closed field. If a flag 𝔻_{•} for it has type w′ which ≤ its canonical type then w′ is the final element corresponding to the canonical type of A.

Proof. The flag 𝔻_{•} has the property, as it is of a type ≤ to the canonical type, that F maps 𝔻_{i}^{(p)} into 𝔻_{ν_{w}(i)}. Consider now the set I of i’s for which 𝔻_{i} is a member of the canonical flag. It clearly contains 0 and is closed under i ↦ \overline{ı}. Furthermore, if i ∈ I, then F (𝔻_{i}^{(p)}) has dimension ν_{w}(i) but is then equal to 𝔻_{ν_{w}(i)} as it is contained in it. Hence I fulfills the conditions of Corollary 2.10 and hence contains the canonical domain which means that 𝔻_{•} is a refinement of the canonical flag and thus ν, the final type of A, and ν_{w′} coincide on the canonical domain of ν and are equal by Corollary 2.10 which means that w′ is the final element of the canonical type. ⊓ ⊔

Corollary 8.4. i) Each stratum U_{w} of F_{g} is smooth of dimension ℓ(w) (over 𝔽_{p}).

ii) The closed stratum \overline{U}_{w} (again of F_{g}) is Cohen-Macaulay, reduced, and normal of dimension ℓ(w) and \overline{U}_{w} is the closure of U_{w} in F_{g} for all w ∈ W_{g}.

iii) If w is final then the restriction of the projection F_{g} → A_{g} to U_{w} is a finite surjective étale covering from U_{w} to V_{w} of degree γ_{g}(w).

Proof. We know that each open Schubert cell of Fℓ_{g} is smooth and each closed one is Cohen-Macaulay by a proof that runs completely along the lines of [Ful, Thm 14.3.]. By a theorem of Chevalley (cf., [Ch94, Cor. of Prop. 3]) they are smooth in codimension 1 so by Serre’s criterion (cf., [Gr65, Thm 5.8.6]) they are normal and reduced. The same statement for the stratification of F_{g} then follows from the theorem. To finish ii), the fact that \overline{U}_{w} = \overline{U_{w}} follows more or less formally from the rest: If x ∈ \overline{U}_{w} then we know that the dimensions of all U_{w′} with w′ < w that pass through in x are ℓ(w′) < ℓ(w) but the dimension of \overline{U}_{w} at that point is ℓ(w) and hence x must lie in the closure of U_{w}.

As for iii), that U_{w} maps into V_{w} follows from the fact that the restriction of a final filtration to its canonical domain is a canonical filtration (Proposition 4.5). That the map U_{w} → A_{g} is unramified follows from the same statement for Schubert cells which is [BGG73, Prop. 5.1]. We next prove that U_{w} → V_{w} is proper. Note that by Proposition 4.5 and by the fact that by definition V_{w} is the image of U_{w} we get that the geometric points of V_{w} consists of the principally polarised abelian varieties with a canonical filtration whose canonical type corresponds to the final type of w. Hence for properness we may assume that we have a principally polarised abelian variety over a discrete valuation ring R such that both its generic and special point are of type w and we suppose given a final flag over the generic point. Hence the canonical decomposition of Spec R for the abelian variety is equal to Spec R and we have a canonical flag over Spec R. As \overline{U}_{w} is proper the map to it from the generic point of Spec R extends to a map from Spec R to \overline{U}_{w} hence giving a flag over its special point. This flag is then of a type ≤ w and hence by Lemma 8.3 its type is equal to w and the image of Spec R lies in U_{w} which proves properness.

Now, V_{w} being by definition the schematic image of U_{w}, it is reduced as U_{w} is. As U_{w} → V_{w} is unramified it has reduced geometric fibres and as it is finite and V_{w} is reduced, to show that it is flat it is enough to show that the cardinalities of the geometric fibres are the same for all geometric points of V_{w}. This however is Lemma 4.6. Being finite, flat, and unramified it is étale. That its degree is γ(w) follows from Lemma 4.6. ⊓ ⊔

Remark 8.5. i) The corollary is true also for the strata of F̃_{g}.

ii) Note that the degree of the map U_{w} → V_{w} is γ_{g}(w). By looking at the proof of Lemma 4.6 it is not difficult to show that it is a covering with structure group a product of linear and unitary groups over finite fields of characteristic p.

9 Punctual flag spaces

Let M be the (contravariant) Dieudonné module of a truncated Barsotti-Tate group of level 1 over an algebraically closed field of characteristic p provided with an alternating perfect pairing (of Dieudonné modules). We let F_{M}, the punctual flag space for M, be the scheme of self-dual admissible complete flags in M for which the middle element equals Im(V). It is well-known that every such M occurs as the Dieudonné module of the kernel of multiplication by p on a principally polarised abelian variety. Then F_{M} is the intersection of \overline{U}_{∅} and the fibre over a point of A_{g} giving rise to M. Also, by a result of Oort (cf., [Oo01]), the canonical type of M determines it (over an algebraically closed field) and hence we shall also use the notation F_{ν} where ν is a final type. For Γ = (I, S) where I ⊆ {1, …, g} with #I equal to the semi-simple rank of M and S a complete V -stable flag of the V -semi-simple part of M we define F_{M}^{Γ} as follows: We let F_{M}^{Γ} be the part of F_{M} ∩ U_{I}^{ss} (U_{I}^{ss} may clearly be defined directly in terms of M) for which the flag induces S on the V -semi-simple part. The p-rank of M is denoted by f and we easily see that F_{M} is the disjoint union of the F_{M}^{Γ} and putting F_{M}^{S} := F_{M}^{({1,...,g−f+1},Γ)} we have maps S_{I}: F_{M}^{I} → F_{M}^{S}. These maps are homeomorphisms by Proposition 4.14. This can be seen directly by decomposing M as M^{mul} ⨁ M^{ℓℓ} ⨁ M^{et}, where V is bijective on M^{mul}, F on M^{et} and F and V nilpotent on M^{ℓℓ}. Any element of an admissible flag over a perfect field will decompose in the same way (as that element is stable under F by definition and under V by duality) and is hence determined by its intersection with M^{mul}, M^{ℓℓ}, and M^{et}. By self-duality the intersection of all the elements of the flag with M^{et} is determined by that with M^{mul} and that part is given by an arbitrary full flag of sub-modules of M^{mul} which is our S. That means that we may indeed reconstitute the whole flag from Γ and the induced flag on M^{ℓℓ} and that any choice of flag on M^{ℓℓ} gives rise to a flag in F_{M}^{Γ}. That means that the map F_{M}^{Γ} → F_{M^{ℓℓ}} is a homeomorphism and we may for all practical purposes focus our attention on the case when F and V are nilpotent on M (i.e., M is local-local). Hence in this section unless otherwise mentioned the Dieudonné modules considered will be local-local. Note that the principal interest in this section will be focused on the question of which U_{w} have a non-empty intersection with F_{M} and that that problem is indeed by the above considerations immediately reduced to the local-local case.

We shall make extensive use of one way to move in each F_{M}:

Consider a w_{∅} ≥ w ∈ W_{g}. Assume that we have an index 1 ≤ i ≤ g − 1 for which r_{w}(g + i − 1, i + 1) ≥ i + 1. This means that for a flag 𝔻_{•} in \overline{U}_{w} we have that F (𝔻_{i+1}) ⊆ 𝔻_{i−1} or equivalently that F is zero on 𝔻_{i+1}/𝔻_{i−1}. Hence if we replace 𝔻_{i} by any 𝔻_{i−1} ⊂ 𝔻 ⊂ 𝔻_{i+1} (replacing also 𝔻_{2g−i} to make the flag self-dual) we shall still have an admissible flag as V (𝔻) ⊆ 𝔻_{i−1}. In order to construct the 𝔼-flag we apply V to the 𝔻-flag which gives us half of the 𝔼flag and we complement by taking orthogonal spaces. In the 𝔼-flag now 𝔼_{g−i} and 𝔼_{g+i} move. This construction gives a mapping from the projective line ℙ(𝔼_{g−i+1}/𝔼_{g−i−1}) to F_{M} and we shall therefore call this family the simple family of index i (with, of course, respect to M) and we shall write P_{w,i} for this simple family . The condition r_{w}(g + i − 1, i + 1) ≥ i + 1 is equivalent to r_{w}(g − i + 1, i − 1) = g − i − 1 and when it is fulfilled we shall say that g − i is movable for w.

Proposition 9.1. Any two points of the local flag space F_{M}^{Γ} can be connected by a sequence of simple families.

Proof. We immediately reduce to the case when M is local (in which case the statement is about F_{M}). We are going to identify F_{M} with the scheme of V -stable flags in Im(V) and we prove the statement for any Dieudonné module N with F = 0 and V nilpotent. Let E_{•} and F_{•} be two V -stable flags in N. If E_{1} = F_{1} then we may consider N/E_{1} and use induction on the length of N to conclude. If not, we use induction on the smallest i such that F_{1} ⊆ E_{i} which we thus may assume to be > 1. We now have F_{1} ⊊ E_{i−1} and hence that F_{1} is a complement to E_{i−1} in E_{i} so that in particular E_{i}/E_{i−2} = (E_{i−1}/E_{i−2}) ⨁(F_{1} + E_{i−2})/E_{i−2}. This has a consequence that V is zero on E_{i}/E_{i−2} which means that every subspace of it is stable under V so that we get a ℙ^{1}-family of flags in E_{i}/E_{i−2} in which both E_{i−1}/E_{i−2} and (F_{1} + E_{i−2})/E_{i−2} are members so that we may move E_{i−1} so that it contains F_{1}. ⊓ ⊔

Recall (cf, [Oo01, 14.3]) that one defines the partial order relation on final types ν_{1} ⊆ ν_{2} (resp. ν_{1} ⊂ ν_{2}) by the condition that V_{ν_{1}} ⊆ \overline{V}_{ν_{2}} (resp. V_{ν_{1}} ⊊ \overline{V}_{ν_{2}}). We shall now see that this relation can be expressed in terms of local flag spaces. For this we let M_{ν} be a Dieudonné module of a principally polarised truncated Barsotti-Tate group of level 1 with final type ν (there is up to isomorphism only one such M_{ν}, [Oo01, Thm. 9.4]).

Theorem 9.2. i) We have that ν′ ⊂ ν precisely when there is a w ∈ W_{g} such that w ≤ ν and there is a flag of type w in F_{M_{ν′}}.

ii) If there is a flag of type w in F_{M_{ν′}}, then there is a w′ ≤ w such that the intersection U_{w′} ∩ F_{M_{ν′}} is finite.

Proof. Consider the image in A_{g} of \overline{U}_{ν}. It is a closed subset containing V_{ν} and hence contains \overline{V}_{ν} and in particular it meets each fibre over a point of \overline{V}_{ν}. Consequently there is a point s in the intersection of \overline{U}_{ν} and the fibre over a point t of V_{ν′}. Now, s lies in some U_{w} ⊆ \overline{U}_{ν} and hence fulfills w ≤ ν and as ν ≤ w_{∅}, s also lies in the local flag space of t and as has been noted this is the “same” as F_{M_{ν′}}. The converse is clear.

As for the second part, the proof of Lemma 6.2 shows that a w′ ≤ w which is minimal for the condition that U_{w′} ∩ F_{M_{ν′}} is non-empty has U_{w′} ∩ F_{M_{ν′}} finite. ⊓ ⊔

The theorem allows us to reprove a result of Oort (cf., [Oo01]); the E-O strata are defined in subsection 4.4.

Corollary 9.3. The E-O stratification on A_{g} is a stratification.

Proof. The condition in 9.2, i) says that ν′ ⊂ ν if and only if the closure \overline{U}_{ν} of U_{ν} has a non-empty intersection with the punctual flag space F_{ν′}. The proof there gives more precisely that a given point s of V_{ν′} lies in \overline{V}_{ν} precisely when \overline{U}_{ν} intersects the fibre over s of the map F_{g} → A_{g}. This condition does not depend on the point s by a result of Oort on Dieudonné modules (cf., [Oo01]). ⊓ ⊔

From Theorem 9.2 it is clear that the condition that U_{w}∩F_{M} ≠ ∅ is important. We shall say that an admissible w ∈ W_{g} occurs in ν, with ν a final type, if U_{w} ∩ F_{M} ≠ ∅ and we shall write it symbolically as w → ν.

Remark 9.4. It is important to realise that a priori this relation w → ν depends on the characteristic that is implicit in all of this article. Hence the notation w →^{p} ν would be more appropriate. It is our hope that the relation will a posteriori turn out to be independent of p. If not and if one is working with several primes p the more precise notation will have to be used.

Hence we can formulate the theorem as saying that ν′ ⊂ ν precisely when there exists an admissible w with w → ν′ and w ≤ ν. Suppose final types ν and ν′ given. For an element w of minimal length in the set of minimal elements of {w ∈ W_{g} : ν > w, w → ν′} in the Bruhat-Chevalley order, we then have the following property. The space U_{w} ∩ F_{M_{ν′}} has dimension 0 for the generic point of V_{ν′}. Clearly, then ℓ(w) ≥ ℓ(ν′) for every w as in 9.2.

Example 9.5. Since E-O strata on A_{g} are defined using the projection from the flag space, the closure of an E-O stratum on A_{g} need not be given by the Bruhat-Chevalley order on the set of final elements and indeed it isn’t. Oort gave the first counterexample for g = 7 based on products of abelian varieties. We reproduce his example and give two other ones, one for g = 5 and one for g = 6 that do not come from products.

i) Let g = 7 and let w_{1} = [1, 2, 4, 6, 7, 10, 12] and w_{2} = [1, 2, 3, 7, 9, 10, 11]. Then w_{1} and w_{2} are final elements of W_{7} and have length ℓ(w_{1}) = 8 and ℓ(w_{2}) = 9. In the Bruhat-Chevalley order neither w_{1} ≤ w_{2} nor w_{2} ≤ w_{1} holds. Despite this we have \overline{V}_{w_{1}} ⊂ \overline{V}_{w_{2}}. The explanation for this lies in the fact that the simple family P_{w_{1},4} hits the stratum U_{w_{3}} with w_{3} the element [1, 2, 3, 7, 6, 10, 11] = s_{3}w_{1}s_{4}, with w_{2} > w_{3} and w_{3} → w_{1}, so by 9.2 it follows that \overline{V}_{w_{1}} ⊂ \overline{V}_{w_{2}}. (That there is such a simple family can be proved directly but for now we leave it as an unsupported claim as a proof “by hand” would be somewhat messy. A more systematic study of these phenomena will appear in a subsequent paper.) This explains the phenomenon observed in [Oo01], p. 406 (but note the misprints there). Also the element w_{2} > w_{4} = [1, 2, 3, 7, 9, 5, 11] → w_{1} will work for w_{1}. The element w_{1} is the final element corresponding to taking the product of a Dieudonné module with final element [135] and a Dieudonné module with final element [1246] whereas similarly w_{2} appears as the “product” of the final elements [135] and [1256]. As [1246] < [1256] there is a degeneration of a Dieudonné module of type [1256] to one of type [1246]. This shows that this example simply expresses the fact that ⊂ must be stable under products whereas the Bruhat-Chevalley order isn’t. (We’d like to thank Ben Moonen for pointing this out to us.)

ii) For g = 5 we consider the final elements w_{1} = [1, 3, 4, 6, 9] and w_{2} = [1, 2, 6, 7, 8] of length 5 and 6 and the non-final element w_{3} = [1, 2, 6, 4, 8] in W_{5}. Then w_{3} < w_{2} and w_{3} → w_{1}, so that V_{w_{1}} lies in the closure of V_{w_{2}}. But in the Bruhat-Chevalley order neither w_{1} < w_{2} nor w_{2} < w_{1} holds.

iii) Let g = 6 and consider the final elements w_{1} = [1, 3, 5, 6, 9, 11] and w_{2} := [1, 2, 6, 8, 9, 10] of length ℓ(w_{1}) = 8 and ℓ(w_{2}) = 9. In the Bruhat-Chevalley order we do not have w_{1} ≤ w_{2}. Nevertheless, V_{w_{1}} occurs in the closure of the E-O stratum V_{w_{2}}. Indeed, the admissible element w_{3} = [1, 2, 6, 8, 4, 10] satisfies w_{2} ≥ w_{3} → w_{1}: U_{w_{3}} has a non-empty intersection with the punctual flag space F_{w_{1}}. This time neither of the elements w_{1} and w_{2} are products in the sense of i). Furthermore, as V_{w_{1}} is of codimension 1 in V_{w_{2}} this example can not be derived by taking the transitive closure of the closure under products of the Bruhat-Chevalley relation. The claim that we have w_{3} → w_{1} and the two preceding ones will be substantiated in a subsequent paper.

There is an approach to the study of the relation of the E-O strata and the strata on F_{g} which is in some sense “dual” to the study of punctual flag spaces; that of considering the image in A_{g} of the U_{w}. The following result gives a compatibility result on these images and the E-O stratification.

Proposition 9.6. i) The image of any U_{w}, w ∈ W_{g}, is a union of strata V_{ν}. In particular the image of a \overline{U}_{w} is equal to some \overline{V}_{ν}.

ii) For any final ν and w ∈ W_{g}, the maps U_{w} ∩ π^{−1}V_{ν} → V_{ν} and \overline{U}_{w} ∩ π^{−1}V_{ν} → V_{ν}, where π is the projection F_{g} → A_{g}, have the property that there is a surjective flat map X → V_{ν} such that the pullback of them to X is isomorphic to the product X × (F_{ν} ∩ U_{w}) resp. X × (F_{ν} ∩ \overline{U}_{w}).

iii) A generic point of a component of U_{w} maps to the generic point of some V_{ν} and that ν is independent of the chosen component of U_{w}.

Proof. The first part follows directly from Oort’s theorem (in [Oo01]) on the uniqueness for the Dieudonné module in a stratum V_{ν}, as it implies that if one fibre of π^{−1}(V_{ν}) → V_{ν} meets U_{w} then they all do. As for the second part it would follow if we could prove that there is a surjective flat map X → V_{ν} such that the pullback of (ℍ, 𝔼, F, V, ⟨−, −⟩) is isomorphic to a the constant data (provided by the Dieudonné module of type ν). For this we first pass to the space X_{ν} of bases of ℍ for which the first g elements form a basis of 𝔼 which is flat surjective over V_{ν}. Over X_{ν} the data is the pullback from a universal situation where F, V, and ⟨−, −⟩ are given by matrices. In this universal situation we have an action of the group G of base changes and two points over an algebraically closed field give rise to isomorphic (ℍ, 𝔼, F, V, ⟨−, −⟩) precisely when they are in the same orbit. By assumption (and Oort’s theorem) the image of X_{ν} lies in an orbit so it is enough to show that the data over an orbit can be made constant by a flat surjective map. However, the map from G to the orbit obtained by letting g act on a fixed point of the orbit has this property.

The third part follows directly from the second. ⊓ ⊔

The proposition gives us a map τ_{p}: W_{g} → W_{g}/S_{g} which to w associates the final type of the open stratum into which each generic point of U_{w} maps. We shall return to this map in Section 13.

Example 9.7. Note that the punctual flag space is in general rather easy to understand as it only depends on the image of V and we are almost talking about the space of flags stable under a nilpotent endomorphism (remember that we have reduced to the local-local case). Almost, but not quite, as the endomorphism is semi-linear rather than linear. What is complicated is the induced stratification. Already the case of ν = s_{3} ∈ W_{3} is an illustrative example. We have then that ker V ∩ Im V is of dimension 2; in fact, we have one Jordan block for V on Im V of size 2 and one of size 1. The first element, 𝔼_{1}, of the flag must lie in ker V ∩ Im V so we get a ℙ^{1} of possibilities for it. If 𝔼_{1} = Im V^{2}, then V is zero on 𝔼_{3}/𝔼_{1} and we can choose 𝔼_{2}/𝔼_{1} as an arbitrary subspace of 𝔼_{3} giving us a ℙ^{1} of choices for 𝔼_{2}. On the other hand, if 𝔼_{1} ≠ Im V^{2}, then 𝔼_{3}/𝔼_{1} has a Jordan block of size 2 and hence there is only one V -stable 1-dimensional subspace and thus the flag is determined by 𝔼_{1}. The conclusion is that the punctual flag space is the union of two ℙ^{1}’s meeting at a single point. The intersection point is the canonical filtration (which is a full flag) and one can show that the rest of the points on one component are flags of type [241] and the rest of the points on the other component are flags of type [315].

10 Pieri formulas

In this section we are going to apply a theorem of Pittie and Ram ([PR99]) to obtain a Pieri type formula for our strata. (It seems to be historically more correct to speak of Pieri-Chevalley type formulas, cf., [Ch94].) The main application of it will not be to obtain cycle class formulas, as Pieri formulas usually do not give formulas for individual strata but only for certain linear combinations. To us the principal use of these formulas will be that it shows that a certain strictly positive linear combination of the boundary components will be a section of an ample line bundle (or close to ample as one of the contributors to ampleness will be λ_{1} which is ample only on the Satake compactification). This will have as consequence affineness for the open strata as well as a connectivity result for the boundary of the closed ones. We shall see in Section 13 that there is also a Pieri formula for the classes of the E-O strata though we know very little about it.

In this section we are going to work with level structures. There are two reasons for this. The first one is that we are going to exploit the ampleness λ_{1} and even formulating the notion of ampleness for a Deligne-Mumford stack is somewhat awkward. The second is that one of the consequences of our considerations will be an irreducibility criterion for strata. Irreducibility for a stratum on A_{g} does not imply irreducibility for the same stratum on the space A_{g,n} of principally polarised abelian varieties with level n-structure, where always p |̸ n. In fact irreducibility for the level n case means irreducibility on A_{g} together with the fact that the monodromy group of the level n cover is the maximum possible. Hence in this section we shall use A_{g,n} but also some toroidal compactification Ã_{g,n} (cf., [FC90]). Everything we have said so far applies to this situation giving us in particular F_{g,n} and F̃_{g,n} but we have the extra property that for n ≥ 3 then Ã_{g,n} and hence F̃_{g,n} are smooth projective varieties.

We now introduce the classes ℓ_{i} := c_{1}(𝔼_{{i}}) for 1 ≤ i ≤ 2g in the Chow ring CH^{∗}(F_{g}). By self-duality of the flag 𝔼_{•} we have that ℓ_{2g+1−i} = −ℓ_{i} and by construction c_{1}(𝔻_{{i}}) = pℓ_{i−g} = −pℓ_{3g+1−i} for g + 1 ≤ i ≤ 2g. (For the notation D_{J} see Subsection 3.2.) Furthermore, ℓ_{1} + ⋯ + ℓ_{g} is the pullback from Ã_{g} of λ_{1}, the first Chern class of the Hodge bundle.

Now we let M_{i} := c_{1}(𝔻_{]2g−i,2g]}), 1 ≤ i ≤ g, and start by noting that if n = (n_{1}, …, n_{g}) then n · M := ∑_{i} n_{i}M_{i} is relatively ample for F̃_{g} → Ã_{g} if n_{i} > 0 for 1 ≤ i < g. Indeed, by construction L_{i} := ℓ_{2g} + ⋯ + ℓ_{2g−i+1}, 1 ≤ i ≤ g is the pullback from the partial flag space F̃_{g}[i] of flags with elements of rank i, 2g − i, and g (and with the rank g-component equal to 𝔼_{g}) and on F̃_{g}[i] we have that ℓ_{2g} + ⋯ + ℓ_{2g−i+1} is ample. It is well-known that any strictly positive linear combination of these elements is relatively ample. From the formulas above we get that M_{i} = p(L_{g−i} + λ_{1}) (where we put L_{0} = 0). On the other hand λ_{1} is almost ample; it is the pullback from A_{g}^{∗} of an ample line bundle.

Now we identify the L_{i} with the fundamental weights of the root system of C_{g}. Note that W_{g} acts on the ℓ_{i} considered as parts of the weight lattice by σ(ℓ_{i}) = ℓ_{σ(i)} (keeping in mind that ℓ_{2g+1−i} = −ℓ_{i}) and then acts accordingly on the L_{i}. Let us also note that (by Chevalley’s characterisation of the Bruhat-Chevalley order) if w′ < w with ℓ(w′) = ℓ(w) − 1 and if w = s_{i_{1}} ⋯ s_{i_{k}}, then w′ is of the form s_{i_{1}} ⋯ ŝ_{i_{r}} ⋯ s_{i_{k}} which can be rewritten as ws_{α}, where s_{α} = (s_{i_{r+1}} ⋯ s_{i_{k}})^{−1}s_{i_{r}}(s_{i_{r+1}} ⋯ s_{i_{k}}) which thus is the reflection with respect to a unique positive root.

Theorem 10.1. For each 1 ≤ i ≤ g and w ∈ W_{g} we have that

(pλ_{1} + pL_{g−i} − wL_{i})[\overline{U}_{w}] = ∑_{w′≺w} c_{w,w′}^{i}[\overline{U}_{w′}] ∈ CH_{ℚ}^{1}(\overline{U}_{w} ⨂ \overline{𝔽}_{p}),

where c_{w,w′}^{i} ≥ 0 and w′ ≺ w means w′ ≤ w and ℓ(w) = ℓ(w′)+1. Furthermore, c_{w,w′}^{i} > 0 precisely when w′ = ws_{α} for α a positive root for which the simple root α_{i} appears with a strictly positive coefficient when α is written as a linear combination of the simple roots.

Proof. We shall use [PR99] which has the following setup: We fix a semi-simple algebraic group G (which in our case is the symplectic group Sp_{2g} but using this in the notation will only confuse) with Borel group B and fix a principal B-bundle E → X over an algebraic variety X. Letting E(G/B) → X be the associated G/B-bundle we have, because its structure group is B and not just G, Schubert varieties Ω_{w} → X (which fibre by fibre are the usual Schubert varieties). For every weight λ ∈ P, P being the group of weights for G, we have two line bundles on E(G/B); on the one hand y^{λ} obtained by regarding λ as a character of B which gives a G-equivariant line bundle on G/B and hence a line bundle on E(G/B), on the other hand the character λ can also be used to construct, with the aid of the principal B-bundle E, a line bundle x^{λ} on X and then by pullback to E(G/B) a line bundle also denoted by x^{λ}. A result of [PR99, Corollary] then says that if λ is a dominant weight then

y^{λ}[O_{Ω_{w}}] = ∑_{η∈T_{w}^{λ}} x^{η(1)}[O_{Ω_{v(η,w)}}] ∈ K_{0}(E(G/B)).  (2)

Here T_{w}^{λ} is a certain set of piecewise linear paths η: [0, 1] → P ⨂ ℝ in the real vector space spanned by P; moreover, v(η, w) is a certain element in the Weyl group of G which is always ≤ w, and T_{w}^{λ} has the property that η(1) ∈ P for all its elements η. An important property of T_{w}^{λ} is that it only depends on w and λ and not on E. It follows immediately from the description of [PR99], that v(η, w) = w in only one case, namely when η is the straight line η(t) = twλ. Hence we can rewrite the formula as

(y^{λ} − x^{wλ})[O_{Ω_{w}}] = ∑_{η∈T_{w}^{λ}}′x^{η(1)}[O_{Ω_{v(η,w)}}],

where the sum now runs over all elements of T_{w}^{λ} for which v(η, w) < w. Taking Chern characters and looking at the top term which appears in codimension codim(w) + 1 we get

(c_{1}(y^{λ}) − c_{1}(x^{wλ}))[Ω_{w}] = ∑_{η∈T_{w}^{λ}}′′[Ω_{v(η,w)}],  (3)

where the sum is now over the elements of T_{w}^{λ} for which ℓ(v(η, w)) = ℓ(w)−1. To determine the multiplicity with which a given [Ω_{w′}] appears in the right hand side we could no doubt use the definition of T_{w}^{λ}. However, it seems easier to note that that multiplicity is independent of E and hence we may assume that X is a point and by additivity in λ that λ is a fundamental weight λ_{i}. In that case one can use a result of Chevalley ([Ch94, Prop. 10]) to get the description of the theorem. However, we want this formula to be true not in the Chow group of E(G/B) but instead in the Chow group of the relative Schubert subvariety of index w of E(G/B). This however is no problem as the (relative) cell decomposition shows that this Chow group injects into the Chow group of E(G/B).

We now would like to claim Formula 3 in the case when X = F_{g} and the principal B-bundle tautological bundle 𝔼_{•}. The deduction of (3) from 2 is purely formal but however [PR99] only claim (2) for X a smooth variety over ℂ. This at least allows us to conclude (3) for X = F_{g,n} ⨂ ℂ, the flag space associated to a smooth toroidal compactification of A_{g,n}, the moduli space of principally polarised abelian varieties with a principal level n-structure for n ≥ 3 (see [FC90, Thm 6.7,Cor 6.9]). By then using the specialisation map for the Chow group we conclude that (3) is valid for X = F_{g,n} ⨂ \overline{𝔽}_{p}. Finally, pushing down under the map F_{g,n} → F_{g} induced from the map \overline{A}_{g,n} → \overline{A}_{g} we get (3) for X = F_{g}.

The final step is to pull back (3) along the section of E(G/B) given by 𝔻_{•}. To make the pullback possible (note that the relative Schubert variety will in general not be smooth over the base) we remove the relative Schubert varieties of codimension 2 in the relative Schubert variety in question. This forces us to remove the part of \overline{U}_{w} where the section encounters the removed locus. This is however a codimension 2 subset by Corollary 8.4 so its removal will not affect CH_{ℚ}^{1}(\overline{U}_{w}). Unraveling the pullback of (c_{1}(y^{λ}) − c_{1}(x^{wλ})) gives the theorem. ⊓ ⊔

To apply the theorem we start by some preliminary results which will be used to exploit the positivity of the involved line bundles.

Lemma 10.2. Let X be a proper (irreducible) variety of dimension > 1 and L a line bundle on X which is ample on some open subset U ⊆ X. Let D := X\U and let H ⊂ U be the zero set of a section of L_{|U}. If D is connected then so is D ∪ H.

Proof. By replacing the section by a power of it, we may assume that L is very ample giving an embedding U ↪ ℙ^{n}. Let Z be the closure of the graph of this map in X ×ℙ^{n}; moreover, let Y be the image of Z under the projection on the second factor giving us two surjective maps X ← Z → Y and let D′ be the inverse image in Z of D. Assume that D ∪ H is the disjoint union of the non-empty closed subsets A and B and let A′ and B′ be their inverse images in Z. Now, Y is irreducible of dimension > 1 and hence H′′ is connected, where H′′ is the hyperplane section of Y corresponding to H, so that the images of A′ and B′ in Y must meet. However, outside of D′ the map Z → Y is a bijection and hence the meeting point must lie below a point of D′ and hence A′ and B′ both meet D′. This implies that A and B both meet D which is a contradiction as D is assumed to be connected. ⊓ ⊔

Proposition 10.3. Let L be the determinant det 𝔼 of the Hodge bundle over A_{g,n}, n ≥ 3 (and prime to p).

i) There is, for each 1 ≤ i < g, an integer m_{i} such that the global sections of Λ^{g−i}(ℍ/𝔼) ⨂ L^{⊗m} generate this bundle over A_{g,n} whenever m ≥ m_{i}. These m_{i} can be chosen independently of p (but depending on g and n).

ii) Putting N_{i} := L_{i} + n_{i}λ_{1} for 1 ≤ i < g and N_{g} := λ_{1}, then ∑_{i} m_{i}N_{i} is ample on F_{g} if m_{i} > 0 for all 1 ≤ i ≤ g.

iii) Fix w ∈ W_{g} and put L := ∑_{i<g} L_{i}, N := ∑_{i<g} N_{i}, and m = ∑_{i} m_{i}. Choose r, s, t, and u such that rN + tλ_{1} − wL resp. sN + uλ − wL_{g} can be written as a positive linear combination of the N_{i} (using that L_{g} = −λ_{1}). Then if p > r + sm and (g − 1)p > t + um we have that p(L + (g − 1)λ_{1}) + pmλ_{1} − wL − mwL_{g} is ample on F_{g,n}. The constants r, s, t, and u can be chosen independently of p.

Proof. Starting with i) it follows directly from the fact that λ_{1} is ample on A_{g,n}. The independence on p follows from the existence of a model of A_{g,n} that exists over Spec ℤ[1/n, ζ_{n}].

As for ii) we have that π_{∗}O(L_{i}) = Λ^{g−i}(ℍ/𝔼), 1 ≤ i < g, as π: F_{g,n} → A_{g,n} can be identified with the space of flags on ℍ/𝔼 and O(L_{i}) is det(ℍ/𝔼_{2g−i}). By definition we then have that π_{∗}O(N_{i}) is generated by global sections on A_{g,n}. We know that on the flag space SL_{g} /B we have that the canonical ring ⨁_{λ} H^{0}(SL_{g} /B, L_{λ}), where λ runs over the dominant weights and L_{λ} is the corresponding line bundle, is generated by the H^{0}(SL_{g} /B, L_{λ_{i}}), 1 ≤ i < g, where λ_{i} is the i’th fundamental weight (see for instance [RR85]). Also O(∑_{i<g} m_{i}N_{i}) is relatively very ample and we have just shown that π_{∗}O(∑_{i<g} m_{i}N_{i}) is generated by global sections. As λ_{1} is ample on A_{g,n} we get that π_{∗}O(∑_{i≤g} m_{i}N_{i}) is ample.

Continuing with iii) we have that N = L + nλ_{1} which gives p(L + (g − 1)λ_{1}) + pmλ_{1} − wL − mwL_{g} = (p − (r + sm))N + (p(g − 1) − (t + um)) + (rN + tλ_{1} − wL) + m(sN + uλ_{1} − wL_{g}). We then conclude by the definitions of r, s, t, and u and ii). ⊓ ⊔

Remark 10.4. i) The constants r, s, t, and u are quite small and easy to compute. We know nothing about the m_{i} but imagine that they would not be too large.

ii) It would seem that the last part would not be applicable for g = 1 but it can be easily modified to do so. On the other hand for g = 1 everything is trivial anyway.

We are now ready for the first application of the Pieri formula.

Proposition 10.5. i) There is a bound depending only on g and n such that if p is larger than that bound, then for an irreducible component Z of some U_{w} ⊆ F̃_{g,n}, w ∈ W_{g}, the union of the complement of Z in \overline{Z}, the closure of Z in F̃_{g,n}, and the intersection of \overline{Z} and F̃_{g,n} \ F_{g,n} is connected if the intersection of \overline{Z} with the boundary F̃_{g,n} \ F_{g,n} is connected or empty.

ii) There is a bound depending only on g and n such that if p is larger than that bound, then for w ∈ W_{g} of semi-simple rank 0 we have that U_{w} is affine.

Proof. By Proposition 10.3 there is a bound depending only on g and n such that if p is larger than it, then M := p(L + (g − 1)λ_{1}) + pmλ_{1} − wL − mwL_{g} is ample on F_{g,n}. Summing up Pieri’s formula (Theorem 10.1) for 1 ≤ i < g and m times the formula for i = g we get that M[\overline{Z}] is supported on \overline{Z} intersected with smaller strata. (Note that the Pieri formula is a priori – and quite likely in reality – only true modulo torsion. We may however simply multiply it with a highly divisible integer and that doesn’t change the support.) We then conclude by Lemma 10.2.

As for the second part, we argue as in the first part and conclude that \overline{U}_{w} \ U_{w} is the support of an ample divisor in \overline{U}_{w} (as in the theorem each component must appear as we are summing up for 1 ≤ i ≤ g and some α_{i} must appear in the expansion of α) and hence U_{w} is affine. ⊓ ⊔

Remark 10.6. i) The first part of the proposition is somewhat difficult to use because of the condition on the intersection with the boundary. In the applications of the next section it turns out that we only need to apply it when the intersection is empty.

ii) For the second part we would like to say more generally that the image of U_{w} is affine in F_{g,n}^{∗} for some appropriate definition of F_{g,n}^{∗} analogous to the Satake compactification. The problem is that it doesn’t seem as if some power of M would normally be generated by its global sections so that we can not define F_{g,n}^{∗} as the image of F̃_{g,n}.

11 Irreducibility properties

In this section we shall prove irreducibility of a large class of strata and also that if the characteristic is large enough and our irreducibility criterion is not fulfilled then (with some extra conditions on the stratum) the stratum is reducible. Our proofs show two advantages of working on the flag spaces. The major one is that our strata are normal so that irreducibility follows from connectedness. The connectedness of the closed E-O strata except \overline{V}_{1} is proven in [Oo01] but the \overline{V}_{w} are most definitely not locally connected and hence that does not say very much about the irreducibility. In the converse direction we also make use of the Pieri formula.

Definition-Lemma 11.1. Let {Z_{α}} be a stratification of a Deligne-Mumford stack X of finite type over a field by which we mean that the strata Z_{α} are locally closed reduced substacks of X such that the closure \overline{Z_{α}} of a stratum is the union of strata. By the k-skeleton of the stratification we mean the union of the strata of dimension ≤ k (which is a closed substack). The boundary of a stratum Z_{α} is the complement of Z_{α} in its closure. Assume furthermore that each Z_{α} is irreducible and that for Z_{α} of dimension strictly greater than some fixed N we have that its boundary is connected (and in particular non-empty). Then the intersection of a connected union Z of closed strata \overline{Z_{α}} with the N-skeleton is connected.

Proof. It is enough, by induction to prove that if we remove a stratum Z′ from Z, whose dimension is maximal and > N, then the result remains connected. Assume that Z \ Z′ is the disjoint union of two closed subschemes Z_{1} and Z_{2}. By assumption the boundary of Z′ is connected and hence lies in Z_{1} say. That means that Z′ ∪ Z_{1} is closed and disjoint from Z_{2} which by the connectedness of Z implies that Z_{2} is empty. ⊓ ⊔

Proposition 11.2. There is a bound depending only on g and n such that the following is true if p is larger than that bound:

Let X ⊆ F̃_{g,n} be a connected union of irreducible components of closed strata \overline{U}_{w} (for possibly different w) which lie inside of F_{g,n}. Then the intersection of X with the 1-skeleton of the stratification is connected.

Proof. This follows directly from Proposition 10.5 and Lemma 11.1. (Note that for level 1 or 2 we may pass to a higher level in order to apply the proposition.) ⊓ ⊔

We now want to interpret this Proposition (and its converse which will be true for any p) in arithmetical terms. Hence we define the 1-skeleton graph of level n as the following edge-colored graph: Its vertices are the points of \overline{U}_{1} ⊂ F_{g,n}, i.e., isomorphism classes of principally polarised superspecial g-dimensional abelian varieties A together with a level n structure and a complete flag 0 = 𝔻_{0} ⊂ 𝔻_{1} ⊂ ⋯ ⊂ 𝔻_{g} = H^{0}(A, Ω_{A}^{1}) on H^{0}(A, Ω_{A}^{1}) for which 𝔻_{g−i}^{⊥} = V^{−1}𝔻_{i}^{(p)}. For each 1 ≤ i ≤ g we connect two vertices by an edge of color i if there is an irreducible component of \overline{U}_{s_{i}} which contains them.

Lemma 11.3. If S ⊆ {1, …, g} has the property that it contains g and for every 1 ≤ i < g we have that either i or g − i belongs to S, then the subgraph of the 1-skeleton graph consisting of all vertices and all edges of colours i ∈ S is connected.

Proof. This follows from [Oo01, Prop. 7.3] and Theorem 7.5. ⊓ ⊔

For a subset S ⊆ {1, …, g} the S-subgraph of the 1-skeleton graph is the subgraph with the same vertices and with only the edges whose colour is in S. This definition allows us to formulate our irreducibility conditions.

Theorem 11.4. i) Let w ∈ W_{g} and let S := {1 ≤ i ≤ g : s_{i} ≤ w }. If the S-subgraph of the 1-skeleton graph is connected then \overline{U}_{w} ⊆ A_{g} is irreducible.

ii) There is a bound depending only on g such that if p is larger than that bound the following is true: If w ∈ W_{g} is admissible and either final or of semi-simple rank 0 and if S := {1 ≤ i ≤ g : s_{i} ≤ w }, then there is a bijection between the irreducible components of \overline{U}_{w} ⊆ A_{g} and the connected components of the 1-skeleton graph.

Proof. The first part is clear as Proposition 6.1 says that each connected component meets \overline{U}_{1} and then by the assumption on connectedness of the S-subgraph \overline{U}_{w} is connected; but by Corollary 8.4 it is normal and hence is irreducible.

As for the second part assume first that w is final but of positive semisimple rank. That means that its Young diagram does not contain a row of length g and hence by Lemma 2.7 and the Chevalley characterisation of the Bruhat-Chevalley order we have that s_{i} ≤ w for all 1 ≤ i ≤ g and hence the S-subgraph is connected by Lemma 11.3 which make the statement trivially true. We may therefore assume that the semi-simple rank is 0 and hence that \overline{U}_{w} lies entirely in F_{g,n}. In that case the result follows from Proposition 11.2 and the fact that two irreducible components of two \overline{U}_{s_{i}} meet only at \overline{U}_{1}. ⊓ ⊔

Projecting down to A_{g} we get the following corollary which shows irreducibility for many E-O strata.

Theorem 11.5. Let w ∈ W_{g} be a final element whose Young diagram Y has the property that there is an ⌈(g + 1)/2⌉ ≤ i ≤ g such that Y does not contain a row of length i. Then \overline{V}_{w} is irreducible and the total space of the étale cover U_{w} → V_{w} is connected.

Proof. This follow from Theorem 11.4 and Lemmas 2.7 and 11.3. ⊓ ⊔

Example 11.6. For g = 2 the locus of abelian surfaces of p-rank ≤ 1 is irreducible. For g = 3 all E-O strata except the superspecial locus (Y = {1, 2, 3}) and the Moret-Bailly locus (Y = {2, 3}) are irreducible.

In [Ha07] S. Harashita has proved that the number of irreducible components of an E-O stratum that is contained in the supersingular locus is given as a class number and as a consequence that, except possibly for small p, these strata are reducible. As has been proved by Oort (cf., [Ha07, Prop. 5.2]) these strata are exactly the ones to which Theorem 11.5 does not apply.

We shall finish this section by showing that the 1-skeleton graph can be described in purely arithmetic terms very strongly reminiscent of the results of Harashita. Note that even for final elements our results are not formally equivalent to Harashita’s as we are dealing with the set of components of the final strata in F_{g} whereas Harashita is dealing with their images in A_{g}. In any case our counting of the number of components uses Theorem 11.4 and hence is valid only for sufficiently large p whereas Harashita’s are true unconditionally.

We start by giving a well-known description of the vertices of the 1-skeleton graph (see for instance [Ek87]) valid when g > 1. We fix a supersingular elliptic curve E and its endomorphism ring D which is provided with the Rosati involution ∗. To simplify life we assume, as we may, that E is defined over 𝔽_{p} and hence D contains the Frobenius map F. It has the property that DF = F D = DF D, the unique maximal ideal containing p. Furthermore we have that D/DF ≅ 𝔽_{p^{2}}. There is then a bijection between isomorphism classes of D-lattices M (i.e., right modules torsion free and finitely generated as abelian groups) of rank g (i.e., of rank 4g as abelian groups) and isomorphism classes of g-dimensional abelian varieties A. The correspondence associates to the abelian variety A the D-module Hom(E, A). Polarisations on A then correspond to positive definite unitary forms, i.e., a bi-additive map ⟨−, −⟩: M × M → D such that ⟨md, n⟩ = ⟨m, n⟩dn, ⟨n, m⟩ = (⟨m, n⟩)^{∗}, and m ≠ 0 ⇒ ⟨m, m⟩ > 0. The polarisation is principal precisely when the form is perfect, i.e., the induced map of right D-modules M → Hom_{D}(M, D) given n ↦ (m ↦ ⟨m, n⟩) is an isomorphism. In general we put M^{∗} := Hom_{D}(M, D) and then the form induces an embedding M → M^{∗} which makes the image of finite index. More precisely we on M ⨂ ℚ we get a non-degenerate pairing with values in D ⨂ ℚ and then we may identify M^{∗} with the set {n ∈ M ⨂ ℚ : ∀m ∈ M: ⟨m, n⟩ ∈ D }. Using this we get a D ⨂ ℚ/ℤ-valued unitary perfect form on M^{∗}/M given by ⟨\overline{m}, \overline{n}⟩ := ⟨m, n⟩ mod D. As usual super-lattices M ⊆ N over D correspond to totally isotropic submodules of M^{∗}/M.

If now S ⊆ {0, …, g} is stable under i ↦ g − i then an arithmetic S-flag consists of the choice of unitary forms on D-modules M_{i} of rank g for i ∈ S and compatible isometric embeddings M_{i} ↪ M_{j} whenever i < j fulfills the following conditions:

• For all i ∈ S with i ≥ g/2 we have that M_{i}^{∗}/M_{i} is killed by F of D and can hence be considered as an D/m = 𝔽_{p^{2}}-vector space with a perfect unitary form.

• We have that F M_{i}^{∗} = M_{g−i} for all i ∈ S.

• The length of M_{j}/M_{i} for i < j is equal to j − i.

Remark 11.7. i) Note that we allow S to be empty in which case there is exactly one S-flag.

ii) As follows (implicitly) from the proof of next proposition, the isomorphism class of an element of an arithmetic S-flag tensored with ℚ is independent of the flag. Hence we may consider only lattices in a fixed unitary form over D ⨂ ℚ and then think of M_{i} as a sublattice of M_{j}.

Proposition 11.8. Let S ⊆ \{1, …, g\} and let \overline{S} ⊆ \{0, …, g\} be the set of integers of the form i or g − i for i ∈ S. Then the set of isomorphism classes of \overline{S}-flags is in bijection with the set of connected components of the S-subgraph of the 1-skeleton graph.

Proof. This follows from the discussion above and Theorem 7.5 once we have proven that an S-flag can be extended to an {0, …, g}-flag. Assume first that g ∉ S and let i ∈ S be the largest element in S. By assumption we have F M_{i}^{∗} = M_{g−i} ⊆ M_{i} ⊂ M_{i}^{∗} and we have that the length of M_{i}^{∗}/F M_{i}^{∗} is g, whereas again by assumption that of M_{i}/M_{g−i} is 2i − g. Together this gives that the length of M_{i}^{∗}/M_{i} is g − (2i − g) = 2(g − i). As the form on M_{i}^{∗}/M_{i} is a non-degenerate unitary 𝔽_{p^{2}}-form and as all such forms are equivalent we get that there is a g − i-dimensional totally isotropic (and hence its own orthogonal) subspace of M_{i}^{∗}/M_{i}; this then gives an M_{i} ⊂ M_{g} ⊂ M_{i}^{∗} and as M_{g}/M_{i} is its own orthogonal we get that the pairing on M_{g} is perfect. We then put M_{0} := F M_{g} and the rest of the flag extension is immediate. ⊓ ⊔

12 The Cycle Classes

If one wishes to exploit our stratification on F_{g} and the E-O stratification on A_{g} fully, one needs to know the cohomology classes (or Chow classes) of the (closed) strata. In this section we show how to calculate these classes.

The original idea for the determination of the cycle classes can be illustrated well by the p-rank strata. If X is a principally polarised abelian variety of dimension g which is general in the sense that its p-rank is g, then its kernel of multiplication by p contains a direct sum of g copies of μ_{p}, the multiplicative group scheme of order p. The unit tangent vector of μ_{p} gives a tangent vector to X at the origin. By doing this in the universal family we thus see that on a suitable level cover of the moduli we have g sections of the Hodge bundle over the open part of ordinary abelian varieties. If the abelian variety looses p-rank under specialization the g sections thus obtained become dependent and the loci where this happens have classes represented by a multiple of the Chern classes of the Hodge bundle.

To calculate the cycle classes of the E-O strata on A_{g} ⊗𝔽_{p} we shall use the theory of degeneration cycles of maps between vector bundles. To this end we shall apply formulas of Fulton for degeneracy loci of symplectic bundle maps to calculate the classes of the \overline{U}_{w} and formulas of Pragacz and Ratajski and of Kresch and Tamvakis for calculating those of \overline{V}_{ν}.

12.1 Fulton’s formulas

Over the flag space F_{g} we have the pull back of the de Rham bundle and the two flags 𝔼_{•} and 𝔻_{•} on it. We denote by ℓ_{i} the roots of the Chern classes of 𝔼 so that c_{1}(𝔼_{i}) = ℓ_{1} + ℓ_{2} + … + ℓ_{i}. We then have c_{1}(𝔻_{g+i}) − c_{1}(𝔻_{g+i−1}) = pℓ_{i}.

Recall (cf., Subsection 4.1 and Section 5) that for each element w ∈ W_{g} we have degeneracy loci \overline{U}_{w} in F_{g} resp. F̃_{g}. Their codimensions equal the length ℓ(w) and it thus makes sense to consider the cycle class u_{w} = [\overline{U}_{w}] in CH_{ℚ}^{codim(w)}(F̃_{g}), where we write F_{g} instead of F_{g} ⊗ 𝔽_{p}.

Fulton’s setup in [Fu96] is the following (or more precisely the part that interests us): We have a symplectic vector bundle H over some scheme X and two full symplectic flags 0 ⊂ ⋯ ⊂ E_{2} ⊂ E_{1} = H and 0 ⊂ ⋯ ⊂ D_{2} ⊂ D_{1} = H. For each w ∈ W_{g} one defines the degeneracy locus \overline{U}_{w} by {x ∈ X : ∀i, j: dim(𝔼_{i,x} ∩ 𝔻_{j,x}) ≤ r_{w}(i, j) } (of course this closed subset is given a scheme structure by considering these conditions as rank conditions for maps of vector bundles). Fulton then defines a polynomial in two sets of variables x_{i} and y_{j}, i, j = 1, …, g, such that if this polynomial is evaluated as x_{i} = c_{1}(E_{i}/E_{i+1}) and y_{j} = c_{1}(D_{j}/D_{j+1}), then it gives the class of \overline{U}_{w} provided that \overline{U}_{w} has the expected codimension codim(w) (and X is Cohen-Macaulay). The precise definition of these polynomials are as follows: For a partition μ = {μ_{1} > μ_{2} > … > μ_{r} > 0} with r ≤ g and μ_{1} ≤ g one defines a Schur function

Δ_{μ}(x) := det(x_{μ_{i}+j−i})_{1≤i,j≤r}

in the variables x_{i} and puts

Δ(x, y) := Δ_{(g,g−1,...,1)}(σ_{i}(x_{1}, …, x_{g}) + σ_{i}(y_{1}, …, y_{g})),

where the σ_{i} is the i’th elementary symmetric function. One then considers the “divided difference operators” ∂_{i} on the polynomial ring ℤ[x_{1}, …, x_{g}] by

∂_{i}(F (x)) = \begin{cases}\frac{F (x)−F (s_{i}x)}{x_{i}−x_{i+1}} & if i < g, \\ \frac{F (x)−F (s′_{g}x)}{2x_{g}} & if i = g, \end{cases}

where s_{i} interchanges x_{i} and x_{i+1} for i = 1, …, g − 1, but s′_{g} sends x_{g} to −x_{g} and leaves the other x_{i} unchanged. We write an element w ∈ W_{g} as a product w = s_{i_{ℓ}}s_{i_{ℓ−1}} … s_{i_{1}} with ℓ = ℓ(w) and set

P_{w} := ∂_{i_{ℓ}} ⋯ ∂_{i_{1}}( ∏_{i+j≤g}(x_{i} − y_{j}) · Δ).  (4)

An application of Fulton’s formulas gives the following.

Theorem 12.1. Let w = s_{i_{ℓ}}s_{i_{ℓ−1}} … s_{i_{1}} with ℓ = ℓ(w) be an element of the Weyl group W_{g}. Then the cycle class u_{w} := [\overline{U}_{w}] in CH_{ℚ}^{codim(w)}(F̃_{g}) is given by

u_{w} = ∂_{i_{1}} ⋯ ∂_{i_{ℓ}} ( ∏_{i+j≤g}(x_{i} − y_{j}) · Δ(x, y))_{|x_{i}=−ℓ_{i},y_{j}=pℓ_{j}} .

Proof. By construction \overline{U}_{w} is the degeneracy locus of the flags 𝔼_{•} and 𝔻_{•}. By Corollary 8.4 they are Cohen-Macaulay and have the expected dimension and hence the degeneracy cycle class is equal to the class of \overline{U}_{w}. ⊓ ⊔

For a final element w ∈ W_{g} the map \overline{U}_{w} → \overline{V}_{w} is generically finite of degree γ_{g}(w). By applying the Gysin map to the formula of Theorem 12.1 using formula 3.1 we can in principle calculate the cohomology classes of all the push-downs of final strata, hence of the E-O strata.

Example 12.2. g = 2.

The Weyl group W_{2} consists of 8 elements; we give the cycle classes in F̃_{2} and the push-downs on Ã_{2}.

\begin{matrix}w & s & ℓ & [\overline{U}_{w}] & π_{∗}([\overline{U}_{w}]) \\ [4, 3] & s_{1}s_{2}s_{1}s_{2} & 4 & 1 & 0 \\ [4, 2] & s_{1}s_{2}s_{1} & 3 & (p − 1)λ_{1} & 0 \\ [3, 4] & s_{2}s_{1}s_{2} & 3 & −ℓ_{1} + pℓ_{2} & 1 + p \\ [2, 4] & s_{1}s_{2} & 2 & (1 − p)ℓ_{1}^{2} + (p^{2} − p)λ_{2} & (p − 1)λ_{1} \\ [3, 1] & s_{2}s_{1} & 2 & (1 − p^{2})ℓ_{1}^{2} + (1 − p)ℓ_{1}ℓ_{2} + (1 − p)ℓ_{2}^{2} & p(p − 1)λ_{1} \\ [2, 1] & s_{1} & 1 & (p − 1)(p^{2} + 1)λ_{1}λ_{2} & 0 \\ [1, 3] & s_{2} & 1 & (p^{2} − 1)ℓ_{1}^{2}(ℓ_{1} − pℓ_{2}) & (p − 1)(p^{2} − 1)λ_{2} \\ [1, 2] & 1 & 0 & −(p^{4} − 1)λ_{1}λ_{2}ℓ_{1} & (p^{4} − 1)λ_{1}λ_{2} \end{matrix}

In the flag space F_{2} the stratum corresponding to the empty diagram is U_{s_{2}s_{1}s_{2}} and the strata contained in its closure are the four final ones U_{s_{2}s_{1}s_{2}}, U_{s_{1}s_{2}}, U_{s_{2}} and U_{1} and the two non-final ones U_{s_{1}} and U_{s_{2}s_{1}}. The Bruhat-Chevalley order on these is given by the diagram:

[figure: s_{2}s_{1}s_{2}; s_{2}s_{1}; s_{1}s_{2}; s_{1}; s_{2}; 1]

The four final strata U_{s_{2}s_{1}s_{2}}, U_{s_{1}s_{2}}, U_{s_{2}} and U_{1} lie étale of degree 1 over the p-rank 2 locus, the p-rank 1 locus, the locus of abelian surfaces with p-rank 0 and a-number 1, and the locus of superspecial abelian surfaces (a = 2). The locus U_{s_{1}} is an open part of the fibres over the superspecial points. The locus U_{s_{2}s_{1}} is of dimension 2 and lies finite but inseparably of degree p over the p-rank 1 locus. Then E_{1} corresponds to an α_{p} and E_{2}/E_{1} to a μ_{p}. In the final type locus U_{s_{1}s_{2}} the filtration is μ_{p} ⊂ μ_{p} ⊕ α_{p}. Note that this description is compatible with the calculated classes of the loci.

We have implemented the calculation of the Gysin map in Macaulay2 (cf., [M2]) and calculated all cycle classes for g ≤ 5. For g = 3, 4 the reader will find the classes in the appendix. (The Macaulay2 code for performing the cal-

culations can be found at http://www.math.su.se/~teke/strata.m2.) We

shall return to the qualitative consequences one can draw from Theorem 12.1 in the next section.

12.2 The p-rank strata

It is very useful to have closed formulas for the cycle classes of important strata. We give the formulas for the strata defined by the p-rank and by the a-number. The formulas for the p-rank strata can be derived immediately from the definition of the strata.

Let V_{f} be the closed E-O stratum of Ã_{g} of semi-abelian varieties of p-rank ≤ f. It has codimension g − f. To calculate its class we consider the element w_{∅}, the longest final element. The corresponding locus \overline{U}_{∅} is a generically finite cover of A_{g} of degree γ_{g}(w_{∅}) = ∏_{i=1}^{g−1}(p^{i} + p^{i−1} + … + 1). The map of U_{∅} to the p-rank g locus is finite. The space \overline{U}_{∅} contains the degeneracy loci U_{w} for all final elements w ∈ W_{g}. The condition that a point x of F_{g} lies in \overline{U}_{∅} is that the filtration 𝔼_{i} for i = 1, …, g is stable under V. By forgetting part of the flag and considering flags 𝔼_{j} with j = i, …, g we find that \overline{U}_{∅} → A_{g} is fibered by generically finite morphisms

\overline{U}_{∅} = \overline{U}^{(1)}⟶^{π_{1}}\overline{U}^{(2)}⟶^{π_{2}} … ⟶^{π_{g−1}}\overline{U}^{(g)} = A_{g}.

We shall write π_{i,j} for the composition π_{j}π_{j−1} ⋯ π_{i}: \overline{U}^{(i)} → \overline{U}^{(j)} and π_{∅} = π_{1,g}.

Since V_{g−1} is given by the vanishing of the map det(V): det(𝔼_{g}) → det(𝔼^{(p)}) the class of V_{g−1} is (p − 1)λ_{1}. The pull-back of V_{g−1} to \overline{U}_{∅} decomposes in g irreducible components

π_{∅}^{−1}(V_{g−1}) = ∪_{i=1}^{g}Z_{i},

where Z_{i} is the degeneracy locus of the induced map ϕ_{i} = V_{|L_{i}}: L_{i} → L_{i}^{(p)}. Note that the Z_{i} are the \overline{U}_{w} for the w which are shuffles of the final element u_{g−1} (see Subsection 4.2) defining the (open) E-O stratum of p-rank f and Z_{g} is the stratum corresponding to the element u_{g−1} ∈ W_{g}. An abelian variety of p-rank g − 1 (and thus with a-number 1) has a unique subgroup scheme α_{p}. The index i of Z_{i} indicates where this subgroup scheme can be found (i.e. its Dieudonné module lies in 𝔼_{i} but not in 𝔼_{i−1}).

It follows from the definition of Z_{i} as degeneracy set that the class of Z_{i} on \overline{U}_{∅} equals (p − 1)ℓ_{i} since ϕ_{i} can be interpreted as a section of L_{i}^{(p)} ⊗ L_{i}^{−1}. We also know by 4.3 that the map Z_{i} → Z_{i+1} is inseparable. Therefore π_{∅}([Z_{i}]) = p^{n(i)}π_{∅}([Z_{g}]) for some integer n(i) ≥ g − i. Using the fact that (π_{∅})_{∗}([Z_{g}]) = γ_{g}(u_{g−1})[V_{g−1}] = deg(π_{1,g−1})[V_{g−1}] we see that

(π_{∅})_{∗}(π_{∅}^{∗}([V_{g−1}]) = ∑_{i=1}^{g}(π_{∅})_{∗}([Z_{i}]) = ∑_{i=1}^{g} p^{n(i)} deg(π_{1,g−1})[V_{g−1}],

while on the other hand

(π_{∅})_{∗}(π_{∅}^{∗}([V_{g−1}]) = deg(π_{∅})[V_{g−1}] = (1 + p + … + p^{g−1}) deg(π_{1,g−1})[V_{g−1}].

Comparison yields that n(i) = g − i and so we find

(π_{∅})_{∗}(ℓ_{i}) = p^{g−i} deg(π_{1,g−1})λ_{1}

and

(π_{∅})_{∗}([Z_{i}]) = (p − 1)p^{g−i} deg(π_{1,g−1})λ_{1}.

Lemma 12.3. In the Chow groups with rational coefficients of \overline{U}^{(i)} and \overline{U}^{(i+1)} we have for the push down of the j’th Chern class λ_{j}(i) of 𝔼_{i} the relations:

π_{∗}^{i}λ_{j}(i) = p^{j}(p^{i−j} + p^{i−j−1} + ... + p + 1)λ_{j}(i + 1)

and

p^{f(g−f)} (π_{1,g})_{∗}(ℓ_{g}ℓ_{g−1} ⋯ ℓ_{f+1}) = (π_{1,g})_{∗}(ℓ_{1}ℓ_{2} ⋯ ℓ_{g−f}).

Proof. The relation (π_{1})_{∗}([Z_{1}]) = p [Z_{2}] translates into the case j = 1 and i = 1. Using the push-pull formula and the relations π_{i}^{∗}(λ_{j}(i + 1) = ℓ_{i+1}λ_{j−1}(i) + λ_{j}(i) the formulas for the push downs of the λ_{j}(i) follow by induction on j and i. ⊓ ⊔

We now calculate the class of all p-rank strata V_{f}.

Theorem 12.4. The class of the locus V_{f} of semi-abelian varieties of p-rank ≤ f in the Chow ring CH_{ℚ}(Ã_{g}) equals

[V_{f}] = (p − 1)(p^{2} − 1) ⋯ (p^{g−f} − 1) λ_{g−f},

where λ_{i} denotes the ith Chern class of the Hodge bundle.

Proof. The class of the final stratum \overline{U}_{u_{f}} on \overline{U}_{∅} is given by the formula

(p − 1)^{g−f}ℓ_{g}ℓ_{g−1} ⋯ ℓ_{f+1}

as it is the simultaneous degeneracy class of the maps ϕ_{j} for j = f + 1, …, g. By pushing down this class under π_{∅} = π_{1,g} we find using Lemma 12.3 and the notation λ_{j}(i) = c_{j}(𝔼_{i}) that

(π_{1,g})_{∗}(ℓ_{g}ℓ_{g−1} ⋯ ℓ_{f+1}) = p^{−f(g−f)}(π_{1,g})_{∗}(ℓ_{1}ℓ_{2} ⋯ ℓ_{g−f})
= p^{−f(g−f)} (π_{1,g})_{∗}(π_{1,g−f}^{∗}(λ_{g−f}(g − f)))

= p^{−f(g−f)} deg(π_{1,g−f}) (π_{g−f,g})_{∗}(λ_{g−f}(g − f)).

Applying Lemma 12.3 repeatedly we find

\begin{aligned}(π_{g−f,g})_{∗}(λ_{g−f}(g − f)) & = p^{f(g−f)}(1 + p)(1 + p + p^{2}) ⋯ (1 + … + p^{f−1})λ_{g} \\ & = p^{f(g−f)} γ_{g}(u_{f}) λ_{g}, \end{aligned}

with γ_{g}(u_{f}) the number of final filtrations refining the canonical filtration of u_{f}. Hence we get (π_{∅})_{∗}(\overline{U}_{u_{f}}) = (p − 1)^{g−f} deg(π_{1,g−f})γ_{g}(u_{f})λ_{g}. On the other hand, we have that (π_{∅})_{∗}(\overline{U}_{u_{f}}) = γ_{g}(u_{f}) [V_{f}]. All together these formulas prove the result. ⊓ ⊔

12.3 The a-number Strata

Another case where we can find attractive explicit formulas is that of the E-O strata V_{w} with w the element of W_{g} associated to Y = {1, 2, …, a}. We denote these by T_{a}. Here we can work directly on A_{g}. The locus T_{a} on A_{g} may be defined as the locus {x ∈ A_{g}: rank(V)|_{𝔼_{g}} ≤ g − a}. We have T_{a+1} ⊂ T_{a} and dim(T_{g}) = 0. We apply now formulas of Pragacz and Ratajski, cf. [PR97] for the degeneracy locus for the rank of a self-adjoint bundle map of symplectic bundles globalizing the results in isotropic Schubert calculus from [Pr91]. Before we apply their result to our case we have to introduce some notation.

Define for a vector bundle A with Chern classes a_{i} the expression

Q_{ij}(A) := a_{i}a_{j} + 2 ∑_{k=1}^{j}(−1)^{k}a_{i+k}a_{j−k} for i > j.

A subset β = {g ≥ β_{1} > … > β_{r} ≥ 0} of {1, 2, …, g} (with r even, note that β_{r} may be zero) is called admissible and for such subsets we set

Q_{β} = Pfaffian(x_{ij}),

where the matrix (x_{ij}) is anti-symmetric with entries x_{ij} = Q_{β_{i},β_{j}}. Applying the formula of Pragacz-Ratajski to our situation gives the following result:

Theorem 12.5. The cycle class [T_{a}] of the reduced locus T_{a} of abelian varieties with a-number ≥ a is given by

[T_{a}] = ∑_{β} Q_{β}(𝔼^{(p)}) · Q_{ρ(a)−β}(𝔼^{∗}),

where the sum is over the admissible subsets β contained in the subset ρ(a) = {a, a − 1, a − 2, ..., 1}.

Example 12.6.

[T_{1}] = (p − 1)λ_{1}
[T_{2}] = (p − 1)(p^{2} + 1)λ_{1}λ_{2} − (p^{3} − 1)2λ_{3}
…
[T_{g}] = (p − 1)(p^{2} + 1) … (p^{g} + (−1)^{g})λ_{1}λ_{2} ⋯ λ_{g}.

As a corollary we find a result of one of us (cf. [Ek87]) on the number of principally polarised abelian varieties with a = g.

Corollary 12.7. We have

∑_{X} \frac{1}{#Aut(X)} = (−1)^{g(g+1)/2}2^{−g}[ ∏_{j=1}^{g}(p^{j} + (−1)^{j})] · ζ(−1)ζ(−3) … ζ(1 − 2g),

where the sum is over the isomorphism classes (over 𝔽̄_{p}) of principally polarised abelian varieties of dimension g with a = g, and ζ(s) is the Riemann zeta function.

Proof. Combine the formula for T_{g} with the Hirzebruch-Mumford Proportionality Theorem (see [Ge99]) which says that

deg(λ_{1}λ_{2} ⋯ λ_{g}) = (−1)^{\frac{g(g+1)}{2}} ∏_{j=1}^{g} \frac{ζ(1 − 2j)}{2},

when interpreted for the stack A_{g}. ⊓ ⊔

The formulas for the cycles classes of the p-rank strata and the a-number strata can be seen as generalizations of the classical formula of Deuring (known as Deuring’s Mass Formula) which states that

∑_{E} \frac{1}{#Aut(E)} = \frac{p − 1}{24},

where the sum is over the isomorphism classes over 𝔽̄_{p} of supersingular elliptic curves. It is obtained from the formula for V_{g−1} or T_{1} for g = 1, i.e., [V_{1}] = (p − 1)λ_{1}, by remarking that the degree of λ_{1} is 1/12 times the degree of a generic point of the stack Ã_{1}.

One can obtain formulas for all the E-O strata by applying the formulas of Pragacz-Ratajski or those of Kresch-Tamvakis [KT02, Cor. 4]. If Y is a Young diagram given by a subset {ξ_{1}, …, ξ_{r}} we call |ξ| = ∑_{i=1}^{r} ξ_{i} the weight and r the length of ξ. Moreover, we need the excess e(ξ) = |ξ| − r(r + 1)/2 and the intertwining number e(ξ, η) of two strict partitions with ξ ∩ η = ∅ by

e(ξ, η) = ∑_{i≥1} i #{j : ξ_{i} > η_{j} > ξ_{i+1}}

(where we use ξ_{k} = 0 if k > r). We put ρ_{g} = {g, g − 1, …, 1} and ξ′ = ρ_{g} − ξ and have then e(ξ, ξ′) = e(ξ). The formula obtained by applying the result of Kresch and Tamvakis interpolates between the formulas for the two special cases, the p-rank strata and a-number strata, as follows:

Theorem 12.8. For a Young diagram given by a partition ξ we have

[\overline{V}_{Y}] = (−1)^{e(ξ)+|ξ′|} ∑_{α} Q_{α}(𝔼^{(p)}) ∑_{β}(−1)^{e(α,β)}Q_{(α∪β)′}(𝔼^{∗}) det(c_{β_{i}−ξ′_{j}}(𝔼_{g−ξ′_{j}}^{∗})),

where the sum is over all admissible α and all admissible β that contain ξ′ with length ℓ(β) = ℓ(ξ′) and α ∩ β = ∅.

12.4 Positivity of tautological classes

The Hodge bundle possesses certain positivity properties. It is well-known that the determinant of the Hodge bundle (represented by the class λ_{1}) is ample on A_{g}. Over ℂ this is a classical result, while in positive characteristic this was proven by Moret-Bailly [MB85]. On the other hand, the Hodge bundle itself is not positive in positive characteristic. For example, for g = 2 the restriction of 𝔼 to a line from the p-rank 0 locus is O(−1)⊕O(p), [MB81]. But our Theorem 12.4 implies the following non-negativity result.

Theorem 12.9. The Chern classes λ_{i} ∈ CH_{ℚ}(A_{g} ⊗ 𝔽_{p}) (i = 1, …, g) of the Hodge bundle 𝔼 are represented by effective classes with ℚ–coefficients.

13 Tautological rings

We shall now interpret the results of previous sections in terms of tautological rings. Recall that the tautological ring of Ã_{g} is the subring of CH_{ℚ}(Ã_{g}) generated by the Chern classes λ_{i}. To obtain maximal precision we shall use the subring and not the ℚ-subalgebra (but note that is still a subring of CH_{ℚ}(Ã_{g}) not of the integral Chow ring CH^{∗}(Ã_{g})). As a graded ring it is isomorphic to the Chow ring CH^{∗}(Sp_{2g} /P_{H}) and as an abstract graded ring it is generated by the λ_{i} with relations coming from the identity 1 = (1 + λ_{1} + ⋯ + λ_{g})(1 − λ_{1} + ⋯ + (−1)^{g}λ_{g}). This implies that it has a ℤ-basis consisting of the square free monomials in the λ_{i}. (Note however, that the degree maps from the degree g(g+1)/2 part are not the same; on Sp_{2g} /P_{H} the degree of λ_{1} … λ_{g} is ±1, whereas for Ã_{g} it is given by the Hirzebruch-Mumford proportionality theorem as in the previous section.) As F̃_{g} → Ã_{g} is an SL_{g} /B-bundle we can express CH_{ℚ}(F̃_{g}) as an algebra over CH_{ℚ}(Ã_{g}); it is the algebra generated by the ℓ_{i} and the relations are that the elementary symmetric functions in them are equal to the λ_{i}. This makes it natural to define the tautological ring of F̃_{g} to be the subring of CH_{ℚ}(F̃_{g}) generated by the ℓ_{i}. It will then be the algebra over tautological ring of Ã_{g} generated by the ℓ_{i} and with the relations that say that the elementary symmetric functions in the ℓ_{i} are equal to the λ_{i}. Again this means that the tautological ring for F̃_{g} is isomorphic to the integral Chow ring of Sp_{2g} /B_{g}, the space of full symplectic flags in a 2g-dimensional symplectic vector space. Note furthermore that the Gysin maps for Sp_{2g} /B_{g} → Sp_{2g} /P_{H} and F̃_{g} → Ã are both given by Formula 3.1.

Theorem 12.1 shows in particular that the classes of the \overline{U}_{w} and \overline{V}_{ν} lie in the respective tautological rings. However, we want to both compare the formulas for these classes with the classical formulas for the Schubert varieties and take into account the variation of the coefficients of the classes when expressed in a fixed basis for the tautological ring. Hence as the rest of this section is purely algebraic we shall allow ourselves the luxury of letting p temporarily be also a polynomial variable. We then introduce the ring ℤ{p} which is the localisation of the polynomial ring ℤ[p] at the multiplicative subset of polynomials with constant coefficient equal to 1. Hence evaluation at 0 extends to a ring homomorphism ℤ{p} → ℤ which we shall call the classical specialisation. An element of ℤ{p} is thus invertible precisely when its classical specialisation is invertible. By a modulo n consideration we see that an integer polynomial with 1 as constant coefficient can have no integer zero n ≠ ±1. That means in particular that evaluation at a prime p induces a ring homomorphism ℤ{p} → ℚ taking the variable p to the integer p which we shall call the characteristic p specialisation (as p will be an integer only when this phrase is used there should be no confusion because of our dual use of p). We now extend scalars of the two tautological rings from ℤ to ℤ{p} and we shall call them the p-tautological rings. We shall also need to express the condition that an element is in the subring obtained by extension to a subring of ℤ{p} and we shall then say that the element has coefficients in the subring. We may consider the Fulton polynomial P_{w} of (4) as a polynomial with coefficients in ℤ{p} and when we evaluate them on elements of the tautological ring as in Theorem 12.1 we get elements [\overline{U}_{w}] of the p-tautological ring of F̃_{g}. If ν is a final element we can push down the formula for [\overline{U}_{ν}] using (3.1) and then we get an element in the p-tautological ring of Ã_{g}. We then note that γ(w) is a polynomial in p with constant coefficient equal to 1 and hence we can define [\overline{V}_{ν}] := γ(w)^{−1}π_{∗}[\overline{U}_{ν}], where π: F̃_{g} → Ã_{g} is the projection map. By construction these elements map to the classes of \overline{U}_{w} resp. \overline{V}_{ν} under specialisation to characteristic p. We shall need to compare them with the classes of the Schubert varieties. To be specific we shall define the Schubert varieties of Sp_{2g} /B_{g} by the condition dim E_{i} ∩ D_{j} ≥ r_{w}(i, j), where D_{•} is a fixed reference flag (and then the Schubert varieties of Sp_{2g} /P_{H} are the images of the Schubert varieties of Sp_{2g} /B_{g} for final elements of W_{g}).

Theorem 13.1. i) The classes [\overline{U}_{w}] and [\overline{V}_{ν}] in the p-tautological ring of F̃_{g} map to the classes of the corresponding Schubert varieties under classical specialisation.

ii) The classes [\overline{U}_{w}] and [\overline{V}_{ν}] form a ℤ\{p\}-basis for the respective ptautological rings.

iii) The coefficients of [\overline{U}_{w}] and [\overline{V}_{ν}] when expressed in terms of the polynomials in the λ_{i} are in ℤ[p].

iv) For w ∈ W_{g} we have that ℓ(w) = ℓ(τ_{p}(w)) (see Section 9 for the definition of τ_{p}) precisely when the specialisation to characteristic p of π_{∗}[\overline{U}_{w}] is non-zero. In particular, there is a unique map τ: W_{g} → (W_{g}/S_{g}) ⨆{0} such that τ(w) = 0 precisely when π_{∗}[\overline{U}_{w}] = 0 which implies that ℓ(w) ≠ ℓ(τ_{p}(w)) and is implied by ℓ(w) ≠ ℓ(τ_{p}(w)) for all sufficiently large p. Furthermore if τ(w) ≠ 0 then ℓ(w) = ℓ(τ(w)) and π_{∗}[\overline{U}_{w}] is a non-zero multiple of [\overline{V}_{τ(w)}].

Proof. The first part is clear as putting p = 0 in our formulas gives the Fulton formulas for x_{i} = −ℓ_{i} and y_{i} = 0 which are the Fulton formulas for the Schubert varieties in Sp_{2g} /B_{g}. One then obtains the formulas for the Schubert varieties of Sp_{2g} /P_{H} by pushing down by Gysin formulas. The remaining compatibility needed is that the classical specialisation of γ(w) is the degree of the map from the Schubert variety of Sp_{2g} /B_{g} for a final element to the corresponding Schubert variety of Sp_{2g} /P_{H}. However, the classical specialisation of γ(w) is 1 and the map between Schubert varieties is an isomorphism between Bruhat cells.

As for the second part we need to prove that the determinant of the matrix expressing the classes of the strata in terms of a basis of the tautological ring (say given by monomials in the ℓ_{i} resp. the λ_{i}) is invertible. Given that an element of ℤ{p} is invertible precisely when its classical specialisation is, we are reduced to proving the corresponding statement in the classical case. However, there it follows from the cell decomposition given by the Bruhat cells which give that the classes of the Schubert cells form a basis for the integral Chow groups.

To prove iii) it is enough to verify the conditions of Proposition 13.2. Hence consider w ∈ W_{g} (resp. a final element ν) and consider an element m in the tautological ring of degree complementary to that of [\overline{U}_{w}] (resp. [\overline{V}_{ν}]). By the projection formula the degree of m[\overline{U}_{w}] (resp. m[\overline{V}_{ν}]) is the degree of the restriction of m to \overline{U}_{w} (resp. \overline{V}_{ν}) and it is enough to show that the denominators of these degrees are only divisible by a finite number of primes (independently of the characteristic p). However, if the characteristic is different from 3 we may pull back to the moduli space with a level 3 structure and there the degree is an integer as the corresponding strata are schemes. Hence the denominator divides the degree of the level 3 structure covering Ã_{g,3} → Ã_{g} which is independent of p.

Finally for iv), it is clear that in the Chow ring of Ã_{g} the class π_{∗}[\overline{U}_{w}] is non-zero precisely when π: [\overline{U}_{w}] → \overline{V}_{τ_{p}(w)} is generically finite as all fibres have the same dimension by Proposition 9.6. This latter fact also gives that it is generically finite precisely when \overline{U}_{w} and \overline{V}_{τ_{p}(w)} have the same dimension which is equivalent to ℓ(w) = ℓ(τ_{p}(w)). When this is the case we get that π_{∗}[\overline{U}_{w}] is a non-zero multiple of [\overline{V}_{τ_{p}(w)}] again as the degree over each component of \overline{V}_{τ_{p}(w)} is the same by Proposition 9.6. Consider now instead π_{∗}[\overline{U}_{w}] in the ptautological ring and expand π_{∗}[\overline{U}_{w}] as a linear combination of the [\overline{V}_{ν}]. Then what we have just shown is that for every specialisation to characteristic p at most one of the coefficients is non-zero. This implies that in the p-tautological ring at most one of the coefficients is non-zero. If it is zero then π_{∗}[\overline{U}_{w}] is always zero in all characteristic p specialisations and we get ℓ(w) ≠ ℓ(τ_{p}(w)) for all p. If it is non-zero, then the coefficient is non-zero for all sufficiently large p. This proves iv). ⊓ ⊔

To complete the proof of the theorem we need to prove the following proposition.

Proposition 13.2. Let a be an element of the p-tautological ring for F̃_{g} or Ã_{g}. Assume that there exists an n ≠ 0 such that for all elements b of the tautological ring of complementary degree and all sufficiently large primes p we have that deg(\overline{a}\overline{b}) ∈ ℤ[n^{−1}], where \overline{a} and \overline{b} are the specialisations to characteristic p of a resp. b. Then the coefficients of a are in ℤ[p].

Proof. If r(x) is one of the coefficients of a, then the assumptions say that r(p) ∈ ℤ[n^{−1}] for all sufficiently large primes p. Write r as g(x)/f(x) where f and g are integer polynomials with no common factor. Thus there are integer polynomials s(x) and t(x) such that s(x)f(x) + t(x)g(x) = m, where m is a non-zero integer. If g is non-constant there are arbitrarily large primes ℓ such that there is an integer k with ℓ|f(k) (by for instance the fact that there is a prime which splits completely in a splitting field of f). By Dirichlet’s theorem on primes in arithmetic progressions there are arbitrarily large primes p such that f(p) ≠ 0 and f(p) ≡ f(k) ≡ 0 mod ℓ. By making ℓ so large so that ℓ |̸ m we get that ℓ |̸ g(q) (as s(q)f(q) + t(q)g(q) = m) and hence ℓ appears in the denominator of r(q). By making ℓ so large so that ℓ |̸ n we conclude. ⊓ ⊔

Example 13.3. If w is a shuffle of a final element ν we have τ(w) = ν.

We can combine Theorem 13.1 with our results on the punctual flag spaces to give an algebraic criterion for inclusion between E-O strata.

Corollary 13.4. Let ν′ and ν be final elements. Then for sufficiently large p we have that ν′ ⊆ ν if there are w, w′ ∈ W_{g} for which w′ ≤ w, τ(w) = ν, and τ(w′) = ν′.

Proof. Assume there are w, w′ ∈ W_{g} for which w′ ≤ w, τ(w) = ν, and τ(w′) = ν′. By Proposition 9.6 we have that for π: F̃_{g} → Ã_{g} the following image relations π(\overline{U}_{w}) = \overline{V}_{ν_{1}} and π(\overline{U}_{w′}) = \overline{V}_{ν′_{1}} for some ν_{1} and ν′_{1} and by the theorem ν_{1} = τ(w) and ν′_{1} = τ(w′) for sufficiently large p. As w′ ≤ w we have that \overline{U}_{w′} ⊆ \overline{U}_{w} which implies that π\overline{U}_{w′} ⊆ π(\overline{U}_{w}). ⊓ ⊔

14 Comparison with S(g, p)

We shall now make a comparison with de Jong’s moduli stack S(g, p) of Γ_{0}(p)- structures (cf., [Jo93]). Recall that for a family A → S of principally polarised g-dimensional abelian varieties a Γ_{0}(p)-structure consists of the choice of a flag 0 ⊂ H_{1} ⊂ ⋯ ⊂ H_{g} ⊂ A[p] of flat subgroup schemes with H_{i} of order p^{i} and H_{g} totally isotropic with respect to the Weil pairing. We shall work exclusively in characteristic p and denote by \overline{S(g, p)} the mod p fibre of S(g, p). We now let S(g, p)^{0} be the closed subscheme of \overline{S(g, p)} defined by the condition that the group scheme H_{g} is of height 1. This means that the (relative) Frobenius map F_{A/S(g,p)}, where π: A → S(g, p) is the universal abelian variety, is zero on it. For degree reasons we then get that H_{g} equals the kernel of F_{A/S(g,p)}. Using the principal polarisation we may identify the Lie algebra of π with R^{1}π_{∗}O_{A} and hence we get a flag 0 ⊂ Lie(H_{1}) ⊂ Lie(H_{2}) ⊂ ⋯ ⊂ Lie(H_{g}) = R^{1}π_{∗}O_{A}. By functoriality this is stable under V. Completing this flag by taking its annihilator in E gives a flag in \overline{U}_{w_{∅}} thus giving a map S(g, p)^{0} → \overline{U}_{w_{∅}}.

Theorem 14.1. The canonical map S(g, p)^{0} → \overline{U}_{w_{∅}} is an isomorphism. In particular, S(g, p)^{0} is the closure of its intersection with the locus of ordinary abelian varieties and is normal and Cohen-Macaulay.

Proof. Starting with the tautological flag \{𝔼_{i}\} on \overline{U}_{w_{∅}} we consider the induced flag {𝔼_{g+i}/𝔼_{g}} in R^{1}π_{∗}O_{A}. This is a V -stable flag of the Lie algebra of a height 1 group scheme so by, for instance [Mu70, Thm. §14], any V -stable subbundle comes from a subgroup scheme of the kernel of F_{A/S(g,p)} and thus the flag {𝔼_{g+i}/𝔼_{g}} gives rise to a complete flag of subgroup schemes with H_{g} equal to the kernel of the Frobenius map and hence a map from \overline{U}_{w_{∅}} to S(g, p)^{0} which clearly is the inverse of the canonical map.

The rest of the theorem now follows from Corollary 8.4. ⊓ ⊔

626  Torsten Ekedahl and Gerard van der Geer

15 Appendix g = 3, 4

15.1 Admissible Strata for g = 3 In the following matrix one finds the loci lying in \overline{U}_{w_{∅}}. In the sixth column we give for each w an example of a final ν such that w → ν.

ℓ Y  w  ν  word w → ν
0 {1, 2, 3} [123] {0, 0, 0}  Id  [123] Superspecial
1  [132] {0, 0, 0}  s_{2}  [123] Fibre over s.s.
1  [213] {0, 0, 0}  s_{1}  [123] Fibre over s.s.
1 {2, 3} [124] {0, 0, 1}  s_{3}  [124] Moret-Bailly
2  [142] {0, 0, 1}  s_{3}s_{2}  [135]
2  [214] {0, 0, 1}  s_{3}s_{1}  [135]
2  [231] {0, 0, 0}  s_{1}s_{2}  [123] Fibre over s.s.
2  [312] {0, 0, 0}  s_{2}s_{1}  [123] Fibre over s.s.
2 {1, 3} [135] {0, 1, 1}  s_{2}s_{3}  [135] f = 0, a = 2
3  [153] {0, 1, 1} s_{2}s_{3}s_{2}  [236] Shuffle of {1, 2}
3  [241] {0, 0, 1} s_{3}s_{1}s_{2}  [124]
3  [315] {0, 1, 1} s_{2}s_{3}s_{1}  [124]
3  [321] {0, 0, 0} s_{1}s_{2}s_{1}  [123] Fibre over s.s.
3  [412] {0, 0, 1} s_{3}s_{2}s_{1}  [236] Shuffle of {1, 2}
3 {3} [145] {0, 1, 2} s_{3}s_{2}s_{3}  [145]  f = 0
3 {1, 2} [236] {1, 1, 1} s_{1}s_{2}s_{3}  [236]  a = 2
4  [154] {0, 1, 2} s_{3}s_{2}s_{3}s_{2} [246] Shuffle of {2}
4  [326] {1, 1, 1} s_{1}s_{2}s_{3}s_{1} [236]
4  [351] {0, 1, 1} s_{2}s_{3}s_{1}s_{2} [236]
4  [415] {0, 1, 2} s_{3}s_{2}s_{3}s_{1} [246] Shuffle of {2}
4  [421] {0, 0, 1} s_{3}s_{1}s_{2}s_{1} [236]
4 {2} [246] {1, 1, 2} s_{3}s_{1}s_{2}s_{3} [246]  f = 1
5  [426] {1, 1, 2} s_{3}s_{1}s_{2}s_{3}s_{1} [356] Shuffle of {1}
5  [451] {0, 1, 2} s_{3}s_{2}s_{3}s_{1}s_{2} [356] Shuffle of {1}
5 {1} [356] {1, 2, 2} s_{2}s_{3}s_{1}s_{2}s_{3} [356]  f = 2
6 {} [456] {1, 2, 3} s_{3}s_{2}s_{3}s_{1}s_{2}s_{3} [456]  f = 3

15.2 E-O Cycle Classes for g = 3 We give the cycle classes of the (reduced) E-O strata for g = 3.

\begin{matrix}Y & class \\ ∅ & 1 \\ \{1\} & (p − 1)λ_{1} \\ \{2\} & (p − 1)(p^{2} − 1)λ_{2} \\ \{1, 2\} & (p − 1)(p^{2} + 1)λ_{1}λ_{2} − 2(p^{3} − 1)λ_{3} \\ \{3\} & (p − 1)(p^{2} − 1)(p^{3} − 1)λ_{3} \\ \{1, 3\} & (p − 1)^{2}(p^{3} + 1)λ_{1}λ_{3} \\ \{2, 3\} & (p − 1)^{2}(p^{6} − 1)λ_{2}λ_{3} \\ \{1, 2, 3\} & (p − 1)(p^{2} + 1)(p^{3} − 1)λ_{1}λ_{2}λ_{3} \end{matrix}

15.3 E-O Cycle Classes for g = 4

We give the cycle classes of the (reduced) E-O strata for g = 4.

\begin{matrix}Y & class \\ ∅ & 1 \\ \{1\} & (p − 1)λ_{1} \\ \{2\} & (p − 1)(p^{2} − 1)λ_{2} \\ \{1, 2\} & (p − 1)(p^{2} + 1)λ_{1}λ_{2} − 2(p^{3} − 1)λ_{3} \\ \{3\} & (p − 1)(p^{2} − 1)(p^{3} − 1)λ_{3} \\ \{1, 3\} & (p − 1)^{2}(p + 1)((p^{2} − p + 1)λ_{1}λ_{3} − 2(p^{2} + 1)λ_{4}) \\ \{2, 3\} & (p − 1)^{2}((p^{6} − 1)λ_{2}λ_{3} − (2p^{6} + p^{5} − p − 2)λ_{1}λ_{4} \\ \{1, 2, 3\} & (p − 1)(p^{2} + 1)((p^{3} + 1)((p^{3} − 1)λ_{1}λ_{2}λ_{3} − 2(3p^{3} + p^{2} − p + 3)λ_{2}λ_{4}) \\ \{4\} & (p − 1)(p^{2} − 1)(p^{3} − 1)(p^{4} − 1)λ_{4} \\ \{1, 4\} & (p − 1)^{3}(p + 1)(p^{4} + 1)λ_{1}λ_{4} \\ \{2, 4\} & (p − 1)^{3}(p^{8} − 1)λ_{2}λ_{4} \\ \{1, 2, 4\} & (p − 1)^{2}(p^{4} − 1)((p^{2} + 1)λ_{1}λ_{2} − 2(p^{2} + p + 1)λ_{3})λ_{4} \\ \{3, 4\} & (p − 1)^{2}(p^{2} + 1)(p^{3} − 1)(p^{2} − p + 1)((p + 1)^{2}λ_{3} − pλ_{1}λ_{2})λ_{4} \\ \{1, 3, 4\} & (p − 1)^{2}(p^{4} − 1)(p^{6} − 1)λ_{1}λ_{3}λ_{4} \\ \{2, 3, 4\} & (p − 1)(p^{6} − 1)(p^{8} − 1)λ_{2}λ_{3}λ_{4} \\ \{1, 2, 3, 4\} & (p − 1)(p^{2} + 1)(p^{3} − 1)(p^{4} + 1)λ_{1}λ_{2}λ_{3}λ_{4} \end{matrix}

References

[BGG73] I. N. Bernstein, I. M. Gelfand, S. I. Gelfand, Schubert cells, and the cohomology of the spaces G/P, Uspehi Mat. Nauk 28 (1973), no. 3(171), 3–26.

[BL00] S. Billey, V. Lakshmibai, Singular loci of Schubert varieties, Progr. in Math., vol. 182, Birkhäuser Boston Inc., Boston, MA, 2000.

[Ch94] C. Chevalley, Sur les décompositions cellulaires des espaces G/B, Algebraic groups and their generalizations: classical methods (University Park, PA, 1991), Proc. Sympos. Pure Math., vol. 56, Amer. Math. Soc., Providence, RI, 1994, With a foreword by Armand Borel, pp. 1–23.

[Ek87] T. Ekedahl, On supersingular curves and abelian varieties, Math. Scand. 60 (1987), 151–178.

[FC90] G. Faltings, C.-L. Chai, Degeneration of abelian varieties., Ergebnisse der Math., no. 22, Springer Verlag, 1990.

[FP98] W. Fulton, P. Pragacz, Schubert varieties and degeneracy loci, SLN, vol. 1689, Springer-Verlag, Berlin, 1998, Appendix J by the authors in collaboration with I. Ciocan-Fontanine.

[Fu96] W. Fulton, Determinantal formulas for orthogonal and symplectic degeneracy loci, J. Differential Geom. 43 (1996), no. 2, 276–290.

[Ful] W. Fulton, Intersection theory, vol. 3, Ergebnisse der Mathematik und ihrer Grenzgebiete, no. 2, Springer Verlag, 1984.

[Ge99] G. van der Geer, Cycles on the moduli space of abelian varieties, Moduli of curves and abelian varieties (C. Faber and E. Looijenga, eds.), Aspects Math., E33, Vieweg, Braunschweig, 1999, pp. 65–89.

[Gr65] A. Grothendieck, Éléments de géométrie algébrique. IV. Étude locale des schémas et des morphismes de schémas. II, Inst. Hautes Études Sci. Publ. Math. (1965), no. 24, 231.

[GrLie4-6] N. Bourbaki, Groupes et algèbres de Lie. IV–VI, Hermann, Paris, 1968.

[HW61] H. Hasse, E. Witt, Zyklische unverzweigte Erweiterungskörper von primzahlgrade p über einem algebraischen Funktionenkörper der Charakteristik p, Monatsh. Math. Phys. 43 (1963), 477–492.

[Ha07] S. Harashita, Ekedahl-Oort strata and the first Newton slope strata, J. of Algebraic Geom. 16 (2007), no. 1, 171–199.

[Jo93] A. J. de Jong, The moduli spaces of principally polarized abelian varieties with Γ_{0}(p)-level structure, J. of Algebraic Geom. 2 (1993), no. 4, 667–688.

[KS03] S. Keel, L. Sadun, Oort’s conjecture for A_{g} ⊗ ℂ, J. Amer. Math. Soc. 16 (2003), no. 4, 887–900 (electronic).

[KT02] A. Kresch, H. Tamvakis, Double Schubert polynomials and degeneracy loci for the classical groups, Ann. Inst. Fourier (Grenoble) 52 (2002), no. 6, 1681–1727.

[M2] D. R. Grayson, M. E. Stillman, Macaulay 2, a software system for research in algebraic geometry, Available at http://www.math.uiuc.edu/

Macaulay2.

[MB81] L. Moret-Bailly, Familles de courbes et variétés abéliennes sur ℙ^{1}., Séminaire sur les pinceaux de courbes de genre au moins deux. (L. Szpiro, ed.), Astérisque, vol. 86, Soc. math. de France, 1981, pp. 109–140.

[MB85] L. Moret-Bailly, Pinceaux de variétés abéliennes, Astérisque (1985), no. 129, 266.

[MW04] B. Moonen, T. Wedhorn, Discrete invariants of varieties in positive characteristic, IMRN (2004), no. 72, 3855–3903,

Eprint: math.AG/0306339.

[Mo01] B. Moonen, Group schemes with additional structures and Weyl group cosets, Moduli of abelian varieties (Texel Island, 1999), Progr. Math., vol. 195, Birkhäuser, Basel, 2001, pp. 255–298.

[Mu70] D. Mumford, Abelian varieties, Tata Institute of Fundamental Research Studies in Mathematics, vol. 5, Oxford University Press., London, 1970.

[Oo01] F. Oort, A stratification of a moduli space of abelian varieties, Moduli of abelian varieties (Texel Island, 1999), Progr. Math., vol. 195, Birkhäuser, Basel, 2001, pp. 345–416.

[Oo95] F. Oort, Complete subvarieties of moduli spaces, Abelian varieties (Egloffstein, 1993), de Gruyter, Berlin, 1995, pp. 225–235.

[PR97] P. Pragacz, J. Ratajski, Formulas for Lagrangian and orthogonal degeneracy loci; Q̃-polynomial approach, Compositio Math. 107 (1997), no. 1, 11–87.

[PR99] H. Pittie, A. Ram, A Pieri-Chevalley formula in the K-theory of a G/B-bundle, Electron. Res. Announc. Amer. Math. Soc. 5 (1999), 102– 107 (electronic).

[Pr91] P. Pragacz, Algebro-geometric applications of Schur S- and Qpolynomials, Topics in invariant theory (Paris, 1989/1990), Lecture Notes in Math., vol. 1478, Springer, Berlin, 1991, pp. 130–191.

[RR85] S. Ramanan, A. Ramanathan, Projective normality of flag varieties and Schubert varieties, Invent. Math. 79 (1985), no. 2, 217–224.

[S] N. Sloane, The on-line encyclopedia of integer sequences, http://www.

research.att.com/~njas/sequences.

[Sj07] J. Sjöstrand, Bruhat intervals as rooks on skew ferrers boards, To appear in J. Combin. Theory Ser. A, Eprint: math.CO/0601615.