Splittings and automorphisms of relatively hyperbolic groups

Abstract

We study automorphisms of a relatively hyperbolic group $G$. When $G$ is one-ended, we describe Out$(G)$ using a preferred JSJ tree over subgroups that are virtually cyclic or parabolic. In particular, when $G$ is toral relatively hyperbolic, Out$(G)$ is virtually built out of mapping class groups and subgroups of ${\mathrm{GL}}_n(\mathbb{Z})$ fixing certain basis elements. When more general parabolic groups are allowed, these subgroups of ${\mathrm{GL}}_n(\mathbb{Z})$ have to be replaced by McCool groups: automorphisms of parabolic groups acting trivially (i.e. by conjugation) on certain subgroups.

Given a malnormal quasiconvex subgroup $P$ of a hyperbolic group $G$, we view $G$ as hyperbolic relative to $P$ and we apply the previous analysis to describe the group Out$(P\nearrow G)$ of automorphisms of $P$ that extend to $G$: it is virtually a McCool group. If Out$(P\nearrow G)$ is infinite, then $P$ is a vertex group in a splitting of $G$. If $P$ is torsion-free, then Out$(P\nearrow G)$ is of type VF, in particular finitely presented.

We also determine when Out$(G)$ is infinite, for $G$ relatively hyperbolic. The interesting case is when $G$ is infinitely-ended and has torsion. When $G$ is hyperbolic, we show that Out$(G)$ is infinite if and only if $G$ splits over a maximal virtually cyclic subgroup with infinite center. In general we show that infiniteness of Out$(G)$ comes from the existence of a splitting with infinitely many twists, or having a vertex group that is maximal parabolic with infinitely many automorphisms acting trivially on incident edge groups.