An inverse problem in Pólya–Schur theory. I. Non-degenerate and degenerate operators

Abstract

Given a linear ordinary differential operator $T$ with polynomial coefficients, we study the class of closed subsets of the complex plane such that $T$ sends any polynomial (respectively, any polynomial of degree exceeding a given positive integer) with all roots in a given subset to a polynomial with all roots in the same subset or to $0$. Below we discuss some general properties of such invariant subsets, as well as the problem of existence of the minimal under inclusion invariant subset.