...scientists became frightened by their own power to create new and potentially devastating hybrids for which there was no easy niche, no natural enemy, no obvious way to control.
What is "easy niche"?
A niche is the biological/environmental space that a plant or animal species inhabits, that it is adapted to better than any other species, through aeons of evolution. For example, the niche of the Ivory-billed Woodpecker is extensive mature hardwood forests of the southeastern United States.
The genetically-modified hybrids mentioned in your example have no naturally-evolved niche, no obvious position that they hold in relation to the complex web of relationships with other species in the biome.