Roton-Type Mode Softening In A Dissipative Quantum Many-Body System With Cavity-Mediated Long-Range Interactions