I gave the Nvidia GPU another shot, and no matter what I did, i.e.:
- Nvidia's open drivers (not to be confused with nouveau)
- Nvidia's closed drivers as-is
- Nvidia's closed drivers with GSP disabled
the performance was always equally horrible in Gnome Wayland. So I can conclude that the real problem might be with Gnome
