Terrible Performance on M1 MacBook Pro

I have two laptops, an intel MacBook Pro and a M1 MacBook Pro.

I have tidalcycles set up on each. Everything is functioning correctly, but when it's time to perform on the M1 MacBook Pro, there are terrible delays "late" visible in the SuperCollider IDE. And you can hear it, too; it's complete pauses where no sound happens and then BOOM lots of sounds trigger at once.

late 2.579986487
late 6.981062993
late 6.981062993
late 5.647717103
late 5.647717103
late 3.015854555
late 3.015854555

Has anyone else experienced this? The delays make using this machine unworkable. The Intel mac works fine.