Thank you all for the feedback!
It is great to have so many knowledgeable people share their thoughts and experience. When I used the 40 grit, I was using a belt sander and then switched to an orbital. My main issue was that I switched straight from 40 to 120 grit. I should've added at least one other grit in between. Now I know!

Would anyone recommend adding a filler on top of the PU to fill in the scratches, or is resanding really the only way? Also, what are the consequences of sanding on wood that already has a couple coats of PU?

Thanks again!