Does the underside of your Apple Magic Mouse accumulate a lot of filth? I’ve noticed that mine tends to pick up bits of dust and other stuff along it’s two rails. It’s not so bad, and is obvioulsy related to the cleanliness of my desk, but sometimes a piece of dirt can create a grinding noise! Maybe I should use a mousepad.