Now, I could be entirely wrong here, Freaky, but I would be willing to bet that those pants have sequins on them, or are sparkly, thereby producing your white specks. You've just ruined a probably expensive pair of pants. :P
the first time I went over to eve's place - I wore black wool pants[winter] uniform too.
and within 2 minutes I was covered - So being that ladies like little kitties - its probalby kitty fur - and it would not matter how much you cleaned anything - IMO - wiz