2 things - is your power supply sufficiently powerful? aka over 300w? The card doesn't suck much, but your proc does. Grab a 350-430w if your short on juice.
Next, are you having heat problems? Go download MotherboardMonitor from
www.motherboardmonitor.com and check your temperatures once it's configured.
Now, if all these check out clean, you know it's a problem with the Hyper-Threading. I would first recommend disabling it, and running it without HT. If it fixes it and the performance is fine, leave it, but if you want it enabled all the time, ask someone who deals with P4's (not me).
Happy troubleshooting!