The phrase is a fairly recent one, and as you say encompasses nuclear, biological and chemical weapons (which is a bit of a mouthful, so "WMD" trips off the tongue more easily for politicians and journalists). ICBMs are nuclear strategic weapons, so they fall into the category.
The main criteria are: they can be used to attack large numbers of people/buildings simultaneously, and do not discriminate between combatants and non-combatants.
The latter point can be tempered by their usage - i.e. all have valid military applications - there are circumstances where their use against troop concentrations, a carrier fleet, or a bunker might be appropriate - but on the whole "weapons of mass destruction" tends to refer to their deliberate usage against civilians, either for plain terror, or to destroy morale or the economic base of a nation at war.
Under the Geneva convention, it is
very naughty to do this, not that it has ever stopped anyone.
As for the comparison with regular bombing, the difference is only really one of scale - it would take hundreds of B-52 carpet bombings to equal a single 10kt nuclear blast on a city. (370 loads in fact). Bio-weapons could potentially affect millions, especially with modern air transport to spread disease. Chemical weapons tend to be more local, although as Halabja proved, still potent enough to strip a town of life in an afternoon.
As for who has them - the US, UK, Russia, China, France and South Africa have at some point had all of them, Israel, Pakistan and India now have nukes, and pretty much any other country with a university-level laboratory could potentially make bio and chem stuff.
You might find the relevant pages at
http://www.fas.org/ to be of interest, they monitor proliferation and development of such weapons.