you have to boot the machine with the hard drive with the operating system on it.
sounds like you need to jumper the drives properly. one needs to be master(drive with O/S on it), the other slave. what wire are you talking about? the IDE (usually grey colored) cable? in most cases that wont matter with the cable position. you need to look at the drive itself and set the proper drive for "master" most drives show you the way to put the jumpers
here's a rough idea of what i mean. its a cdrom, but youll get the idea.