Build or buy a computer. The most stressful and time-consuming thing to build is going to be the computer, but it is a great way to learn about how exactly a computer works. If you do opt to build, try to re-use old parts from existing or abandoned computers. Many of the components will still work. The minimum specs you want to be aiming for to be able to record music to a decent quality are 250 GB of memory space and 1 GB of RAM. Alternatively, if you have the cash, buy a computer matching these specs. Or you may well find you have a computer that meets the specs already, in which case stick with what you have.
Install only the software and hardware you need for music production on your computer. There are music programs you can download for free, but the quality and options on these are not great. Seriously consider investing in a decent music software program: Cakewalk sonar studio for PC, Apple logic Pro for a Mac. Both are good options. Don't put other media center features, Instant Messaging tools and file-sharing programs such as Vuze onto your computer. They take up unnecessary space and mean you have less memory available for your recordings. If you really can't live without these online tools, it may be worth investing in another cheap computer for recreational purposes.
Buy a high-end sound card. If you have bought a computer in the last few years, use the sound card that came with it. If not, you will need to invest in one. The sound card performs the same functions as a MIDI (Musical Instrument Digital Interface) and an audio interface (mixer). If you are looking to get professional quality sound it is probably worth building or investing in these two pieces of recording furniture. If you just want to mess about with music on your computer for fun, use a sound card.
Attach a decent microphone to your computer and you have created your own DIY recording furniture. You can use the microphone that comes with your computer if you wish, but the sound quality of your recordings will be affected. There are also electrical stores that sell build-your-own microphone kits. This is a good way to get a better quality microphone for less cash. Hook up all your recording furniture and have some music-making fun.