After enough research and the right use of specific keywords in Google I stumbled accross the concept of setting up a local host. A local host basically makes your computer a server, and its free. Now there are other details some Googling can explain better but it is a cheap, straight forward approach to messing around and learning a new CMS to see how it fits your style and needs.
This is not a step-by-step guide on how to install and set up a local host on your computer, just a little summary on what I learned.
I am currently fiddling around with Drupal, the local host I use is called XAMPP, it is open-source and free. It is compatible with Windows, Mac, Unix and Solaris, there are different versions for each OS.
I am running XAMPP and Drupal on a Windows XP laptop and it works fine. XAMPP basically puts all the subcomponents needed to set up a local host into a tidy, little interface. It is quite a small file and a fast download. XAMPP contains Apache, PHP and MySQL and Perl. Just navigate to ApacheFriends.org. For the most part the install and set up is plug-n-play.
Although I am not completely sure about this, but other CMSs like Joomla can be installed and run using XAMPP. There are other free local host/Apache distributions such as WampServer.
I wish I had learnt about local servers earlier so that I could`ve jumped on the Drupal bandwagon and gotten work on my website faster.
I hope this helps anyone considering an open-source CMS like Drupal who has qualms about the associated hosting.
Note: Local hosts should only be used for developing and testing, it is not recommended to use as your sole website server (go public).