Technically, you can't get a free domain name. Usually, free domain name providers give you a subdomain, such as mywebsite.weebly.com. If you want your own domain, such as mywebsite.com, you will have to buy your own domain, which can cost from $5 a year upwards. Now, once you have decided to get a domain name, whether free or paid, you need a hosting service to host your website. Most free services offer both the subdomain a hosting together, and even some sites offer free hosting if you buy your domain name off of them. However, free hosting and paid hosting have very important differences you should know before proceeding.
First of all, if you have little/no knowledge of HTML, free hosting sites is the way to go. You can build a website for free without knowing how to code, using simple drag-and-drop interfaces. Paid hosting will give you much more complete control of your website, but will not give you a simple way to build your website, instead giving you nothing and having to build your website from scratch through code. However, paid hosting allows you to do things such as embed java applets that you can not do through sites such as weebly. When you upload a file through weebly, it is saved on their server. Using html to embed the applet requires you to know the location of where you uploaded the file, but it is difficult to find. I am still trying to get around this, drop a comment if you know. Also, free hosting only allows you to run client-side languages with HTML and Javascript. THese are still powerful tools you should use if building a website, as it allows interactability with your user. However, free hosting/domains don't' give you the ability ot use server-side languages such as PHP, while paid hosting/domains do. Server-side languages offer stability as well as more control, because the code for your website is run on the server, so if it works once, it will work on all machines. You can process user input on your site and do things automatically and efficiently with PHP. Last thing to remember though: if you are set on using a free hosting/domain, I recommend Weebly.com. It's what my site was built on, and offers quite a lot of control and gives your site a professional look with features such as drop-down menus easily.
In general, rules to follow: If you are new and do not know much about programming, building a website just for fun, or don't want to invest money on a website, try free website domain/hosting. If you want complete control over your website and have proficiency at building a website, or can hire someone to build you a professional website, choose paid domain/hosting.
Above all, have fun building your website!