Understanding output overrides
It's always a bad practice to "hack" Joomla's core files. This means making any modifications to the joomla core components or it's libraries is not a good idea. When you later need to update Joomla, the update will wipe out the changes you made. Plus depending on the update your modification to the core may no longer be needed. Anyone who has ever used early versions of Jooomla knows how hard it was to control the output of Joomla. Sometimes its necessary to modify the way a page looks (i.e. the registration page) to make the page work with whatever type of site you are trying to build with Joomla. Output overrides allow you to completely override the way a page looks and the modifications are stored in whatever template you are using so that they won't be overwritten when you upgrade Joomla. It is also worth mentioning that there is a huge difference between the output that is generated on a page and the template. The template just controls the framework of each page and it's positions. The modules and content that is actually displayed in the page is not created by the template but by modules and components.
Joomla Components
Components are the most complex type of extension you can install into your Joomla site. The complexity here is because components can be very large pieces of software with much functionality and because of this components have many views which in turn can have multiple layouts. If this is confusing then please see our article on MVC. Views are the part of Joomla that are in control of displaying output to the user. This is the part you see when you use the site. The point of views is to display data to the users and to give users an place to input data if need be. This data could be a table of something or just a page with a login.
Joomla Modules
Modules are much more simple than components. Modules typically just display a layout to a position on a page and don't have views or multiple layouts. But it is possible to override module layouts as well so that you can get that customized look that you're after.
Overriding the output
Overrides are held in the html folder of your template. To create an override you can just add the components folder with the layouts inside to the html folder of your template. The common location is the html folder but some templates change the location of the template overrides folder so be careful that you know the location on your template. To illustrate how this would work lets pretend we want to override the way the login page looks. We would go to the html folder in the template and add a new directory called com_users and inside this directory we would add the default_login.php file that is present in the com_users component. We can now make our changes to this file and they will show and not be in danger of being overwritten by updates to Joomla's core.