Leveling up has thankfully been signfiicantly changed from the convoluted system in Oblivion to a more intuitive, simpler system... albeit at the unfortunate loss of diversity, but that's an argument for another time. Simply put, in Skyrim when you level up skills (which is done by using said skill) you'll gain experience towards your overall level. These can easily be separated by referring to the former as skill levels and the latter as character levels. Swing a sword and you'll level up One-Handed. Cast a spell an you'll level up that spell school. Brew potions and you'll level up Alchemy. As these level up, your character levels up. The higher your skill level, the more experience you'll get towards your character level, and the higher your character level, the more experience you'll need.
Every time your character level increases you'll be able to distribute one attribute point (you'll be forced to do this next time you open your skills menu) and one perk point (you can save these for later), each of which will be described in greater detail in the next few pages.