Reader Brian writes in to ask "Why can you stay on a bicycle when moving, but not when it's standing still?"
Think of something like a table or a couch. It has four legs that touch the floor and form a base of support (a polygon formed by an object's contact points with the ground) and as long as the table or couch's center of gravity (the mean location of the gravitational force acting on an object, or the effective point at which gravity acts) is above this base of support, it'll be statically stable, or stable when at rest.
A bicycle, on the other hand, is statically unstable because it only has two contact points with the ground (whatever portion of the front and back wheels happen to be touching the ground) and its base of support is a line segment. A good base of support needs at least three contact points with the ground, so bikes are hard to keep upright when they're still. Bikes are, however, dynamically stable, or stable when moving forward, because steering allows a rider to move the bike's points of support around under the center of gravity and keep it balanced, often with steering adjustments small enough that the rider may not even realize they're making them. It's sort of like standing on one foot. If you don't hop around a little, and you start falling sideways, you can't recover and you fall over. If you do hop, though, you can move your foot around to keep your center of gravity above it and keep your balance.
A bike has two features that help this dynamic stability immensely: its wheels. Spinning wheels have angular momentum, and when you're sitting on a bike, you and it and its wheels make up a system that obeys the principle of conservation of angular momentum. Unless torque, or twisting force, is applied from outside the system to change the wheels' angular momentum, that momentum and the direction of the momentum remain constant. In a nutshell, once the wheels line up a certain way, they want to stay lined up like that. It's easy for you to move them, but hard for an outside force to do the same, and so the bike is easy to keep balanced but doesn't topple easily. A non-moving bike has wheels that aren't spinning and zero angular momentum, which makes it very easy for external torque to change the wheels' direction, making the bike harder to balance.
Even when staying relatively motionless, though, a rider can balance a bike with some effort. By steering the front wheel to one side or the other and moving forward and backward slightly, a rider can keep the line between the bike's two contact points with the ground under the bike and rider's combined center of gravity. You can see this physics lesson in action whenever a cyclist is stopped at a red light.
(Image at left from Wikipedia user AndrewDressel.)
That was a short and sweet way to answer Brian's direct question, but there's a lot more to bicycle physics. If you're so inclined, Wikipedia's page on bicycle and motorcycle dynamics is a good starting point to learn about some of the other internal and external forces, motions and dynamics that are involved in a simple bike ride.