Arrow Keys Up/Down: Accelerate/Stop the horse. Notice the tiredness bar at the top right corner. If a horse gallops it gets tired quickly and will trot instead.
Space Key: Jump. Remember you can only jump when the horse gallops.
Arrow Keys Left/Right: Turn the horse left/right