Asteroids...be scared or not?

Honestly i think we should be concerned. If we know ahead of time we can get all the people evacuated and away safely! The human race can't end...thats just not a good thing. We could completly avoid the affects of a shoemaker levy 9 asteroid because if we know it's coming, we could possibly destroy it or at least get people to safety. But then again, a football sized asteroid is a threat to the earth every million years...im not sure we could destroy an even bigger asteroid if we can't do anything about that one. Then...the human race would be gone..and...yeah. that would suck.