Does high school matter?

I am a junior and now that I am a little older I am worried that the fact that I wasn't tought to well in high school and that I slacked off my first two years. I am also in foundations (special ed) I am not stupid I just slacked off to much in middle school and was placed there. Now I am a junior and I am working my but off studying and all that stuff. But I am worried about my future and SAT's and all of that. After PSAT I worried that I will do so bad at least on the english section. I knew no words for the vocab and I am worried that once I graduate high school will my education from high school determine how smart I really am and my ability to have a better vocabulary and all of that. Or is adulthood and college and all of that just who works the hardest, studies the best, and makes a good career?