This is actually a pretty good comment.

From adults (like 25+), mostly the ones who have had international connections know English. This includes academic people and upper management of companies. English has been taught in schools since the early 00's, but the fundamental way of teaching things is very different from most Western places; adaptive learning isn't really a thing, so it often results in memorizing. This is why younger people can communicate in English somewhat, but due to almost 0 usage they aren't confident with their skills. This can partly be attributed to the concept of losing face. Some people come off shy as they don't want to speak in a foreign language with low confidence as they're afraid to make mistakes and possibly be misunderstood.

Nowadays most students are eager to learn English. A lot of them want to study abroad just to learn the language. The economy's been pretty bad and some are afraid of not finding future employment. To find work abroad, English is a requirement. Japan itself doesn't use English at all, so the average person thus far has been able to get through life without any exposure to it. MEXT (Japanese Ministry of Education etc.) has made some moves to increase English classes, but budgets are being cut and a lot of schools have had to lower the hours and/or quality of tutelage at their institutions. Small-budget high schools may have a local teach ready sentences with a broken accent, while most universities offer advanced courses with native English literature professors. I attended some English courses at both Tohoku University and a smaller local school and the differences were like night and day.

So, I agree with this_sort_of_thing. It's both people not knowing anything but basic English and the unwillingness to use the little they know due to low confidence. It's a tough situation.