I always consider something a sport if there is someone you're directly competing against at the same time, not turn-taking or doing solo.

Skateboarding is ridiculously hard to master and extremely fun to watch. It's a grueling competition that takes precision and endurance, I just personally don't count it as a sport because there isn't someone riding at the same time to skate directly against.

For example, I consider fencing a sport because your thrust could be dodged, parried, or riposted against by the opponent, you don't control the other competitor but their actions directly affect you. But I don't consider archery a sport because the competitors can not directly modify each other's games.

//for me it's not about the difficulty it's all about the competition aspect. Also, objective rules/scoring.