How does the bike run in the higher revs once you get past the stumble?
Make sure your fuel lines are unobstructed and your fuel filter is clean. If a line is pinched even the slightest bit your bike will run lean. I cooked a snowmobile motor once because the line had a kink in it. On a long shot, take the petcock out of the tank and make sure its working properly and theres nothing in the tank that would be plugging it.
The heat would make the motor run richer and feel doggy but it would still run. This sounds more like a fuel starvation problem.
04 Husky SMR 450
85 Yamaha RZ 500