The truth of the matter is that it could be a weak alternator, weak battery, or belt slip. IMO it is more likely to be a bad alternator. Which is why the 1st suggestion was to test the alternator.
As RPM's increase, more power is needed to run the engine's electronics. If the alternator is boarder line, the extra power requirements could push it over the edge.
Reminder, the battery light is driven off of battery voltage only. Further, the PCM reduces charging current at WOT (to save load on the motor). Consider if the battery is weak or the terminals are dirty, when the alternator output is decreased by design, the extra power has to be made up by the battery. This could overwhelm the battery's ability to keep up.
That is why it is also good idea to check/clean the battery terminals at the same time. Ensure that the top of the battery is clean as well. Todays car will not run without a solid battery.
Belt slip should be evident on the belt and around the pulleys. But, IMO, what makes the most sense is to TEST the alternator first. Almost any auto parts store will test your alternator for free. Many will do a basic function test with the alternator still on the car.
While the alternator is off, INSPECT the belt looking for evidence of slipping/cracking/wear. If found, replace it. Also, inspect the tensioner. Replace it if it is weak.
To recap, the absolute 1st thing to do should be to clean the battery terminals. Clean the top of the battery. Next, take your car to the auto parts store. Ask them to test your battery and the alternator. Go from there.