How much surface area is inside your intercooler?
We all go to great lengths to keep that thing cool, and while obviously, it sheds energy much more effectively due to surface area and alloying, the air charge is howling through there just as fast, and thermal conductivity doesn't much care which direction it is going.
Flow velocity at the wall is much lower, so it has more contact time as well. Throw in the turbulent flow so it mixes into the main flow better.
The biggest gain is keeping the IM from heat soaking to begin with. All that heat goes into the radiator... and is then blown back over the engine.
If we keep using big engineering words, this may have to move to technical discussions.