Why Evaluate on Downstream Tasks?