Visual Learning Beyond Direct Supervision