https://qz.com/1274131/googles-engineers-say-that-lack-of-rigor-is-ruining-ai-research/
Rahimi co-authored a paper (pdf) that says that knowledge, nominally the goal of scientific research, is currently in second place to “wins,” the practice of beating a benchmark as a way to getting recognized in the AI community. The authors go on to highlight how this may be skewing the true nature of progress in AI and contributing to wasted effort and suboptimal performance. For example, researchers showed how stripping “bells and whistles” from a translation algorithm made it work better, which highlighted how its creators didn’t know which parts of the AI were doing what. It’s not uncommon for the core of an AI to be “technically flawed.”