One Head is Better Than Two