I recently had to switch profiles for this website. In the process of doing that, I immediately afterwards said “wow, I feel like other people have had this issue”. Sure enough they have… on this blog early last year by our own @aitala in the post How to Remove Drupal Install Profile.