I was also seeing this error in Drupal 8.3.2. There were two breadcrumb block...

I was also seeing this error in Drupal 8.3.2. There were two breadcrumb blocks created, but they were placed in different regions. One of the block was disabled. And while I was installing a module I was getting

Drupal\Core\Entity\EntityStorageException: 'block' entity with ID 'custom_admin_theme_breadcrumb...' already exists. in Drupal\Core\Entity\EntityStorageBase->doPreSave() (line 430 of /Users/micahjoyner/www/cmm.dev/core/lib/Drupal/Core/Entity/EntityStorageBase.php).

After I deleted the disabled block, the theme installed without any error.