comm -1 -3 <( cd dir1 && find -maxdepth 1 -type d | sort ) <( cd dir2 && find -maxdepth 1 -type d | sort ) | ( cd dir2 && xargs rm -rf )
with line breaks for readability:
comm -1 -3 <( cd dir1 && find -maxdepth 1 -type d | sort ) \
<( cd dir2 && find -maxdepth 1 -type d | sort ) \
| ( cd dir2 && xargs rm -rf )
explanation
find -maxdepth 1 -type d
list only directories without subdirectories.
cd dir1 && find -maxdepth 1 -type d | sort
first change to directory then list directories.
<( ... )
process substitution.
comm -1 -3 <( ... ) <( ... )
take first input and second input and print lines unique to second input. in effect this will print directories which are in dir2 but not in dir1.
... | ( cd dir2 && xargs rm -rf )
change working directory to dir2 then execute rm -rf with output of previous command as arguments. in effect this will delete the directories which are in dir2 but not in dir1.
test first by removing the pipe to xargs and inspecting the output.
dir2