This appears to be a limitation of the toolchain. The output of processing the partitions.csv file doesn't make it back to the compiler to generate the correct message. The different partition schemes available in the menu come from boards.txt of the ESP32 package. They can be updated, but would be overwritten when the toolchain is updated. For the compiler input, the setting in the menu is relevant, but the actual size of the partitions comes from the partitions.csv file (if it exists).
This means that one has to manually make sure that the size of the factory partition in the csv file is larger or equal to the actual code size. Ideally, it is also equal to the selected partition size from the menu (to get the correct message and eventually error if the binary size exceeds the allocated space).
Some testing shows that if the actual size of the factory partition is smaller than the code size (and smaller than the compiler size limit) the application will successfully build and upload, but fail to run. The chip will enter a startup-reboot-forever loop and not execute any user code. However, whether that is always the case is not guaranteed.