ESP8226 ROM boot loader determines what to do at boot time by the state of GPIO pins. As you probably know, pulling GPIO0 to ground makes it enter firmware download mode, which is used for flashing firmware. There are more combinations - in total, 6 pins are checked and various combinations will cause the device to boot in different modes (for example, GPIO15 has to be 0 all the time or device will enter one of the SIP boot modes).
The bits that ROM uses to determine the state of the GPIO pins are so-called strapping bits. The state of GPIO pins is sampled at hardware reset time and stored in these strapping bits, which are read-only and cannot be changed until the next hardware reset.
You may already see what the problem is: even though GPIO0 may have been released, the strapping bit corresponding to it will remain active until the next hardware reset.
system_restart() will do all the right things, reboot the device, then ROM code will take over and happily enter firmware download mode, like the strapping bits tell it to. Thus, any software-initiated reboot after first flashing will land in the flash download mode, which to the user will appear as a failure to boot. But if the user resets or power-cycles the device, it will work again! Needless to say, this is very confusing.
Unfortunately, we’ve been unable to find a way to reliably reboot ESP device after flashing. If you know a way, please share!