Remove Single Handle from Within the Pen-Tool
If you want to REMOVE a handle by clicking, this can be achieved by using the Convert Anchor Point-Tool and Clicking on it.
Now, all other Pen-Tool Functions can be accessed by pushing modifier keys like Ctrl and Alt, except for this one. It requires you to change out of the tool.
I therefore suggest adding the functionality to the ALT-Mode of the Pen-Tool, which activates the Anchor Point-Tool, that if you click on a handle, this handle gets removed.
This does NOT conflict with the current function, which is to disconnect handles, since this require Click and Drag.