MAKE. THE. HANDLER. BIGGER
make that tiny little ANCHOR and HANDLER bigger. I'm tired of failing to click that tiny rectangle every single second. PLEASE. I BEG YOU
The size of handles can be changed in Preferences > Selection & Anchor Display, using the Size slider.
As for the selection tolerance for these — the problem with it is known and reported in a separate bug report: https://illustrator.uservoice.com/forums/601447-illustrator-desktop-bugs/suggestions/20358952-selection-tolerance-for-selecting-handles
Please vote if you want to get it fixed.
Raynhard M. Kemal commented
Pardon my ignorance. It turns out you can make it a little bigger from Preference - Anchor Display, or change the snapping in Preference - Smart Guides
Subin George commented
You can already do this with illustrator. Explore the preferences options and you'll find