I remember it being a bug, when handles were not snapping to grid... but sometimes it’s actually useful! Currently we have to undo, disable the grid, and tweak the handle.
There should either be a hard option we can toggle (preferably with an ability to assign it to a hotkey), or a hotkey to toggle this behavior on the fly, the one we can tap while dragging and disable snapping for this operation only... or both!

Please take a look at this related request as well:
Introduce a tolerance for Snap to Grid / Pixel, so that snapping happens only when points are close enough to grid intersections
https://illustrator.uservoice.com/forums/333657/suggestions/49616738
This should also apply to handles, of course.