— Create a new Print document with default swatches.
— Delete ALL the presets: brushes, symbols, graphic styles...
— Now Open Swatches and run Select All Unused command from its flyout menu.
White and Black are not selected (expected in a way).
But notice that 'C=0 M=0 Y=0 K=100' is also not selected.
So when all the selected swatches get deleted, this black duplicate stays, and the color group stays, and you have to delete it manually — each time.

I tried to understand what forces Ai to keep it.
As I said, not presets remain, and I redefine the [Default] graphic style with something else than black-and-white, and change the [Normal Paragraph Style] even — then Select All Unused swatches selects White (!) but still both black are ignored.

Why? The second black has a different name. If any preset (which one though) still uses it — why it follows the value and not the name?

And if this command looks into presets — why it doesn’t look into symbols? (here is the report: http://illustrator.uservoice.com/forums/601447/suggestions/42368422)

So what forces Ai to keep both blacks? What uses it? Arrowheads maybe?

For now we can use this script to delete all the swatches but None and Registration ones:
https://github.com/sky-chaser-high/adobe-illustrator-scripts?tab=readme-ov-file#deleteAllSwatchesjs

Another older script:
https://github.com/johnwun/js4ai/blob/master/delete_fluff.js

Discussions at forums:
https://community.adobe.com/t5/illustrator-discussions/deleting-unused-swatches/td-p/2066066
https://community.adobe.com/t5/illustrator-discussions/action-to-remove-all-unused-swatches-nearly-works/td-p/6628647