Select Artwork by Swatch
This is how it should work:
Select a Swatch in the Swatches palette, choose from the menu: ':Select all object that use this swatch' (as Fill / Stroke).
Comes in very handy to select items via the Swatches palette in complex cartoon style drawings. It is also a way to - kind of - create long awaited multi layer groups.

-
Yes, true. A clumsy workaround we now have is to create a dummy object, fill and stroke it with a swatch, use Select Same Fill, unselect the dummy object, hide or lock selection (unless you have something already hidden or locked), select the dummy again, use Select Same Stroke, hide-lock this too, unhide-unlock... Just nonsense.
Meanwhile you can use a free script by Sergey Osokin to achieve this with a click:
https://github.com/creold/illustrator-scripts/blob/master/md/Select.md#selectbyswatches
At the moment it forces you to choose between stroke or fills, but not both, but I bet it can be improved.