We can put a rectangle with no-fill and no-stroke under a tile art of a pattern brush to tell Illustrator about the bounds we want to use.
Also, we can clip the tile art with a copy of this rectangle.

When used together, this allows to create a tile which is clipped and uses the bounds we expect... Well, almost — the preview of the tile in the dialog is going to be wrong, but the preview of the brush itself and the way brush behaves is going to be correct.

But without the bottom copy Illustrator won’t respect the clipping group’s bounds — while there is no reason to not to. Ai should limit the tile to the clipping bounds, an not the bounds of the clipped content automatically.