In other words, an engine made for a larger bore and stroke made smaller in displacement would fare worse than an engine made for smaller displacement and bored out. If the engine is optimized for smaller displacement and made larger than it wouldn't fare as well as an engine made for the larger displacement. But if an engine was optimized for larger displacement and made smaller it would perform like utter crap.
But this is never the case. And we end up increasing displacement with just bore and stroke, then trying to maximize air flow for that particular engine displacement with the port shapes and volumes available.
Engine power can be moved up and down the rpm without adjusting displacement at all too. That's done with cam lift vs duration and overlap plus timing and port flow length, shape, and cross sectional area. A LT1 Camaro can have a 1500-5800 rpm power band then go to a 2500-6500 power band with nothing more than a cam swap and tune. But, if the engine were designed to be a high revver the builder can optimize the rest of the intake and exhaust track to match the cam.
This is without even considering the area vs volume advantage that grow exponentially as displacement grows. The bigger the displacement the smaller the frictional surface area to volume relation gets.