Heterostructured materials (HSMs) can be created by introducing differently sized constituent components to enhance their performance by disentangling conflicting materials' properties, through the synergistic coupling effect of the constituents. This strategy has been successfully applied to structural materials to overcome the trade-off between strength and ductility and achieve superior mechanical properties; however, it remains less explored for functional materials. Beyond the random distribution of the constituents in HSMs, the ordering of constituents, e.g., grains, phases, and domain structures, can further enhance their coupling effect, thus leading to improved material properties or even transformative new functionalities. In this short perspective article, permanent magnetic materials are used as examples to review the recent progress in achieving enhanced properties and/or creating new physical mechanisms by building HSMs with ordered structures. This paper demonstrates that high-performance or revolutionary functional materials can be achieved by creating ordered HSMs.