Stainless steel components are widely used in aerospace, marine engineering, and railway transportation owing to their outstanding corrosion resistance and high mechanical strength. Welding is a key joining technique for stainless steel structural components. However, substantial residual stresses are inevitably introduced into components owing to the complex coupling effects of force and heat during welding. Because of the critical role of residual stress in the performance and service life of components, considerable attention has been given to its experimental measurement and theoretical modeling, as well as to the effects of microstructure (e.g., phase transformation and grain size) on residual stress. However, the microscopic mechanism through which grain size, especially the interaction between coarse and fine grains, influences residual stress distribution remains an open question. To clarify this mechanism, this study investigates grain-size distribution and interactions between coarse and fine grains in the weld zone of stainless steel samples prepared via electron beam welding using a combination of X-ray and neutron diffraction. A clear correlation is observed between grain-size variation in the weld zone and residual stress distribution. Samples with a narrow grain-size distribution (~32 μm) exhibit minimal transverse residual stress fluctuations, with a stress amplitude of ~160 MPa. However, samples with a wide grain-size distribution (~53 μm) show substantial transverse residual stress fluctuations, with a stress amplitude of ~316 MPa. Owing to the high dislocation density at grain boundaries and within grains, fine grains possess greater deformation resistance than coarse grains. The interplanar spacing of fine grains is larger than that of coarse grains, indicating that fine grains undergo less deformation and exert an extrusion effect on coarse grains. The size difference between coarse and fine grains is a key microscopic parameter that determines the distribution of welding residual stresses. By regulating the uniformity of grain-size distribution, local stress concentration can be reduced, enabling effective control of residual stress and enhancing the structural stability of welded components.