Compared with the low heat input welding steel structures, the high strength low alloy (HSLA) steel structures after high heat input welding keep high temperature with longer time, and the cooling speed is slower, then the austenite crystal grains of coarse-grained heat affected zones (CGHAZ) grow up sharply, and coarse upper bainite (UB) and ferrite side plate (FSP) are generated easily in original austenite crystal, thus toughness of CGHAZ deteriorates seriously. At present, the approach of improving toughness of CGHAZ is to produce massive interleaved acicular ferrite (AF) in the original austenite crystal. However, with the improvement of welding capability for thick plate, welding heat input will be greater, and the hold time of high temperature will be more prolonged. In this case, AF coarsens much seriously, thus the improvement of CGHAZ toughness is limited severely. In this work, a new method for improving the toughness of CGHAZ in high heat input welding steels by studying the distribution map of HAZ impact value was proposed. This new method changes the grain boundary ferrite (GBF) and AF of the CGHAZ to polygonal ferrite (PF) of the fine-grained heat affected zones (FGHAZ) at same peak temperature, which improves the toughness of CGHAZ significantly. Comparing the microstructures and toughness of CGHAZ in Ti-V-N and Al-Ti-V-N micro alloy welding steels, the transformation condition and nucleation mechanism of PF in the CGHAZ of Al-Ti-V-N steel were analyzed. It is found that micron oxide inclusions is a key factor to inducing the nucleation of massive PF in CGHAZ, and nanoscale carbonitride is a key factor to draging and pinning the grain boundaries of austenite and ferrite. Therefore, the effective combination of above two factors guarantees the generation of a large number of PF, which improves the impact toughness greatly at low temperature.