We investigated site-specific magnetic behaviors of multiferroic Ba(0.5)Sr(1.5)Zn(2)(Fe(1-x)Al(x))(12)O(22) using Fe L(2,3)-edge x-ray magnetic circular dichroism. The Al dopants mostly replace the Fe(3+) ions at octahedral (O(h)) sites, which contribute unquenched angular momenta through off-centering displacements. This replacement greatly reduces the magnetic anisotropy energy to change the magnetic order from a helical to a heliconical type with enhanced magnetoelectric susceptibility (α(ME)). The tetrahedral (T(d)) Fe sites exhibit magnetic hysteresis distinguishable from that of the O(h) sites, especially at low magnetic fields. These results provide essential clues for the heliconical order with a giant α(ME) and multibit memory effects in the Al-doped Y-type hexaferrite.