Yes you do have to ammend your permit for an additional caliber. The only serial number that matters is the one on the frame. For the 320 it might be the number on the fire control mechanism.Just curious but do you have to note that on your permit that you have the other calibers? I realize that it's the same serial number but wasn't sure if one needed to do that or not.