A unit only needs to make one Panic test in each phase of the game, even if there are multiple reasons to take Panic tests. Most Panic tests are made immediately, and in larger games you may find it helpful to mark units that have already made a Panic test.
In addition, a unit is not required to make a Panic test if:
It is making a Charge move.
It is engaged in combat.
It is already fleeing and has yet to rally.