Multi-WAN / SD-WAN traffic on fail-over interface
I have a Firebox T35 (OS v 12.5.2.B6xxx). I have an external Fibre connection and a fail-over 4G connection on eth3. I have set metrics up on the Fibre, but not the 4G as it is a pre-paid service, and should only be used when the Fibre goes down. I did have the setting in the fall-back for the Multi-WAN to be gradual (they have VOIP systems and did not want to kill calls when Fibre came back up).
Event happened, lost Fibre - fail-over to 4G worked as intended. Fibre came back up, could see majority of traffic switching back from 4G, all good. Knowing that the VOIP phones will stay connected, later after the office closed I changed the setting to Immediate. Saw more connections drop from 4G, so job done. Next day, changed Multi-Wan back to Gradual (I confess I did not look at the traffic). A month later I am doing some other configuration work and noticed the 4G had more usage than I expected. I go into HostWatch and see a few systems using that interface. I changed the setting back to immediate - no change, they were still using the interface. I left it overnight, no change - I used HostWatch to manually terminate those connections the next day. In the past few hours, no activity on that port (aside from me checking I could manage the 4G box).
I cannot say when they started using the 4G (if they ever stopped), but I guess my concern is that the immediate setting did not appear to function as intended, plus that those systems were using it all (they were not the VOIP phones) - can anyone shed light on this? Do I need a policy for this - from what I read, from v 12.3(or 4) a policy is not needed for Multi-WAN fail-over/fall-back - it is a global setting. I created an SD-WAN Action (Immediate), but it is not tied into any policies. Has anyone else experienced this?