When I do “All supply off”, the light on it still keeps blinking.
All Supply Off will open the relay of any connected Arc and disconnect power from the DUT. It will not power off the Arc. When the light is slowly flashing the Arc is powered but the (red & black bananas) is turned off. When the light is lit, the supply is on. If you have project settings open and the Arc selected, the light will flash faster so you can identify easily which of your Arc you have selected.
There currently is no way to turn off the light on the Arc short of disconnecting the USB from it.