For some reason, when connected to a VPN, macOS can choose to not route your traffic through the VPN and instead route it through the regular connection.
That seems completely backward, but perhaps it’s in case the VPN connection is too slow, they can route the traffic through the regular connection? This can be baffling as macOS will still show you are connected to the VPN and it breaks assumptions people make about privacy and whether or not their true IP address / location will be exposed when using a VPN.
There’s a simple way to change that behavior though. Under System Preferences > Network select Set Service Order… under the gear icon:
Then move your VPN to the very top of the list, above any WiFi, Ethernet, Bluetooth, or USB interfaces, and save. This should tell macOS to prioritize sending traffic through the VPN when it’s connected.