First, I am a software engineer by day so when diving into more complex networking, I have to defer to the experience of those who have actually been through these parts of the woods.

Currently we have a Verizon FIOS connection that runs part of Pushshift.io -- requests come in to various ports and the FIOS router along with NGINX know where to send the request (which API server) based on the port of the incoming request or the DNS of the server where the request is sent.

Recently, I purchased a T-Mobile 5G home internet router that I want to use as a secondary backup in the event that the Verizon FIOS connection goes down. The issue with T-Mobile is that they do not have a static IP address and I'd like to use one for incoming requests that are channeled through the backup connection (T-Mobile).

What I believe I am trying to build is a site-to-site VPN where requests come in to a static IP (let's say on a Ubuntu 20.04 / 22.04 server within Digital Ocean or Linode) and all traffic is routed to my local area network here on-premise (on-premise being the basement). Any requests for a specific port would automatically be transferred to a server on-premise and then that server could route appropriately.

1) Is this one of the capabilities of a site-to-site VPN?

2) Where can I find solid documentation for setting up something like this using OpenVPN, etc.

3) Is OpenVPN a decent program to use for this type of setup or should I be thinking of using something else?

For an example of what the end-result would look like -- we fire up a Linode server with a public IP of 1.2.3.4. A server on-premise will act as the gateway on this site and connect to the 1.2.3.4 server where any traffic from the internet sent to 1.2.3.4 will end up at the on-premise server. Responses from the on-premise LAN will then be routed back to 1.2.3.4 and sent to whoever originally sent the request.

Is there an open-source out of the box linux solution for this that uses OpenVPN? Should I be using something else? Are there any challenges I should anticipate when setting this up?

If there is a FAQ already for this somewhere, I'd greatly appreciate being nudged to that -- I did spend a half hour Googling different solutions but wanted to check in with the experts here because I may be googling the wrong things / terminology.

Thanks for all your assistance with this -- it is greatly appreciated!