Because you haven't included any details regarding the VPN (e.g., is it Windows-based, hardware-based or third-party software?) and any possible security restrictions, I'll provide some ideas that you'll be able to use to help guide you towards the best possible security implementation.

Perhaps the most important step will be to ensure the client is originating from a specific network. This will require you to set up a few access lists that will only allow specific IP addresses or network(s) to connect to your VPN server. This way, you'll be able to limit the possibility of someone unknown trying to connect to your VPN server.

In addition, depending on your VPN implementation, you might also be able to apply strict policies to allow access to specific resources such as your Terminal Server. Usually, these policies are placed on the user or group that the user belongs to and automatically gives them access only to hosts or services you want.

From the Terminal Server side, you can use the built-in firewall from where you can also place filters that will block anyone except the IP addresses you assign from connecting to it. There are also third-party programs that can log all activity and connections made to your Terminal Server, should this be desirable.

Further security enhancements can be made by defining the maximum idle time before the user is disconnected, defining specific IP addresses for possible dial-in clients and much more.

If you're looking for a quick solution and do not want to spend too much time, then try and set a few good rule sets on your VPN server and that should do the trick.

Good luck.