It depends on what you really need. If you need a private access, I don’t see another solution but to pass through the private link or an ASE ( so an Isolated SKU).

If you just want a restriction, you may use the IP Restriction : https://docs.microsoft.com/en-us/azure/app-service/app-service-ip-restrictions

Thank you