IPsec and private IP

Question

I'm creating an IPsec / L2TP network and I want to have one server and multiple clients.

On each client, the only way I got IPsec to work properly was to set the IP of the interface that will host the traffic as the leftsourceip. It seems ok when this interface is public, but it seems kinda wrong when the machine is behind a NAT and the interface has a private IP.

Here is an example of the configuration of ipsec from the client :

config setup
    charondebug="ike 1, knl 1, cfg 0"
    uniqueids=no

conn ikev2-vpn
    auto=route
    compress=no
    type=tunnel
    keyexchange=ikev2
    fragmentation=yes
    forceencaps=yes
    dpdaction=clear
    dpddelay=300s
    esp=aes256-sha256-modp4096!
    ike=aes256-sha256-modp4096!
    rekey=no
    right=ipsec.server.url
    rightid=%any
    rightauth=pubkey
    rightprotoport=17/1701
    left=10.0.0.60
    leftsubnet=10.0.0.60/32
    leftid=%any
    leftauth=pubkey
    leftcert=my-cert.crt
    leftsendcert=always
    leftprotoport=17/1701
    leftsourceip=10.0.0.60

The ip of this client is 10.0.0.60, so ipsec is creating tunnel between the server/32 and this ip/32 and it's working well.

But since it's a private IP, I could have two clients with the same IP, so it won't work anymore. And don't think trying to put the public ip of this client is a good idea because I can have multiple clients with the same IP.

I can see at least two solutions. If I manually give a virtual IP to my outgoing interface on my client, I can easily avoid to give the same IP to multiple clients, but it requires a manual action and some clients won't allow me to give a new address (embedded hardware).

The other solution would be to configure ipsec to distribute virtual ip, but in that case I feel like I'm setting up a L2 VPN inside a L3 VPN and I would have two IP network on top of each other.

I'm confused, am I missing something ? What is the right way to configure IPsec with L2TP ?

Thanks