Sebastian Krüger
8de88d96ac
- Add setup guides (SETUP_GUIDE, TAILSCALE_SETUP, DOCKER_GPU_SETUP, etc.) - Add deployment configurations (litellm-config-gpu.yaml, gpu-server-compose.yaml) - Add GPU_DEPLOYMENT_LOG.md with current infrastructure details - Add GPU_EXPANSION_PLAN.md with complete provider comparison - Add deploy-gpu-stack.sh automation script 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
394 lines
8.4 KiB
Markdown
394 lines
8.4 KiB
Markdown
# WireGuard VPN Setup - Connecting GPU Server to VPS
|
|
|
|
## Day 3-4: Network Configuration
|
|
|
|
This guide connects your RunPod GPU server to your VPS via WireGuard VPN, enabling secure, low-latency communication.
|
|
|
|
### Architecture
|
|
|
|
```
|
|
┌─────────────────────────────┐ ┌──────────────────────────────┐
|
|
│ VPS (pivoine.art) │ │ GPU Server (RunPod) │
|
|
│ 10.8.0.1 (WireGuard) │◄───────►│ 10.8.0.2 (WireGuard) │
|
|
├─────────────────────────────┤ ├──────────────────────────────┤
|
|
│ - LiteLLM Proxy │ │ - vLLM (10.8.0.2:8000) │
|
|
│ - Open WebUI │ │ - ComfyUI (10.8.0.2:8188) │
|
|
│ - PostgreSQL │ │ - Training │
|
|
└─────────────────────────────┘ └──────────────────────────────┘
|
|
```
|
|
|
|
### Prerequisites
|
|
|
|
- ✅ VPS with root access
|
|
- ✅ GPU server with root access
|
|
- ✅ Both servers have public IPs
|
|
|
|
---
|
|
|
|
## Method 1: Using Existing wg-easy (Recommended)
|
|
|
|
You already have `wg-easy` running on your VPS. Let's use it!
|
|
|
|
### Step 1: Access wg-easy Dashboard
|
|
|
|
**On your local machine:**
|
|
|
|
1. Open browser: https://vpn.pivoine.art (or whatever your wg-easy URL is)
|
|
2. Login with admin password
|
|
|
|
**Don't have wg-easy set up? Skip to Method 2.**
|
|
|
|
### Step 2: Create GPU Server Client
|
|
|
|
1. In wg-easy dashboard, click **"+ New Client"**
|
|
2. **Name**: `gpu-server-runpod`
|
|
3. Click **"Create"**
|
|
4. **Download** configuration file (or copy QR code data)
|
|
|
|
You'll get a file like: `gpu-server-runpod.conf`
|
|
|
|
### Step 3: Install WireGuard on GPU Server
|
|
|
|
**SSH into GPU server:**
|
|
|
|
```bash
|
|
ssh gpu-pivoine # or your SSH command
|
|
|
|
# Install WireGuard
|
|
apt update
|
|
apt install -y wireguard wireguard-tools
|
|
```
|
|
|
|
### Step 4: Configure WireGuard on GPU Server
|
|
|
|
**Upload the config file:**
|
|
|
|
```bash
|
|
# On your local machine, copy the config to GPU server
|
|
scp gpu-server-runpod.conf gpu-pivoine:/etc/wireguard/wg0.conf
|
|
|
|
# Or manually create it on GPU server:
|
|
nano /etc/wireguard/wg0.conf
|
|
# Paste the configuration from wg-easy
|
|
```
|
|
|
|
**Example config (yours will be different):**
|
|
```ini
|
|
[Interface]
|
|
PrivateKey = <PRIVATE_KEY_FROM_WG_EASY>
|
|
Address = 10.8.0.2/24
|
|
DNS = 10.8.0.1
|
|
|
|
[Peer]
|
|
PublicKey = <VPS_PUBLIC_KEY_FROM_WG_EASY>
|
|
PresharedKey = <PRESHARED_KEY>
|
|
AllowedIPs = 10.8.0.0/24
|
|
Endpoint = <VPS_PUBLIC_IP>:51820
|
|
PersistentKeepalive = 25
|
|
```
|
|
|
|
### Step 5: Start WireGuard
|
|
|
|
```bash
|
|
# Enable IP forwarding
|
|
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
|
|
sysctl -p
|
|
|
|
# Set permissions
|
|
chmod 600 /etc/wireguard/wg0.conf
|
|
|
|
# Start WireGuard
|
|
systemctl enable wg-quick@wg0
|
|
systemctl start wg-quick@wg0
|
|
|
|
# Check status
|
|
systemctl status wg-quick@wg0
|
|
wg show
|
|
```
|
|
|
|
Expected output:
|
|
```
|
|
interface: wg0
|
|
public key: <GPU_SERVER_PUBLIC_KEY>
|
|
private key: (hidden)
|
|
listening port: 51820
|
|
|
|
peer: <VPS_PUBLIC_KEY>
|
|
endpoint: <VPS_IP>:51820
|
|
allowed ips: 10.8.0.0/24
|
|
latest handshake: 1 second ago
|
|
transfer: 1.2 KiB received, 892 B sent
|
|
persistent keepalive: every 25 seconds
|
|
```
|
|
|
|
### Step 6: Test Connectivity
|
|
|
|
**From GPU server, ping VPS:**
|
|
|
|
```bash
|
|
ping 10.8.0.1 -c 4
|
|
```
|
|
|
|
Expected output:
|
|
```
|
|
PING 10.8.0.1 (10.8.0.1) 56(84) bytes of data.
|
|
64 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=25.3 ms
|
|
64 bytes from 10.8.0.1: icmp_seq=2 ttl=64 time=24.8 ms
|
|
...
|
|
```
|
|
|
|
**From VPS, ping GPU server:**
|
|
|
|
```bash
|
|
ssh root@vps
|
|
ping 10.8.0.2 -c 4
|
|
```
|
|
|
|
**Test PostgreSQL access from GPU server:**
|
|
|
|
```bash
|
|
# On GPU server
|
|
apt install -y postgresql-client
|
|
|
|
# Try connecting to VPS postgres
|
|
psql -h 10.8.0.1 -U valknar -d openwebui -c "SELECT 1;"
|
|
# Should work if postgres allows 10.8.0.0/24
|
|
```
|
|
|
|
---
|
|
|
|
## Method 2: Manual WireGuard Setup (If no wg-easy)
|
|
|
|
### Step 1: Install WireGuard on Both Servers
|
|
|
|
**On VPS:**
|
|
```bash
|
|
ssh root@vps
|
|
apt update
|
|
apt install -y wireguard wireguard-tools
|
|
```
|
|
|
|
**On GPU Server:**
|
|
```bash
|
|
ssh gpu-pivoine
|
|
apt update
|
|
apt install -y wireguard wireguard-tools
|
|
```
|
|
|
|
### Step 2: Generate Keys
|
|
|
|
**On VPS:**
|
|
```bash
|
|
cd /etc/wireguard
|
|
umask 077
|
|
wg genkey | tee vps-private.key | wg pubkey > vps-public.key
|
|
```
|
|
|
|
**On GPU Server:**
|
|
```bash
|
|
cd /etc/wireguard
|
|
umask 077
|
|
wg genkey | tee gpu-private.key | wg pubkey > gpu-public.key
|
|
```
|
|
|
|
### Step 3: Create Config on VPS
|
|
|
|
**On VPS (`/etc/wireguard/wg0.conf`):**
|
|
|
|
```bash
|
|
cat > /etc/wireguard/wg0.conf << 'EOF'
|
|
[Interface]
|
|
PrivateKey = <VPS_PRIVATE_KEY>
|
|
Address = 10.8.0.1/24
|
|
ListenPort = 51820
|
|
SaveConfig = false
|
|
|
|
# GPU Server Peer
|
|
[Peer]
|
|
PublicKey = <GPU_PUBLIC_KEY>
|
|
AllowedIPs = 10.8.0.2/32
|
|
PersistentKeepalive = 25
|
|
EOF
|
|
```
|
|
|
|
Replace `<VPS_PRIVATE_KEY>` with contents of `vps-private.key`
|
|
Replace `<GPU_PUBLIC_KEY>` with contents from GPU server's `gpu-public.key`
|
|
|
|
### Step 4: Create Config on GPU Server
|
|
|
|
**On GPU Server (`/etc/wireguard/wg0.conf`):**
|
|
|
|
```bash
|
|
cat > /etc/wireguard/wg0.conf << 'EOF'
|
|
[Interface]
|
|
PrivateKey = <GPU_PRIVATE_KEY>
|
|
Address = 10.8.0.2/24
|
|
|
|
[Peer]
|
|
PublicKey = <VPS_PUBLIC_KEY>
|
|
AllowedIPs = 10.8.0.0/24
|
|
Endpoint = <VPS_PUBLIC_IP>:51820
|
|
PersistentKeepalive = 25
|
|
EOF
|
|
```
|
|
|
|
Replace:
|
|
- `<GPU_PRIVATE_KEY>` with contents of `gpu-private.key`
|
|
- `<VPS_PUBLIC_KEY>` with contents from VPS's `vps-public.key`
|
|
- `<VPS_PUBLIC_IP>` with your VPS's public IP address
|
|
|
|
### Step 5: Start WireGuard on Both
|
|
|
|
**On VPS:**
|
|
```bash
|
|
# Enable IP forwarding
|
|
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
|
|
sysctl -p
|
|
|
|
# Start WireGuard
|
|
chmod 600 /etc/wireguard/wg0.conf
|
|
systemctl enable wg-quick@wg0
|
|
systemctl start wg-quick@wg0
|
|
```
|
|
|
|
**On GPU Server:**
|
|
```bash
|
|
# Enable IP forwarding
|
|
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
|
|
sysctl -p
|
|
|
|
# Start WireGuard
|
|
chmod 600 /etc/wireguard/wg0.conf
|
|
systemctl enable wg-quick@wg0
|
|
systemctl start wg-quick@wg0
|
|
```
|
|
|
|
### Step 6: Configure Firewall
|
|
|
|
**On VPS:**
|
|
```bash
|
|
# Allow WireGuard port
|
|
ufw allow 51820/udp
|
|
ufw reload
|
|
|
|
# Or with iptables
|
|
iptables -A INPUT -p udp --dport 51820 -j ACCEPT
|
|
iptables-save > /etc/iptables/rules.v4
|
|
```
|
|
|
|
**On GPU Server (RunPod):**
|
|
```bash
|
|
# Allow WireGuard
|
|
ufw allow 51820/udp
|
|
ufw reload
|
|
```
|
|
|
|
### Step 7: Test Connection
|
|
|
|
Same as Method 1 Step 6.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### No handshake
|
|
|
|
**Check:**
|
|
```bash
|
|
wg show
|
|
```
|
|
|
|
If "latest handshake" shows "never":
|
|
1. Verify public keys are correct (easy to swap them!)
|
|
2. Check firewall allows UDP 51820
|
|
3. Verify endpoint IP is correct
|
|
4. Check `systemctl status wg-quick@wg0` for errors
|
|
|
|
### Can ping but can't access services
|
|
|
|
**On VPS, check PostgreSQL allows 10.8.0.0/24:**
|
|
|
|
```bash
|
|
# Edit postgresql.conf
|
|
nano /var/lib/postgresql/data/postgresql.conf
|
|
# Add or modify:
|
|
listen_addresses = '*'
|
|
|
|
# Edit pg_hba.conf
|
|
nano /var/lib/postgresql/data/pg_hba.conf
|
|
# Add:
|
|
host all all 10.8.0.0/24 scram-sha-256
|
|
|
|
# Restart
|
|
docker restart core_postgres
|
|
```
|
|
|
|
### WireGuard won't start
|
|
|
|
```bash
|
|
# Check logs
|
|
journalctl -u wg-quick@wg0 -n 50
|
|
|
|
# Common issues:
|
|
# - Wrong permissions: chmod 600 /etc/wireguard/wg0.conf
|
|
# - Invalid keys: regenerate with wg genkey
|
|
# - Port already in use: lsof -i :51820
|
|
```
|
|
|
|
---
|
|
|
|
## Verification Checklist
|
|
|
|
Before proceeding to Day 5:
|
|
|
|
- [ ] WireGuard installed on both VPS and GPU server
|
|
- [ ] VPN tunnel established (wg show shows handshake)
|
|
- [ ] GPU server can ping VPS (10.8.0.1)
|
|
- [ ] VPS can ping GPU server (10.8.0.2)
|
|
- [ ] Firewall allows WireGuard (UDP 51820)
|
|
- [ ] PostgreSQL accessible from GPU server
|
|
- [ ] WireGuard starts on boot (systemctl enable)
|
|
|
|
---
|
|
|
|
## Network Reference
|
|
|
|
**VPN IPs:**
|
|
- VPS: `10.8.0.1`
|
|
- GPU Server: `10.8.0.2`
|
|
|
|
**Service Access from GPU Server:**
|
|
- PostgreSQL: `postgresql://valknar:password@10.8.0.1:5432/dbname`
|
|
- Redis: `10.8.0.1:6379`
|
|
- LiteLLM: `http://10.8.0.1:4000`
|
|
- Mailpit: `10.8.0.1:1025`
|
|
|
|
**Service Access from VPS:**
|
|
- vLLM: `http://10.8.0.2:8000`
|
|
- ComfyUI: `http://10.8.0.2:8188`
|
|
- JupyterLab: `http://10.8.0.2:8888`
|
|
|
|
---
|
|
|
|
## Next: Docker & GPU Setup
|
|
|
|
Once VPN is working, proceed to **Day 5: Docker & NVIDIA Container Toolkit Setup**.
|
|
|
|
**Save connection info:**
|
|
|
|
```bash
|
|
# On GPU server
|
|
cat >> /workspace/SERVER_INFO.md << 'EOF'
|
|
|
|
## VPN Configuration
|
|
- VPN IP: 10.8.0.2
|
|
- VPS VPN IP: 10.8.0.1
|
|
- WireGuard Status: Active
|
|
- Latest Handshake: [Check with: wg show]
|
|
|
|
## Network Access
|
|
- Can reach VPS services: ✓
|
|
- VPS can reach GPU services: ✓
|
|
EOF
|
|
```
|