---
name: docker
description: "Docker on Linux: install Engine + Compose, configure proxy (China/mihomo), run databases (PostgreSQL, MySQL, KingbaseES), MinIO object storage, deploy full-stack apps. Pitfalls: sudo-via-pipe, GPG keys, socket permissions, root-can't-run-db."
tags: [docker, devops, linux, containers, database, postgresql, mysql, kingbase, minio, proxy, deployment, full-stack]
triggers:
  - install docker
  - setup docker
  - docker engine
  - docker compose
  - database docker container
  - run database in docker
  - restore database docker
  - kingbase docker
  - postgresql docker
  - mysql docker container
  - minio docker
  - object storage docker
  - full stack deploy docker
  - prisma docker
  - docker proxy
  - docker hub unreachable
  - docker permission denied
---

# Docker: Install, Configure, and Run Services

Complete Docker workflow on Linux — from installation through running databases, object storage, and full-stack applications in containers.

---

## Part 1: Installation (Ubuntu/Debian)

### Prerequisites Check

```bash
cat /etc/os-release | head -5   # OS version
which docker 2>/dev/null        # already installed?
whoami && id                    # current user
```

### Standard Install (requires root/sudo)

**When sudo needs a password:** Use `echo 'PASSWORD' | sudo -S COMMAND` pattern. See pitfalls below.

1. **Install dependencies**
   ```bash
   sudo apt-get update -y
   sudo apt-get install -y ca-certificates curl gnupg
   ```

2. **Add Docker GPG key** — download first, then dearmor (do NOT pipe curl into gpg when using sudo-via-pipe)
   ```bash
   curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /tmp/docker.gpg
   sudo install -m 0755 -d /etc/apt/keyrings
   sudo gpg --batch --yes --dearmor -o /etc/apt/keyrings/docker.gpg /tmp/docker.gpg
   sudo chmod a+r /etc/apt/keyrings/docker.gpg
   ```

3. **Add Docker apt repo** — write to temp file, then copy (tee consumes stdin)
   ```bash
   ARCH=$(dpkg --print-architecture)
   echo "deb [arch=$ARCH signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" > /tmp/docker.list
   sudo cp /tmp/docker.list /etc/apt/sources.list.d/docker.list
   ```

4. **Install Docker packages**
   ```bash
   sudo apt-get update -y
   sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
   ```

5. **Post-install setup**
   ```bash
   sudo usermod -aG docker $USER    # add user to docker group
   sudo systemctl start docker
   sudo systemctl enable docker
   ```

6. **Verify**
   ```bash
   docker --version
   docker compose version
   docker info | head -5
   ```

### Rootless Docker (no root needed)

For environments without sudo access. Requires `/etc/subuid` and `/etc/subgid` entries for the user.

```bash
cat /etc/subuid | grep $(whoami)
cat /etc/subgid | grep $(whoami)
# If missing, ask admin to add: username:100000:65536

dockerd-rootless-setuptool.sh install
```

### Passwordless sudo (convenience)

```bash
sudo usermod -aG sudo $USER
sudo bash -c 'echo "$USER ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/$USER && chmod 440 /etc/sudoers.d/$USER'
```

⚠️ Full NOPASSWD sudo = unrestricted root. Fine for dev machines, risky in production.

---

## Part 2: Proxy Configuration (China / Restricted Networks)

China Docker mirrors are unreliable (rate-limited, frequently down). Using a local proxy (mihomo/Clash) is more reliable.

### Find proxy port

```bash
ps aux | grep -i clash | grep -v grep
ss -tlnp | grep -E '7890|7891|7892'
```

Common mihomo/Clash ports:
- `7890` — mixed (HTTP + SOCKS5) ← **use this one**
- `7891` — HTTP only
- `7892` — SOCKS5 only
- `9090` — API/dashboard

### Verify proxy works for Docker Hub

```bash
curl -x http://127.0.0.1:7890 -s --connect-timeout 10 https://registry-1.docker.io/v2/
# Should return: {"errors":[{"code":"UNAUTHORIZED",...}]}  ← means it works
```

### Configure Docker systemd proxy

```bash
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/proxy.conf <<'EOF'
[Service]
Environment="HTTP_PROXY=http://127.0.0.1:7890"
Environment="HTTPS_PROXY=http://127.0.0.1:7890"
Environment="NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
EOF
sudo systemctl daemon-reload && sudo systemctl restart docker
```

> **Note:** Do NOT put proxy in `/etc/docker/daemon.json` — that file is for registry mirrors and other daemon config. Systemd override is the correct place for proxy env vars.

### Clear mirror config if present

```bash
sudo tee /etc/docker/daemon.json <<'EOF'
{}
EOF
sudo systemctl restart docker
```

### Test proxy connectivity

```bash
curl -x http://127.0.0.1:7890 -s --connect-timeout 5 https://registry-1.docker.io/v2/
```

**Clash/mihomo:** Use the mixed port (7890) rather than the HTTP port (7891) — the mixed port handles both HTTP and HTTPS better.

### China Mirror Fallback (if no proxy available)

```json
// /etc/docker/daemon.json
{
  "registry-mirrors": [
    "https://mirror.ccs.tencentyun.com",
    "https://registry.docker-cn.com",
    "https://docker.1ms.run"
  ]
}
```
Expect rate limiting (429) on many mirrors.

---

## Part 3: Running Database Containers

### General Workflow

#### 1. Pull and inspect the image

```bash
docker pull IMAGE:TAG
docker inspect IMAGE:TAG --format '{{json .Config.Entrypoint}}'
docker inspect IMAGE:TAG --format '{{json .Config.Env}}'
```

Always inspect the entrypoint script and env vars — different images use different variable names for user/password/database.

#### 2. Start the container

```bash
docker run -d \
  --name db-container \
  --user <db_user> \          # run as the database's expected user, NOT root
  -p 54321:54321 \
  -e DB_USER=myuser \
  -e DB_PASSWORD=mypass \
  -v /host/data:/container/data \
  IMAGE:TAG
```

#### 3. Verify startup

```bash
docker ps | grep db-container
docker logs db-container 2>&1 | tail -20
```

#### 4. Create databases and users

```bash
docker exec db-container bash -c 'source /etc/profile && TOOL -U admin_user -d default_db -c "CREATE DATABASE mydb;"'
```

### Critical: User Permission Issues

**Root cannot run database engines.** Most databases (PostgreSQL, KingbaseES, MySQL) **refuse to run as root** for security:
```
initdb: error: cannot be run as root
```

**Do NOT use `--user root`**. Use the image's intended user (often `postgres`, `kingbase`, `mysql`).

**`--privileged` flag:** Useful when running as the image's intended user (e.g., `kingbase`) and the entrypoint script uses `sudo` heavily. Helps avoid PAM permission errors without needing root:
```bash
docker run -d --name kingbasev8 --user kingbase --privileged -p 54321:54321 ...
```
Do NOT combine `--privileged` with `--user root` — same initdb issue.

### sudo PAM errors in container logs

If the entrypoint script uses `sudo` and you see:
```
sudo: pam_open_session: Permission denied
sudo: policy plugin failed session initialization
```

This is usually **non-critical** — it affects cron/logrotate setup, not the database itself. Check if the database actually started:
```
waiting for server to start.... done
server started   ← this means it's fine
```

### Environment Variable Discovery

Different images use different env var names. **Always inspect the entrypoint script**:

```bash
docker exec db-container cat /home/USER/docker-entrypoint.sh
```

| Image | User var | Password var | Database var |
|-------|----------|-------------|-------------|
| PostgreSQL | `POSTGRES_USER` | `POSTGRES_PASSWORD` | `POSTGRES_DB` |
| KingbaseES | `DB_USER` | `DB_PASSWORD` or `PASSWORD` | (created manually) |
| MySQL | `MYSQL_USER` | `MYSQL_PASSWORD` | `MYSQL_DATABASE` |

### Backup & Restore

**Identify the dump format:**
```bash
file backup.dmp
head -c 500 backup.dmp | cat -v
```

- **SQL text** → restore with `psql` / `ksql` / `mysql`
- **Custom binary** → restore with platform tool (`pg_restore`, `sys_restore`, `mysqlrestore`)

**Copy file into container:**
```bash
docker cp backup.dmp db-container:/tmp/backup.dmp
```

**Restore with --no-owner** (avoids "role does not exist" errors):
```bash
# PostgreSQL
docker exec db-container pg_restore -U user -d dbname --no-owner /tmp/backup.dmp

# KingbaseES
docker exec db-container bash -c 'source /etc/profile && /path/to/sys_restore -U user -d dbname --no-owner /tmp/backup.dmp'
```

**Preview dump contents before restoring:**
```bash
# List all objects in the dump (PostgreSQL)
docker exec db-container pg_restore -l /tmp/backup.dmp | head -60

# KingbaseES
docker exec db-container sys_restore -l /tmp/backup.dmp | head -60
```

**Schema-only restore** (skip data, only create tables/views/functions):
```bash
docker exec db-container sys_restore -U user -d dbname --schema-only --no-owner --no-privileges /tmp/backup.dmp
```

Useful when target DB already has data and you only need to add missing table structures. Combine with `-v` for verbose output to see which objects are created vs skipped.

**Handle "already exists" errors:**
```bash
# Terminate connections
docker exec db-container psql -U user -d default_db -c \
  "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = 'target_db' AND pid <> pg_backend_pid();"

# Drop and recreate
docker exec db-container psql -U user -d default_db -c "DROP DATABASE target_db;"
docker exec db-container psql -U user -d default_db -c "CREATE DATABASE target_db;"

# Restore
docker exec db-container pg_restore -U user -d target_db --no-owner /tmp/backup.dmp
```

**Verify restore:**
```bash
docker exec db-container psql -U user -d dbname -c "\dn"
docker exec db-container psql -U user -d dbname -c \
  "SELECT schemaname, count(*) FROM pg_tables WHERE schemaname NOT IN ('pg_catalog','information_schema') GROUP BY schemaname ORDER BY schemaname;"
```

### Connecting to the Database

From host:
```bash
psql -h 127.0.0.1 -p 54321 -U username -d dbname
```

From another container:
```bash
docker run --rm -it --network host IMAGE psql -h 127.0.0.1 -p 54321 -U user -d db
```

---

## Part 4: MinIO Object Storage

### Deploy MinIO

```bash
sudo docker run -d \
  --name minio \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  -p 9000:9000 \
  -p 9001:9001 \
  -v minio_data:/data \
  --restart unless-stopped \
  minio/minio server /data --console-address ":9001"
```

- **Port 9000** — S3 API endpoint
- **Port 9001** — Web console (browser UI)
- **Volume** — `minio_data` for persistent storage

### Create a bucket

Via CLI (install `mc` first) or via the console at `http://localhost:9001`.

### Connection from app

```env
MINIO_ENDPOINT="http://localhost:9000"
MINIO_ACCESS_KEY="minioadmin"
MINIO_SECRET_KEY="minioadmin"
MINIO_BUCKET="my-bucket"
MINIO_USE_SSL="false"
```

---

## Part 5: Full-Stack Deployment Pattern

For apps with database + object storage + app server (e.g., Next.js + Prisma + PostgreSQL + MinIO):

1. **Docker services:** PostgreSQL, MinIO (and any other infra)
2. **Local app:** `npm install` → `npx prisma db push` → `npm run dev`
3. **`.env` file:** Point `DATABASE_URL` and `MINIO_ENDPOINT` to `localhost` (not Docker service names, since app runs on host)

Key: Docker containers use `--name` for inter-container DNS, but host-to-container uses `localhost` + published ports.

---

## Part 6: GitHub Authentication for Cloning

When `gh` CLI isn't available or token lacks `read:org` scope, use HTTPS + stored credentials:

```bash
git config --global credential.helper store
# Token stored in ~/.git-credentials after first use
git clone https://USER:TOKEN@github.com/USER/REPO.git
```

---

## Pitfalls

### P1: `sudo -S` + pipes break stdin
`echo 'pw' | sudo -S cmd1 | cmd2` — the second pipe consumes stdin, so sudo's password prompt may fail. **Fix:** chain commands with `&&` in a single `sudo -S sh -c '...'` or run each step as separate terminal calls.

### P2: GPG dearmor fails with "cannot open /dev/tty"
When running `gpg --dearmor` non-interactively (via pipe/script), add `--batch` flag:
```bash
sudo gpg --batch --yes --dearmor -o /etc/apt/keyrings/docker.gpg /tmp/docker.gpg
```

### P3: GPG "dearmoring failed: File exists"
If `/etc/apt/keyrings/docker.gpg` already exists, gpg won't overwrite. Add `--yes` flag or `sudo rm` the file first.

### P4: `tee` consumes stdin with sudo pipe
`echo 'pw' | sudo -S tee /path/to/file` works for the tee itself, but if you pipe content INTO it, the content pipe gets broken. **Fix:** write to `/tmp/file` first, then `sudo cp`.

### P5: Docker repo not appearing in apt update
Check `/etc/apt/sources.list.d/docker.list` exists and has correct content. The `tee` stdin issue (P4) often causes an empty file.

### P6: "permission denied" on docker.sock despite user in docker group
After `usermod -aG docker $USER`, the group change doesn't take effect until a new login session.

**Quick fix:**
```bash
sudo chmod 666 /var/run/docker.sock
```

**Better fix** (persists across reboots):
```bash
groups $USER   # verify 'docker' appears
sudo chown root:docker /var/run/docker.sock
sudo chmod 660 /var/run/docker.sock
newgrp docker   # activate group without logout
```

### P7: Docker Hub unreachable (IPv6 timeout or China network)
If `docker pull` fails with timeout on IPv6 address, or China mirrors return 429/timeout:
- **Best option:** Configure proxy via systemd (see Part 2)
- Use mixed proxy port (e.g. 7890), not HTTP-only port
- Verify proxy works: `curl -x http://127.0.0.1:7890 -s https://registry-1.docker.io/v2/`

### P8: "repository does not exist or may require docker login"
If `docker pull` returns this error after proxy is confirmed working, the repo is private:
```bash
curl -x http://127.0.0.1:7890 -s "https://hub.docker.com/v2/repositories/OWNER/REPO/tags/?page_size=10"
# {"message":"object not found"} → repo doesn't exist
# 401/403 → private repo, need: sudo docker login
```

### P9: Kernel upgrade pending
Docker may warn about kernel version mismatch. Not blocking but recommend reboot when convenient.

### P10: "role does not exist" during restore
The dump references a role that doesn't exist in the target. Use `--no-owner` or create the role first.

### P11: "database is being accessed by other users"
Can't drop a database with active connections. Terminate them first (see Backup & Restore section).

### P12: Container starts but database not ready
Some databases take 10-30 seconds to initialize. Wait and check logs:
```bash
sleep 15 && docker logs db-container 2>&1 | tail -10
```

### P13: Default password in image
Many database images have a hardcoded default password (often base64-encoded in the entrypoint). Always set your own via env vars.

### P14: Encoding mismatch
If restoring data with special characters (Chinese, etc.), ensure the target database encoding matches the source (e.g., `UTF-8`, `GBK`).

### P19: GitHub access through mihomo (gnutls TLS failure)
`git clone`, `git fetch`, `curl`, and Python `requests` all fail through mihomo when curl is compiled with gnutls (exit code 35: `gnutls_handshake() failed`). This is a known incompatibility between gnutls and mihomo's TLS interception.

**Proven workaround — use `gh api`:**
```bash
# Authenticate (one-time)
echo "TOKEN" | gh auth login --with-token

# Download repo tarball for specific branch
gh api repos/OWNER/REPO/tarball/BRANCH > /tmp/repo.tar.gz

# Extract
cd /tmp && tar xzf repo.tar.gz
# Creates: OWNER-REPO-HASH/
```

`gh` is a Go binary with its own TLS stack — it bypasses the gnutls issue entirely. No proxy env vars needed.

**Quick check:** `curl --version | head -1` — if it says "GnuTLS", expect TLS failures through mihomo.

### P17: NAT traversal for external access
When the app runs on an internal network and needs internet access:
- **Router port forwarding** is the most reliable
- **Cloudflare quick tunnels** (recommended, no registration): `cloudflared tunnel --url http://localhost:3000` → temporary `*.trycloudflare.com` HTTPS URL
- **ngrok** requires account + auth token; **localtunnel/serveo** often fail on restricted networks
- If the machine has a public IPv4 (check `curl -4 ifconfig.me`), port forwarding is simplest
- Always test external access: `curl -4 -s -o /dev/null -w "%{http_code}" http://PUBLIC_IP:PORT`

### P18: Tailscale Funnel for HTTP services
Tailscale Funnel exposes a local HTTP/HTTPS service directly to the internet through Tailscale's infrastructure, with automatic HTTPS certificates. No port forwarding or public IP needed.

**Prerequisites:**
- Tailscale installed and running on the machine
- Funnel enabled in the Tailscale admin console (one-time per tailnet)
- User granted operator permissions: `sudo tailscale set --operator=$USER`

**Setup:**
```bash
# 1. Grant operator permission (one-time, avoids sudo for funnel commands)
sudo tailscale set --operator=$USER

# 2. Start a local HTTP server (if not already running)
python3 -m http.server 8080  # or your app on its port

# 3. Expose via Tailscale Funnel (background)
tailscale funnel --bg 8080

# Check status
tailscale serve status
# Output shows: https://<hostname>.tail<id>.ts.net/ (Funnel on)
```

**Result:** Service is reachable at `https://<hostname>.tail<id>.ts.net/` from anywhere on the internet.

**Key points:**
- Funnel only supports HTTP/HTTPS (TCP only). It does NOT expose SSH or other protocols.
- The `tailscale funnel` command must stay running (use `--bg` for background).
- If the process exits, the funnel stops. Use `systemd --user` to persist it:
  ```ini
  # ~/.config/systemd/user/tailscale-funnel.service
  [Unit]
  Description=Tailscale Funnel for HTTP server
  After=network-online.target tailscaled.service

  [Service]
  Type=simple
  ExecStart=/usr/bin/tailscale funnel 8080
  Restart=on-failure
  RestartSec=10

  [Install]
  WantedBy=default.target
  ```
  ```bash
  systemctl --user daemon-reload
  systemctl --user enable --now tailscale-funnel.service
  ```
- If you get "foreground listener already exists for port 443", reset first: `tailscale serve reset`
- Local testing: `curl -s -o /dev/null -w "%{http_code}" http://localhost:8080/` (should be 200)
- **Proxy interference:** If you have `http_proxy` env vars set, `curl` may try to route through them and fail for localhost. Unset them for local testing: `unset http_proxy https_proxy`

---

## Reference Files

- `references/kingbasev8.md` — KingbaseES V8 specific notes (image, env vars, tools, restore workflow)
- `references/cloudflare-tunnels.md` — Expose local services to internet via Cloudflare tunnels
- `references/jellyfin-media-server.md` — Jellyfin media/TV streaming server deployment
- `references/mihomo-proxy.md` — mihomo/Clash status checks, API usage, proxy IP verification, curl TLS pitfalls
- `references/tailscale-funnel.md` — Expose local HTTP services to internet via Tailscale Funnel (HTTPS, no port forwarding)