---
name: python-webapp-deploy
description: "Deploy Python/Flask/Node.js web applications from source code. Covers venv setup, dependency installation, env config, missing asset handling, process management, npm CLI workarounds, SQLite config, and password reset."
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
  hermes:
    tags: [devops, python, flask, nodejs, deployment, webapp, self-hosted, sqlite, npm]
    related_skills: [docker]
triggers:
  - deploy python app
  - deploy flask app
  - deploy nodejs app
  - deploy web app from source
  - python webapp deployment
  - npm global install
  - sqlite config
  - self-hosted service deploy
  - service password reset
---

# Web App Deployment (From Source)

Deploy Python, Flask, FastAPI, and Node.js web applications from source code when Docker images aren't available. Covers the full workflow from clone to running service, including npm CLI workarounds and SQLite-based configuration.

## When to Use

- Deploying Flask/FastAPI/Django apps from source
- No Docker image available or Docker not preferred
- Need to modify source code during deployment
- Internal/intranet apps that aren't published to PyPI

## Deployment Workflow

### 1. Clone or Download Source

```bash
# Standard git clone
# 1. Download via gh api (git clone/curl fail through proxy due to gnutls)
export GH_TOKEN=ghp_xxxx
gh api repos/mifengac/multi-rider/tarball/feature/ai-analyst-rag > /tmp/multi-rider.tar.gz
cd /tmp && tar xzf multi-rider.tar.gz
# Dir name will be like: mifengac-multi-rider-<hash>/
```

### 2. Create Virtual Environment

```bash
cd <target-dir>
python3 -m venv .venv
source .venv/bin/activate
python --version  # Verify
```

**Pitfall:** Always use `python3 -m venv`, not `virtualenv`. System may have different Python versions.

### 3. Install Dependencies

```bash
# Standard
pip install -r requirements.txt

# With Chinese mirror (faster in China)
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt

# If pip not in PATH, use uv
uv pip install -r requirements.txt

# For projects with local wheel dependencies
pip install --no-index --find-links ./wheels -r requirements.txt

# For lockfile-based installs
pip install -r requirements.lock
```

**Pitfall:** Large deps (torch, torchvision) may timeout. Use mirrors and increase timeout. `pip install -q` reduces output noise.

**Pitfall:** Some projects reference local `.whl` files in requirements.txt (e.g., `./wheels/clip-1.0-py3-none-any.whl`). Ensure the `wheels/` directory exists or remove those lines.

### 4. Configure Environment Variables

Most Python web apps use one of these patterns:

```bash
# Pattern A: .env file in project root
cp .env.example .env
# Edit .env

# Pattern B: app.env (loaded by app code)
cat > app.env << 'EOF'
FLASK_SECRET_KEY=change-me
APP_HOST=0.0.0.0
APP_PORT=5001
DATABASE_URL=postgresql://user:pass@host:5432/db
EOF

# Pattern C: Export directly
export FLASK_SECRET_KEY=change-me
export APP_PORT=5001
```

**Check the app's config loading code** to know which pattern it uses. Common locations:
- `shared/config/config.py`
- `config.py` or `settings.py`
- `.env.example` or `ops/app.env.example`

### 5. Create Required Directories

Apps often expect directories to exist:

```bash
# Check README or config for required dirs
mkdir -p output upload_tmp logs model datasets
```

### 6. Initialize Database

```bash
# Flask-Migrate / Alembic
flask db upgrade

# Custom migration scripts
python scripts/run_migrations.py

# Or just let the app init SQLite on first run
```

### 7. Start the Application

```bash
# Standard Flask
python app.py

# With gunicorn (production)
gunicorn -w 4 -b 0.0.0.0:5001 app:app

# Background with nohup
nohup python app.py > app.log 2>&1 &

# As a background process (Hermes)
# Use terminal with background=true
```

### 8. Verify

```bash
# Health check
curl -s http://localhost:<port>/healthz
curl -s http://localhost:<port>/api/health

# Main page
curl -s http://localhost:<port>/ | head -20
```

## Common Pitfalls

### Missing Model/Asset Files

Many ML-based apps don't ship model files in the repo (too large). Check:
- README for model download instructions
- `model/` directory structure requirements
- `.gitignore` for excluded paths

The app may start but return degraded health status. This is usually OK for initial testing.

### Oracle/Database Driver Issues

Some apps need Oracle Instant Client or other native drivers:
- Check for `instantclient_*` directories
- May need `ORACLE_IC_DIR` env var
- Thick mode vs thin mode for oracledb

### Port Already in Use

```bash
# Find what's using the port
lsof -i :5001
# or
ss -tlnp | grep 5001

# Kill or change port
export APP_PORT=5002
```

### Import Errors on Startup

Usually means missing dependencies. Check:
- `requirements.txt` vs actual imports
- Local `wheels/` directory for offline deps
- System packages (e.g., `libgl1` for OpenCV)

### Database Schema Mismatch After Migration

When the app crashes with errors like `column "X" does not exist`, the migration scripts may not create all columns the code expects. This happens when:
- `scripts/sql/v*.sql` creates a partial schema
- `scripts/ddl_create_tables.sql` has the full schema but isn't run by the migration script
- Code references columns added in a later version

**Diagnosis:**
```python
import psycopg2
conn = psycopg2.connect(host='...', port=..., database='...', user='...', password='...')
cursor = conn.cursor()
cursor.execute("SELECT column_name FROM information_schema.columns WHERE table_schema='myschema' AND table_name='mytable' ORDER BY ordinal_position")
print([r[0] for r in cursor.fetchall()])
```

**Fix — ALTER TABLE to add missing columns:**
```python
cursor.execute("ALTER TABLE myschema.mytable ADD COLUMN IF NOT EXISTS calc_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP;")
conn.commit()
```

**Pitfall:** Always check both `scripts/sql/v*.sql` AND `scripts/ddl_create_tables.sql` — the DDL file often has the complete schema while migration files are incremental.

### Tailwind CSS Not Built

If the app uses Tailwind and pages look broken:
```bash
npm install
npm run build:css
```

The compiled CSS is usually at `static/dist/tailwind.css`.

## Node.js App Deployment

When no Docker image is available and the app is Node.js-based:

### npm Global Install + Direct Execution

Many npm packages ship a CLI wrapper that may not work in all environments. The fallback is running the actual server directly:

```bash
# 1. Install globally
npm install -g <package-name>

# 2. Try CLI first
<package-name> --help

# 3. If CLI hangs/fails, find the actual entry point
ls -la $(npm root -g)/<package-name>/

# 4. Run directly with node
cd $(npm root -g)/<package-name>/app  # or appropriate dir
PORT=3000 HOSTNAME=0.0.0.0 node server.js
```

**Why CLI wrappers fail:**
- Spawn child processes with detached TTY
- Auto-update checks that hang in non-interactive environments
- System tray mode detection fails on headless servers
- Missing SQLite native deps (need runtime bootstrap)

## SQLite-Based App Configuration

Many self-hosted apps store config in SQLite:

```python
import sqlite3, json

db_path = "~/.<app>/db/data.sqlite"
conn = sqlite3.connect(db_path)
cursor = conn.cursor()

# List tables
cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")

# Read settings
cursor.execute("SELECT * FROM settings;")
rows = cursor.fetchall()
for row in rows:
    data = json.loads(row[1])  # Usually JSON in second column
    print(json.dumps(data, indent=2))
```

### Password Reset for bcrypt-Protected Apps

When you don't know the password and can't access the UI:

```python
import sqlite3, bcrypt, json

db_path = "~/.<app>/db/data.sqlite"
conn = sqlite3.connect(db_path)
cursor = conn.cursor()

new_password = "newpassword"
password_hash = bcrypt.hashpw(new_password.encode(), bcrypt.gensalt()).decode()

cursor.execute("UPDATE settings SET data = ? WHERE id = 1;",
               (json.dumps({"password": password_hash}),))
conn.commit()
```

**Requires:** `pip install bcrypt` or `uv pip install bcrypt`

### API-Based Proxy Configuration

Many services expose proxy settings via REST API:

```bash
# Login first
curl -s -X POST http://localhost:<port>/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"password":"<password>"}' -c cookies.txt

# Update proxy settings (PATCH often works better than PUT)
curl -s -X PATCH -b cookies.txt http://localhost:<port>/api/settings \
  -H "Content-Type: application/json" \
  -d '{"outboundProxyEnabled":true,"outboundProxyUrl":"http://127.0.0.1:7890"}'
```

**Pitfall:** Some APIs use PATCH for partial updates, not PUT. PUT may require all fields.

### mihomo/Clash Proxy Node Management

```bash
# List available proxies
curl -s http://127.0.0.1:9090/proxies

# Switch active node in a proxy group
curl -s -X PUT http://127.0.0.1:9090/proxies/<group-name> \
  -H "Content-Type: application/json" \
  -d '{"name":"<node-name>"}'

# Check node status (URL-encode the node name)
curl -s "http://127.0.0.1:9090/proxies/<url-encoded-name>"
```

**Pitfall:** Node names with emojis/special chars must be URL-encoded. Use Python's `urllib.parse.quote()`.

## Updating an Existing Deployment

When the app is already deployed and you need to pull latest code while preserving local config and data:

```bash
PROJECT_DIR=/home/longshao/multi-rider-rag

# 1. Backup local files that aren't in git
cp $PROJECT_DIR/.env /tmp/.env.bak 2>/dev/null
cp $PROJECT_DIR/app.env /tmp/app.env.bak 2>/dev/null
cp -r $PROJECT_DIR/.venv /tmp/.venv.bak 2>/dev/null
cp -r $PROJECT_DIR/model /tmp/model.bak 2>/dev/null
# Also backup: output/, face_data/, train_runs/, datasets/ if they have data

# 2. Download new code (gh api works through gnutls-broken proxies)
export GH_TOKEN=ghp_xxxx
gh api repos/owner/repo/tarball/branch > /tmp/repo.tar.gz
cd /tmp && tar xzf repo.tar.gz

# 3. Replace code (keep .venv and data dirs)
rm -rf $PROJECT_DIR
mv /tmp/owner-repo-*/ $PROJECT_DIR

# 4. Restore local files
cp /tmp/.env.bak $PROJECT_DIR/.env 2>/dev/null
cp /tmp/app.env.bak $PROJECT_DIR/app.env 2>/dev/null
mv /tmp/.venv.bak $PROJECT_DIR/.venv 2>/dev/null
rm -rf $PROJECT_DIR/model && mv /tmp/model.bak $PROJECT_DIR/model 2>/dev/null

# 5. Install any new dependencies
cd $PROJECT_DIR && source .venv/bin/activate
pip install -r requirements.txt

# 6. Check for new migrations
ls scripts/*.sql scripts/sql/*.sql 2>/dev/null

# 7. Restart
pkill -f "python app.py" 2>/dev/null
# Then start fresh (use terminal background=true in Hermes)
```

**Pitfall:** When `pip install -r requirements.txt` fails because a local `.whl` file is missing (e.g., `./wheels/clip-1.0-py3-none-any.whl`), filter it out and install the rest:
```bash
grep -v "wheels/" requirements.txt > /tmp/reqs-filtered.txt
pip install -r /tmp/reqs-filtered.txt
```
Then source the missing wheel separately if the feature is actually needed.

## Process Management

### Check if Running

```bash
curl -s -o /dev/null -w "%{http_code}" http://localhost:<port>/healthz
```

### Stop

```bash
# Find PID
pgrep -f "python app.py"
# Kill gracefully
kill <PID>
```

### Restart

```bash
kill $(pgrep -f "python app.py") 2>/dev/null
cd <project-dir> && source .venv/bin/activate && python app.py &
```

### npm CLI tools hanging silently
- Symptom: `<tool> --help` produces no output, times out
- Cause: CLI wrapper spawns child process, checks for updates, or waits for TTY
- Fix: Find and run the actual server entry point directly (see Node.js section above)

### SQLite database locked
- Symptom: `database is locked` error
- Cause: Another process has the DB open (the running service)
- Fix: Stop the service before modifying the DB

### bcrypt not installed
- Symptom: `ModuleNotFoundError: No module named 'bcrypt'`
- Fix: `uv pip install bcrypt` (use `uv` if `pip` not in PATH)

### API returns "Unauthorized"
- Most self-hosted services require authentication
- Check for default credentials (admin/admin, admin/password, admin/123456)
- If password unknown, reset via SQLite (see SQLite section above)

### Proxy nodes showing "alive": false
- Node may be temporarily down or blocked
- Switch to another node in the same group
- Check mihomo logs: `~/.config/mihomo/mihomo.log`

## Quick Checklist

1. [ ] Clone/download source
2. [ ] Create venv: `python3 -m venv .venv && source .venv/bin/activate`
3. [ ] Install deps: `pip install -r requirements.txt`
4. [ ] Configure env vars (check config loading pattern)
5. [ ] Create required directories
6. [ ] Initialize database if needed
7. [ ] Start app: `python app.py`
8. [ ] Verify: `curl http://localhost:<port>/healthz`
9. [ ] Check for missing models/assets in health response

## Exposing to the Internet via Tailscale Funnel

For public HTTPS access without opening firewall ports:

```bash
# One-time setup
sudo tailscale set --operator=$USER

# Enable Funnel in admin console (visit the URL printed by this command)
tailscale funnel <port>

# After admin approval, start serve + funnel in background
tailscale serve reset
tailscale serve --bg <port>
tailscale funnel --bg <port>
```

The app will be available at `https://<hostname>.<tailnet>.ts.net/` with automatic TLS. See `github-repo-management` skill's `references/github-trending-collector.md` for the full step-by-step with all pitfalls.

## Reference Files

- `references/multi-rider-example.md` — Flask + YOLO deployment with KingBase (real-world example)
- `references/schema-mismatch-and-models.md` — DDL vs migration schema mismatch fix pattern
- `references/nextjs-fullstack-deploy.md` — Next.js + Prisma + Docker infrastructure deployment
- `references/9router-deployment.md` — npm global CLI tool deployment (Node.js, SQLite config, proxy)