# GitHub Trending Repository Collector

Pattern for building a daily GitHub trending projects collector that generates a static HTML page.

## GitHub Search API for Trending Repos

```python
from datetime import datetime, timedelta
from urllib.request import Request, urlopen
import json

GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")

def github_api(path: str) -> dict:
    url = f"https://api.github.com/{path}"
    headers = {
        "Accept": "application/vnd.github.v3+json",
        "User-Agent": "GitHub-Trending-Collector/1.0",
    }
    if GITHUB_TOKEN:
        headers["Authorization"] = f"token {GITHUB_TOKEN}"
    req = Request(url, headers=headers)
    with urlopen(req, timeout=30) as resp:
        return json.loads(resp.read().decode())

# Search repos created in last 7 days, sorted by stars
since_date = (datetime.utcnow() - timedelta(days=7)).strftime("%Y-%m-%d")
languages = ["python", "javascript", "typescript", "go", "rust", "java", "c++", "all"]

for lang in languages:
    lang_filter = f"+language:{lang}" if lang != "all" else ""
    query = f"created:>={since_date}{lang_filter}"
    path = f"search/repositories?q={query}&sort=stars&order=desc&per_page=10"
    result = github_api(path)
    # Process result["items"]...
```

## Key API Parameters

| Parameter | Value | Purpose |
|-----------|-------|---------|
| `q` | `created:>=YYYY-MM-DD` | Filter by creation date |
| `q` | `+language:python` | Filter by language |
| `sort` | `stars` | Sort by popularity |
| `order` | `desc` | Descending order |
| `per_page` | `10` | Results per page (max 100) |

## Rate Limiting

- Without token: 60 requests/hour
- With token: 5000 requests/hour
- Always use a token for production use
- Deduplicate by repo `id` when searching across languages

## Deployment Pattern

Complete pipeline for periodic data collection + static web display:

1. **Collector script** — fetches data via API, generates HTML
2. **HTTP server** — `python3 -m http.server 8080` in the output directory
3. **Cron job** — daily schedule via `hermes cron create`
4. **Systemd service** — auto-start HTTP server on boot

### Systemd User Service for HTTP Server

```ini
# ~/.config/systemd/user/github-trending.service
[Unit]
Description=GitHub Trending HTTP Server
After=network.target

[Service]
Type=simple
User=%u
WorkingDirectory=/path/to/output
ExecStart=/usr/bin/python3 -m http.server 8080
Restart=on-failure
RestartSec=10

[Install]
WantedBy=default.target
```

```bash
systemctl --user daemon-reload
systemctl --user enable github-trending.service
systemctl --user start github-trending.service
```

### Hermes Cron Job

```bash
hermes cron create "0 9 * * *"  # Daily at 9 AM
# Prompt: "Run /path/to/fetch_trending.py to update the trending page"
```

## HTML Generation Tips

- Dark theme matching GitHub's color scheme (`#0d1117` background)
- Language color dots using GitHub's language colors
- Responsive grid layout (`grid-template-columns: repeat(auto-fill, minmax(350px, 1fr))`)
- Language-based navigation with anchor links
- Truncate descriptions to ~120 chars for consistent card heights
- Format large numbers: `1.5k` instead of `1500`

## Public Access via Tailscale Funnel

To expose the HTTP server to the internet (not just the tailnet):

### Prerequisites

```bash
# Check current Tailscale account and status
tailscale status --json | python3 -c "
import sys, json
d = json.load(sys.stdin)
self = d.get('Self', {})
print(f'Account: {self.get(\"UserID\")}')
print(f'Hostname: {self.get(\"HostName\")}')
print(f'Tailscale IP: {self.get(\"TailscaleIPs\", [\"N/A\"])[0]}')
print(f'DNS: {self.get(\"DNSName\")}')
"
```

### Step-by-Step Setup

**1. Grant operator permissions (one-time, if not root):**
```bash
sudo tailscale set --operator=$USER
```
Without this, `tailscale funnel` fails with "Access denied: serve config denied."

**2. Enable Funnel in Tailscale admin console (one-time):**
```bash
tailscale funnel 8080
# Prints a URL like: https://login.tailscale.com/f/funnel?node=xxxxx
# Open this URL in a browser, logged into the same Tailscale account
# Enable the Funnel toggle for this node
```
The command will time out — that's expected. It's waiting for admin approval.

**3. Reset any stale serve config:**
```bash
tailscale serve reset
```

**4. Start Serve (tailnet-internal HTTPS proxy):**
```bash
tailscale serve --bg 8080
# Output: Available within your tailnet: https://<host>.<tailnet>.ts.net/
```

**5. Start Funnel (public internet exposure):**
```bash
tailscale funnel --bg 8080
# Output: Available on the internet: https://<host>.<tailnet>.ts.net/
```

**6. Verify:**
```bash
tailscale serve status
# Should show:
#   # Funnel on:
#   https://<host>.<tailnet>.ts.net (Funnel on)
#   |-- / proxy http://127.0.0.1:8080
```

### Common Pitfalls

- **`tailscale funnel 8080` hangs/times out**: This is the foreground mode. Always use `--bg` for persistent operation. The foreground mode is only useful for the initial admin authorization step.
- **"Access denied: serve config denied"**: Run `sudo tailscale set --operator=$USER` first.
- **"Funnel is not enabled on your tailnet"**: Must enable Funnel in the admin console via the one-time authorization URL.
- **"foreground listener already exists for port 443"**: A previous serve/funnel instance is still running. Run `tailscale serve reset` to clear it.
- **Must run `serve` before `funnel`**: Funnel builds on top of Serve. Always `tailscale serve --bg <port>` first, then `tailscale funnel --bg <port>`.
- **Local curl test fails through proxy**: If `curl https://<host>.<tailnet>.ts.net/` fails with SSL errors, unset proxy env vars (`unset http_proxy https_proxy`). The Tailscale Funnel domain resolves through Tailscale's DNS, not through external proxies.
- **Funnel only works with HTTPS (port 443 via Tailscale's reverse proxy).** The local service can run on any port (e.g., 8080) — Tailscale handles TLS termination and provides a valid Let's Encrypt certificate automatically.