Last updated: 2026-05-27 This document tracks the public reverse proxy surface for `rproxy.cfts.co`. Use it before changing the Caddyfile so each hostname has an explicit exposure decision, upstream, auth layer, header profile, logging posture, and follow-up work. ## Current Edge Posture `rproxy.cfts.co` firewall posture observed on 2026-05-17 after HTTP/3 was disabled: ```text Default incoming: deny Default outgoing: allow 4422/tcp allowed from 172.16.198.0/24 80/tcp allowed from anywhere 443/tcp allowed from anywhere 161/udp allowed from 172.16.198.50 ``` Listening services observed: ```text Caddy: 80/tcp, 443/tcp SSH: 4422/tcp SNMP: 161/udp Caddy admin API: 127.0.0.1:2019 only ``` Fail2Ban posture verified on 2026-05-15, with Caddy jail source configs added on 2026-05-17: ```text fail2ban.service: active (running), enabled active jail: sshd sshd jail port: 4422 backend: systemd current bans: 0 planned Caddy jails: caddy-unknown-host, caddy-scanner-paths ``` This is acceptable for the reverse proxy role. The main remaining risk is which applications are exposed through Caddy on public `80/443`. ## Reusable Caddy Profiles The Caddyfile imports reusable snippets from: ```text /etc/caddy/includes/00-common.caddy ``` Source workspace copy: ```text F:\Avery_Vault_LLM\07-Projects\CFTS\Platforms\rproxy.cfts.co\caddy\includes\00-common.caddy ``` Current snippets: | Snippet | Intended use | | --- | --- | | `json_access_log` | Standard dedicated JSON access log with the shared retention settings. | | `safe_headers` | Strict headers for simple public sites that can tolerate CSP, Permissions-Policy, and frame denial; no unsafe inline script/style. | | `lose_headers_no_csp` | Safer minimum headers for apps that may break under strict CSP or frame denial. | | `strip_response_identity` | Removes common software-identifying response headers such as `Server`, `Via`, `X-Powered-By`, and framework/version breadcrumbs. | | `strip_spoofed_request_headers` | Removes client-supplied proxy/client-IP headers before upstream proxying; Caddy-generated proxy headers are still added by `reverse_proxy`. | | `strip_upstream_security_headers` | Removes upstream security headers inside selected proxy blocks when Caddy owns the public header policy. | | `no_uploads_expected` | Caps request bodies at `1MB` for sites where browser-based uploads are not expected; requires Caddy v2.10.0 or newer. | | `error_page_headers` | Security/cache headers for shared static Caddy error pages. | | `default_error_pages` | Serves `/etc/caddy/errors/403.html`, `/etc/caddy/errors/404.html`, and `/etc/caddy/errors/500.html` for Caddy-generated `403`, `404`, and `5xx` errors. | | `safe_tls` | TLS 1.2/1.3 settings, plus shared response identity and request spoofing header scrubs for all current sites. | | `vsphere_tls` | TLS 1.2/1.3 settings plus response identity scrub, but no request-header scrub; use only for fragile ESXi/vSphere consoles. | | `public_site_base` | Compression, `safe_tls`, and `safe_headers` for simple public sites. | Do not apply `public_site_base` blindly to app dashboards, ticket systems, consoles, or legacy apps without browser testing. ## Default Error Pages Every active HTTPS site block imports `default_error_pages`. Caddy-generated `403`, `404`, and `5xx` errors render shared HTML from `/etc/caddy/errors`, and LAN-only deny paths call `error 403` so they use that shared page. Normal upstream responses from `reverse_proxy` are still passed through unchanged. A backend-owned `404` or `500` will not be replaced by the shared Caddy page unless a deliberate per-site `handle_response` rule is added. POC result on 2026-05-26: an external browser request to `https://download.cfts.co/` rendered the shared `403 Access restricted` page. Keep the feature as a tidy Caddy-side fallback set; do not treat it as an application error-page override. Testing notes: - Test LAN-only denials from outside `172.16.198.0/24`; testing from `rproxy.cfts.co` or the LAN can correctly return upstream `200`. - `isp-status.cfts.co` WAN fallbacks can return `401` because Basic Auth owns that path. - Public app `404` responses, such as docs app misses, remain upstream-owned unless a site-specific `handle_response` rule is added. ## Explicit HTTP Routing / Unknown Host Catch-All The Caddyfile sets `auto_https disable_redirects`, then defines a single `http://` block for plaintext HTTP. Known hostnames in `@known_hosts` redirect to HTTPS; unknown hostnames are logged to `/var/log/caddy/unknown-host-access.log` and closed with `abort`. As of 2026-05-27, `@known_hosts` includes the current active hostnames, including `edge-01.cfts.co`, `edge-02.cfts.co`, `inventory.cfts.co`, and `rp-logs.cfts.co`. It also retains older `vps.cfts.co` / `hvps.cfts.co` names for compatibility/history. No broad `https://` catch-all is configured. Unknown HTTPS names should fail during TLS/SNI handling unless Caddy has a matching certificate. ## Global Server Guardrails The Caddyfile configures `:80` and `:443` server-level timeouts and protocol limits: | Listener | Guardrail | | --- | --- | | `:80` | `read_header 10s`, `idle 2m`, `max_header_size 64KB`, `protocols h1` | | `:443` | `read_header 10s`, `idle 2m`, `max_header_size 64KB`, `protocols h1 h2`, `strict_sni_host on`, `0rtt off` | HTTP/3 is intentionally disabled, and public `443/udp` exposure has been removed from UFW. ## Log-Based Banning And Patch Automation Source configs for easy-win automation live under: ```text security/fail2ban/filter.d/caddy-unknown-host.conf security/fail2ban/filter.d/caddy-scanner-paths.conf security/fail2ban/jail.d/caddy-rproxy.local security/apt/apt.conf.d/20auto-upgrades security/apt/apt.conf.d/52unattended-upgrades-cfts-security ``` Planned Fail2Ban web jails: | Jail | Signal | Threshold | | --- | --- | --- | | `caddy-unknown-host` | Unknown plaintext HTTP hostnames in `/var/log/caddy/unknown-host-access.log` | 20 hits in 10 minutes -> 1 hour ban | | `caddy-scanner-paths` | Common scanner paths across `/var/log/caddy/*-access.log` | 5 hits in 10 minutes -> 4 hour ban | The unattended-upgrades config enables Ubuntu security upgrades without automatic reboots. ## Planned PRTG Integration PRTG integration is the next monitoring milestone. Keep it low-noise and focused on edge health, security automation, and certificate/service drift. Suggested first sensors: | Sensor | Signal | | --- | --- | | HTTPS checks | `https://docs.cfts.co/`, `https://isp-status.cfts.co/view`, and selected public app login pages return expected status codes. | | Certificate expiry | Public certificates have healthy expiry windows. | | Caddy service | `caddy.service` is active and `caddy validate --config /etc/caddy/Caddyfile` passes. | | Fail2Ban jails | `sshd`, `caddy-unknown-host`, and `caddy-scanner-paths` are active; ban counts are visible. | | Firewall/listeners | Only expected public listeners are exposed: `80/tcp`, `443/tcp`; no `443/udp`. | | Disk/log growth | `/var/log/caddy` and root filesystem have safe free space. | | Patch cadence | Unattended-upgrades logs show recent successful security checks. | Avoid noisy per-request alerting at first. Alert on service down, cert expiry, disk pressure, missing Fail2Ban jails, unexpected listeners, or repeated validation failures. ## Exposure Matrix | Hostname | Public intent | Upstream | Current auth/gating | Header/TLS profile | Logging | Notes / next hardening | | --------------------- | ------------------------------------------ | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `docs.cfts.co` | Public documentation portal | `172.16.198.22:8090` | Public. App gates `/internal`, `/raw`, and `/api/v1` by trusted networks. | `public_site_base`; `no_uploads_expected` | Dedicated JSON log: `/var/log/caddy/docs-access.log` | Good first-pass public hardening. Confirm docs VM UFW allows `8090/tcp` only from `172.16.198.60`. | | `tickets.cfts.co` | Public ticket portal | `172.16.198.13:80` | Upstream app auth expected. | `lose_headers_no_csp` plus `safe_tls` | Dedicated JSON log: `/var/log/caddy/tickets-access.log` | Review upstream auth/session security periodically. | | `tickets.kee.go.ug` | Public/partner-facing ticket portal | `172.16.198.13:80` | Upstream app auth expected. | `safe_tls` plus custom CSP/frame/cache headers | No dedicated Caddy log yet | Has bespoke frame/CSP rules for KEE/CFTS embedding. Preserve carefully. | | `isp-status.cfts.co` | Public status surface with protected paths | `172.16.198.26:8080` | LAN full access; `/view` public guest view; WAN fallback requires Caddy `basic_auth`; selected internal paths LAN-only. | `public_site_base`; `no_uploads_expected` | Journald JSON log | Good pattern for mixed public/private app. Confirm Caddy auth hash owner and rotation process. | | `download.cfts.co` | Public DNS name, LAN-only use | `172.16.198.31:8080` | `client_ip 172.16.198.0/24`; others receive 403. | `lose_headers_no_csp` plus `safe_tls` | No dedicated Caddy log yet | Good LAN gate. Consider dedicated log if troubleshooting external probes matters. | | `console.dns.cfts.co` | Public DNS name, LAN-only use | `172.16.198.49:5380` | `client_ip 172.16.198.0/24`; others receive 403. | `lose_headers_no_csp` plus `safe_tls` | No dedicated Caddy log yet | Good LAN gate for DNS console. | | `ai.cfts.co` | Public DNS name, LAN-only use | `172.16.198.28:3000` | `client_ip 172.16.198.0/24`; others receive 403. | `lose_headers_no_csp` plus `safe_tls` | No dedicated Caddy log yet | Good LAN gate. Revisit before making public. | | `inventory.cfts.co` | Public DNS name, LAN-only inventory app | `172.16.198.67:80` | `client_ip 172.16.198.0/24`; others receive 403. | `public_site_base` | Dedicated JSON log: `/var/log/caddy/inventory.log` | Good LAN gate. Keep the origin firewall limited to rproxy where practical. | | `rp-logs.cfts.co` | LAN-only reverse-proxy log report GUI | Static files from `/var/www/caddy-log-report` | `client_ip 172.16.198.0/24`; others receive 403. Report is generated from Caddy logs by GoAccess every five minutes. | `lose_headers_no_csp` plus `safe_tls`; `no_uploads_expected` | Dedicated JSON log: `/var/log/caddy/rp-logs-access.log` | Gives Caddy logs a human-readable dashboard without exposing raw logs. Keep LAN-only; consider Basic Auth later if broader admin networks are added. | | `edge-01.cfts.co` | Public DNS name, LAN-only admin surface | `172.16.198.5:443` | `client_ip 172.16.198.0/24`; others receive 403. Upstream vSphere auth still expected. Caddy Basic Auth was tested on earlier `vps.cfts.co` naming and removed because it triggered vSphere web client errors. | Diagnostic baseline: no Caddy header/TLS protection snippets, no upstream `Host` override, strips `Authorization` only on `/sdk*` and `/screen*`, upstream TLS verification skipped | Dedicated JSON log: `/var/log/caddy/vps-access.log` | High-risk and fragile vSphere surface. Caddy now blocks non-LAN clients before proxying. Add trusted VPN/static admin CIDRs to `@admin_clients` if remote access is required, and keep other hardening one change at a time with browser testing. | | `edge-02.cfts.co` | Public DNS name, LAN-only admin surface | `172.16.198.6:443` | `client_ip 172.16.198.0/24` plus `45.195.74.128/32`; others receive 403. Upstream vSphere auth still expected. Caddy Basic Auth was tested on earlier `hvps.cfts.co` naming and removed because it triggered vSphere web client errors. | Diagnostic baseline: no Caddy header/TLS protection snippets, no upstream `Host` override, strips `Authorization` only on `/sdk*` and `/screen*`, upstream TLS verification skipped | Dedicated JSON log: `/var/log/caddy/hvps-access.log` | High-risk and fragile vSphere surface. Caddy now blocks non-allowed clients before proxying. Add trusted VPN/static admin CIDRs to `@admin_clients` if remote access is required, and keep other hardening one change at a time with browser testing. | | `monitor.cfts.co` | Public PRTG monitoring surface | `172.16.198.50:443` | Public by design for offsite monitoring checks; upstream PRTG auth expected. | `lose_headers_no_csp` plus `safe_tls`; `no_uploads_expected`; upstream TLS verification skipped | Dedicated JSON log: `/var/log/caddy/monitor-access.log` | Keep public unless there is a deliberate replacement remote-access pattern. Review upstream PRTG auth, accounts, and monitoring data exposure periodically. | | `redmine.cfts.co` | Public project/work tracking surface | `172.16.198.21:80` | Upstream app auth expected; public signup disabled. | `lose_headers_no_csp` plus `safe_tls` | Dedicated JSON log: `/var/log/caddy/redmine-access.log` | Review upstream auth and password policy periodically. | | `tracks.cfts.co` | Public app surface | `172.16.198.8:80` | Upstream app auth expected. | `lose_headers_no_csp` plus `safe_tls` | Dedicated JSON log: `/var/log/caddy/tracks-access.log` | Confirm upstream auth posture periodically. | ## Hardening Backlog In suggested order: 1. Confirm each origin VM firewall allows its application port only from `172.16.198.60` where practical. This is normal CFTS operating procedure; check periodically. 2. Review upstream authentication on `edge-01.cfts.co`, `edge-02.cfts.co`, public `monitor.cfts.co`/PRTG, `redmine.cfts.co`, `tracks.cfts.co`, and ticket portals. 3. Deploy and observe the Caddy Fail2Ban web jails; tune thresholds only after reviewing real matches. 4. Implement PRTG integration for edge health, certificates, Fail2Ban jail state, listener drift, disk/log growth, and unattended-upgrades status. 5. Consider adding or strengthening Caddy `basic_auth`, IP allowlists, or mixed LAN/public policies in front of other high-risk admin surfaces if upstream authentication is not strong enough. 6. Keep strict CSP limited to simple/public sites until each app is browser-tested. 7. Record validation results after each Caddy change: ```bash sudo caddy validate --config /etc/caddy/Caddyfile sudo systemctl reload caddy systemctl status caddy --no-pager ``` ## Deployment Notes Because SFTP does not write directly to `/etc/caddy`, stage files under: ```text /home/sysops/temp ``` Then copy with `sudo`: ```bash sudo mkdir -p /etc/caddy/includes /etc/caddy/errors sudo cp /home/sysops/temp/Caddyfile /etc/caddy/Caddyfile sudo cp /home/sysops/temp/includes/00-common.caddy /etc/caddy/includes/00-common.caddy sudo cp /home/sysops/temp/errors/*.html /etc/caddy/errors/ sudo chown root:root /etc/caddy/Caddyfile /etc/caddy/includes/00-common.caddy /etc/caddy/errors/*.html sudo chmod 644 /etc/caddy/Caddyfile /etc/caddy/includes/00-common.caddy /etc/caddy/errors/*.html ``` Dedicated log files must be writable by the `caddy` service user before reload. Example for docs: ```bash sudo install -d -o caddy -g caddy -m 0750 /var/log/caddy sudo touch /var/log/caddy/docs-access.log sudo chown caddy:caddy /var/log/caddy/docs-access.log sudo chmod 0640 /var/log/caddy/docs-access.log ``` If a reload reports `opening log writer` with `permission denied`, create the named file and set `caddy:caddy` ownership before reloading again. This was required for `/var/log/caddy/inventory.log` during the 2026-05-26 inventory deployment. The LAN-only log GUI is served at `https://rp-logs.cfts.co/` from `/var/www/caddy-log-report/index.html`. The report is generated by `observability/goaccess/render-caddy-goaccess-report.sh` through the `caddy-goaccess-report.timer` systemd timer.