mirror of
https://github.com/fosrl/pangolin.git
synced 2026-09-17 16:19:48 +02:00
Compare commits
1 Commits
1.23.0
..
03ab01ca11
| Author | SHA1 | Date | |
|---|---|---|---|
| 03ab01ca11 |
@@ -1,31 +0,0 @@
|
||||
---
|
||||
name: crud-endpoints
|
||||
description: Use whenever asked to add, create, or scaffold a CRUD endpoint, router, or entity in this repo's server (create/list/get/update/delete handlers, new `server/routers/<entity>/` or `server/private/routers/<entity>/` folder). Points to the established file layout, middleware, ActionsEnum, and route-registration conventions before writing any code.
|
||||
---
|
||||
|
||||
Before writing any router/handler/middleware code for a new entity, read
|
||||
`docs/crud-endpoints.md` in full. It documents, with real examples from
|
||||
`server/routers/aiProvider/` (public) and `server/private/routers/alertRule/`
|
||||
(enterprise-only), how this repo structures CRUD endpoints:
|
||||
|
||||
- Directory/file layout per entity (`index.ts`, `types.ts`, `validation.ts`,
|
||||
one file per operation).
|
||||
- The standard handler anatomy (zod parsing, OpenAPI registry, response
|
||||
envelope, error handling).
|
||||
- Where access-control middleware (`verify<Entity>Access`) lives and when
|
||||
it's needed vs. plain `verifyOrgAccess`.
|
||||
- How to wire up `ActionsEnum` entries, `verifyUserHasAction`, and
|
||||
`logActionAudit`.
|
||||
- Which of the four router files (`server/routers/external.ts`,
|
||||
`server/routers/internal.ts`, `server/private/routers/external.ts`,
|
||||
`server/private/routers/internal.ts`) to register routes in, and the
|
||||
middleware chain template per HTTP verb.
|
||||
- The repo's non-standard verb convention: **`PUT` = create, `POST` =
|
||||
update** (backwards from typical REST) — don't "fix" this to standard
|
||||
REST verbs, match the existing convention.
|
||||
- The `#dynamic` import alias, for the rare case of a hook needing different
|
||||
implementations in OSS vs. enterprise builds.
|
||||
|
||||
Follow that doc's checklist (§8) step by step rather than improvising a
|
||||
structure. If the doc and the actual code in `aiProvider`/`alertRule` ever
|
||||
disagree, trust the code and flag the doc as stale.
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
When adding submit buttons, don't change the text of the button during the loading state. Text should stay static and you should use the loading prop on the button.
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
When creating UI for popup dialogs or modals, use the Credenza componennt. This component is mobile responsive and works on desktop and wraps the dialog component and sheet into one.
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
Always localize strings and use the `t` function to convert keys to strings. Add the keys to the en-us.json file. Never edit the other language files, as en-us.json is the single source of truth.
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
Don't write or edit migrations in `server/setup` unless specificall instructed to do so.
|
||||
@@ -1,7 +0,0 @@
|
||||
---
|
||||
description:
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
Proxy resources = public resources
|
||||
Private resources = client resources = site resources
|
||||
@@ -1,7 +0,0 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
When writing TypeScript:
|
||||
|
||||
Prefer to use types instead of interfaces.
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
alwaysApply: true
|
||||
---
|
||||
|
||||
When creating forms, use React form for validation and use Zod schemas.
|
||||
@@ -34,5 +34,3 @@ build.ts
|
||||
tsconfig.json
|
||||
Dockerfile*
|
||||
drizzle.config.ts
|
||||
allowedDevOrigins.json
|
||||
scratch/
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
# These are supported funding model platforms
|
||||
|
||||
github: [fosrl]
|
||||
@@ -14,13 +14,12 @@ body:
|
||||
label: Environment
|
||||
description: Please fill out the relevant details below for your environment.
|
||||
value: |
|
||||
- OS Type & Version:
|
||||
- OS Type & Version: (e.g., Ubuntu 22.04)
|
||||
- Pangolin Version:
|
||||
- Edition (Community or Enterprise):
|
||||
- Gerbil Version:
|
||||
- Traefik Version:
|
||||
- Newt Version:
|
||||
- Client Version:
|
||||
- Olm Version: (if applicable)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -34,14 +33,6 @@ body:
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: AI Disclosure
|
||||
description: |
|
||||
If you used AI to help write this issue, please disclose it here. This is important for transparency and helps maintain the integrity of the issue tracking process.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Expected Behavior
|
||||
|
||||
@@ -4,10 +4,6 @@ perpetual license to use, modify, and redistribute these contributions under any
|
||||
choose, including both the AGPLv3 and the Fossorial Commercial license terms. I
|
||||
represent that I have the right to grant this license for all contributed content.
|
||||
|
||||
## AI Disclosure
|
||||
|
||||
> Please disclose how AI was used in this pull request. The use of AI does not preclude this from being merged but is an important factor in how we review your request.
|
||||
|
||||
## Description
|
||||
|
||||
|
||||
|
||||
+28
-18
@@ -1,42 +1,52 @@
|
||||
version: 2
|
||||
|
||||
updates:
|
||||
- package-ecosystem: "npm"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "daily"
|
||||
open-pull-requests-limit: 1
|
||||
groups:
|
||||
npm-dependencies:
|
||||
patterns:
|
||||
- "*"
|
||||
dev-patch-updates:
|
||||
dependency-type: "development"
|
||||
update-types:
|
||||
- "patch"
|
||||
dev-minor-updates:
|
||||
dependency-type: "development"
|
||||
update-types:
|
||||
- "minor"
|
||||
prod-patch-updates:
|
||||
dependency-type: "production"
|
||||
update-types:
|
||||
- "patch"
|
||||
prod-minor-updates:
|
||||
dependency-type: "production"
|
||||
update-types:
|
||||
- "minor"
|
||||
|
||||
- package-ecosystem: "docker"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "daily"
|
||||
open-pull-requests-limit: 1
|
||||
groups:
|
||||
docker-dependencies:
|
||||
patterns:
|
||||
- "*"
|
||||
patch-updates:
|
||||
update-types:
|
||||
- "patch"
|
||||
minor-updates:
|
||||
update-types:
|
||||
- "minor"
|
||||
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
open-pull-requests-limit: 1
|
||||
groups:
|
||||
github-actions-dependencies:
|
||||
patterns:
|
||||
- "*"
|
||||
|
||||
- package-ecosystem: "gomod"
|
||||
directory: "/install"
|
||||
schedule:
|
||||
interval: "daily"
|
||||
open-pull-requests-limit: 1
|
||||
groups:
|
||||
go-install-dependencies:
|
||||
patterns:
|
||||
- "*"
|
||||
patch-updates:
|
||||
update-types:
|
||||
- "patch"
|
||||
minor-updates:
|
||||
update-types:
|
||||
- "minor"
|
||||
|
||||
+96
-40
@@ -62,7 +62,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Monitor storage space
|
||||
run: |
|
||||
@@ -77,7 +77,7 @@ jobs:
|
||||
fi
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
|
||||
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
|
||||
with:
|
||||
registry: docker.io
|
||||
username: ${{ secrets.DOCKER_HUB_USERNAME }}
|
||||
@@ -134,7 +134,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Monitor storage space
|
||||
run: |
|
||||
@@ -149,7 +149,7 @@ jobs:
|
||||
fi
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
|
||||
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
|
||||
with:
|
||||
registry: docker.io
|
||||
username: ${{ secrets.DOCKER_HUB_USERNAME }}
|
||||
@@ -201,10 +201,10 @@ jobs:
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Log in to Docker Hub
|
||||
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
|
||||
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
|
||||
with:
|
||||
registry: docker.io
|
||||
username: ${{ secrets.DOCKER_HUB_USERNAME }}
|
||||
@@ -256,7 +256,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Extract tag name
|
||||
id: get-tag
|
||||
@@ -264,9 +264,9 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
- name: Install Go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
|
||||
with:
|
||||
go-version: 1.26
|
||||
go-version: 1.25
|
||||
|
||||
- name: Update version in package.json
|
||||
run: |
|
||||
@@ -407,27 +407,35 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
- name: Login to GitHub Container Registry (for cosign)
|
||||
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
|
||||
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
|
||||
with:
|
||||
registry: ghcr.io
|
||||
username: ${{ github.actor }}
|
||||
password: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Install cosign
|
||||
# cosign is used to sign container images using keyless (OIDC) signing
|
||||
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
|
||||
with:
|
||||
cosign-release: v3.0.6
|
||||
# cosign is used to sign and verify container images (key and keyless)
|
||||
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
|
||||
|
||||
- name: Sign (GHCR, keyless)
|
||||
# Sign each GHCR image by digest using keyless (OIDC) signing via Sigstore/Rekor.
|
||||
# Signatures are stored in the registry alongside the image.
|
||||
- name: Dual-sign and verify (GHCR & Docker Hub)
|
||||
# Sign each image by digest using keyless (OIDC) and key-based signing,
|
||||
# then verify both the public key signature and the keyless OIDC signature.
|
||||
env:
|
||||
TAG: ${{ env.TAG }}
|
||||
COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
|
||||
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
|
||||
COSIGN_PUBLIC_KEY: ${{ secrets.COSIGN_PUBLIC_KEY }}
|
||||
COSIGN_YES: "true"
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
issuer="https://token.actions.githubusercontent.com"
|
||||
id_regex="^https://github.com/${{ github.repository }}/.+" # accept this repo (all workflows/refs)
|
||||
|
||||
# Track failures
|
||||
FAILED_TAGS=()
|
||||
SUCCESSFUL_TAGS=()
|
||||
|
||||
# Determine if this is an RC release
|
||||
IS_RC="false"
|
||||
if [[ "$TAG" == *"-rc."* ]]; then
|
||||
@@ -455,47 +463,95 @@ jobs:
|
||||
)
|
||||
fi
|
||||
|
||||
FAILED_TAGS=()
|
||||
SUCCESSFUL_TAGS=()
|
||||
# Sign each image variant for both registries
|
||||
for BASE_IMAGE in "${GHCR_IMAGE}" "${DOCKERHUB_IMAGE}"; do
|
||||
for IMAGE_TAG in "${IMAGE_TAGS[@]}"; do
|
||||
echo "Processing ${BASE_IMAGE}:${IMAGE_TAG}"
|
||||
TAG_FAILED=false
|
||||
|
||||
for IMAGE_TAG in "${IMAGE_TAGS[@]}"; do
|
||||
echo "Processing ${GHCR_IMAGE}:${IMAGE_TAG}"
|
||||
TAG_FAILED=false
|
||||
# Wrap the entire tag processing in error handling
|
||||
(
|
||||
set -e
|
||||
DIGEST="$(skopeo inspect --retry-times 3 docker://${BASE_IMAGE}:${IMAGE_TAG} | jq -r '.Digest')"
|
||||
REF="${BASE_IMAGE}@${DIGEST}"
|
||||
echo "Resolved digest: ${REF}"
|
||||
|
||||
(
|
||||
set -e
|
||||
DIGEST="$(skopeo inspect --retry-times 3 docker://${GHCR_IMAGE}:${IMAGE_TAG} | jq -r '.Digest')"
|
||||
REF="${GHCR_IMAGE}@${DIGEST}"
|
||||
echo "Resolved digest: ${REF}"
|
||||
echo "==> cosign sign (keyless) --recursive ${REF}"
|
||||
cosign sign --recursive "${REF}"
|
||||
|
||||
echo "==> cosign sign (keyless) --recursive ${REF}"
|
||||
cosign sign --recursive "${REF}"
|
||||
) || TAG_FAILED=true
|
||||
echo "==> cosign sign (key) --recursive ${REF}"
|
||||
cosign sign --key env://COSIGN_PRIVATE_KEY --recursive "${REF}"
|
||||
|
||||
if [ "$TAG_FAILED" = "true" ]; then
|
||||
echo "⚠️ WARNING: Failed to sign ${GHCR_IMAGE}:${IMAGE_TAG}"
|
||||
FAILED_TAGS+=("${GHCR_IMAGE}:${IMAGE_TAG}")
|
||||
else
|
||||
echo "✓ Successfully signed ${GHCR_IMAGE}:${IMAGE_TAG}"
|
||||
SUCCESSFUL_TAGS+=("${GHCR_IMAGE}:${IMAGE_TAG}")
|
||||
fi
|
||||
# Retry wrapper for verification to handle registry propagation delays
|
||||
retry_verify() {
|
||||
local cmd="$1"
|
||||
local attempts=6
|
||||
local delay=5
|
||||
local i=1
|
||||
until eval "$cmd"; do
|
||||
if [ $i -ge $attempts ]; then
|
||||
echo "Verification failed after $attempts attempts"
|
||||
return 1
|
||||
fi
|
||||
echo "Verification not yet available. Retry $i/$attempts after ${delay}s..."
|
||||
sleep $delay
|
||||
i=$((i+1))
|
||||
delay=$((delay*2))
|
||||
# Cap the delay to avoid very long waits
|
||||
if [ $delay -gt 60 ]; then delay=60; fi
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
echo "==> cosign verify (public key) ${REF}"
|
||||
if retry_verify "cosign verify --key env://COSIGN_PUBLIC_KEY '${REF}' -o text"; then
|
||||
VERIFIED_INDEX=true
|
||||
else
|
||||
VERIFIED_INDEX=false
|
||||
fi
|
||||
|
||||
echo "==> cosign verify (keyless policy) ${REF}"
|
||||
if retry_verify "cosign verify --certificate-oidc-issuer '${issuer}' --certificate-identity-regexp '${id_regex}' '${REF}' -o text"; then
|
||||
VERIFIED_INDEX_KEYLESS=true
|
||||
else
|
||||
VERIFIED_INDEX_KEYLESS=false
|
||||
fi
|
||||
|
||||
# Check if verification succeeded
|
||||
if [ "${VERIFIED_INDEX}" != "true" ] && [ "${VERIFIED_INDEX_KEYLESS}" != "true" ]; then
|
||||
echo "⚠️ WARNING: Verification not available for ${BASE_IMAGE}:${IMAGE_TAG}"
|
||||
echo "This may be due to registry propagation delays. Continuing anyway."
|
||||
fi
|
||||
) || TAG_FAILED=true
|
||||
|
||||
if [ "$TAG_FAILED" = "true" ]; then
|
||||
echo "⚠️ WARNING: Failed to sign/verify ${BASE_IMAGE}:${IMAGE_TAG}"
|
||||
FAILED_TAGS+=("${BASE_IMAGE}:${IMAGE_TAG}")
|
||||
else
|
||||
echo "✓ Successfully signed and verified ${BASE_IMAGE}:${IMAGE_TAG}"
|
||||
SUCCESSFUL_TAGS+=("${BASE_IMAGE}:${IMAGE_TAG}")
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
# Report summary
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo "Sign Summary"
|
||||
echo "Sign and Verify Summary"
|
||||
echo "=========================================="
|
||||
echo "Successful: ${#SUCCESSFUL_TAGS[@]}"
|
||||
echo "Failed: ${#FAILED_TAGS[@]}"
|
||||
echo ""
|
||||
|
||||
if [ ${#FAILED_TAGS[@]} -gt 0 ]; then
|
||||
echo "Failed tags:"
|
||||
for tag in "${FAILED_TAGS[@]}"; do
|
||||
echo " - $tag"
|
||||
done
|
||||
echo "⚠️ WARNING: Some tags failed to sign, but continuing anyway"
|
||||
echo ""
|
||||
echo "⚠️ WARNING: Some tags failed to sign/verify, but continuing anyway"
|
||||
else
|
||||
echo "✓ All images signed successfully!"
|
||||
echo "✓ All images signed and verified successfully!"
|
||||
fi
|
||||
shell: bash
|
||||
|
||||
|
||||
@@ -21,15 +21,12 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
# Match the version the app ships on (Dockerfile /
|
||||
# Dockerfile.dev both use 24.18.1) rather than floating
|
||||
# to the latest 24.x.
|
||||
node-version: '24.18.1'
|
||||
node-version: '24'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
@@ -23,7 +23,7 @@ jobs:
|
||||
skopeo --version
|
||||
|
||||
- name: Install cosign
|
||||
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
|
||||
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
|
||||
|
||||
- name: Input check
|
||||
run: |
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
name: Restart Runners
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 0 */7 * *'
|
||||
|
||||
permissions:
|
||||
id-token: write
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
ec2-maintenance-prod:
|
||||
runs-on: ubuntu-latest
|
||||
permissions: write-all
|
||||
steps:
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v6
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/${{ secrets.AWS_ROLE_NAME }}
|
||||
role-duration-seconds: 3600
|
||||
aws-region: ${{ secrets.AWS_REGION }}
|
||||
|
||||
- name: Verify AWS identity
|
||||
run: aws sts get-caller-identity
|
||||
|
||||
- name: Start EC2 instance
|
||||
run: |
|
||||
aws ec2 start-instances --instance-ids ${{ secrets.EC2_INSTANCE_ID_ARM_RUNNER }}
|
||||
aws ec2 start-instances --instance-ids ${{ secrets.EC2_INSTANCE_ID_AMD_RUNNER }}
|
||||
echo "EC2 instances started"
|
||||
|
||||
- name: Wait
|
||||
run: sleep 600
|
||||
|
||||
- name: Stop EC2 instance
|
||||
run: |
|
||||
aws ec2 stop-instances --instance-ids ${{ secrets.EC2_INSTANCE_ID_ARM_RUNNER }}
|
||||
aws ec2 stop-instances --instance-ids ${{ secrets.EC2_INSTANCE_ID_AMD_RUNNER }}
|
||||
echo "EC2 instances stopped"
|
||||
@@ -0,0 +1,160 @@
|
||||
name: SAAS Pipeline
|
||||
|
||||
# CI/CD workflow for building, publishing, mirroring, signing container images and building release binaries.
|
||||
# Actions are pinned to specific SHAs to reduce supply-chain risk. This workflow triggers on tag push events.
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write # for GHCR push
|
||||
id-token: write # for Cosign Keyless (OIDC) Signing
|
||||
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "[0-9]+.[0-9]+.[0-9]+-s.[0-9]+"
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
pre-run:
|
||||
runs-on: ubuntu-latest
|
||||
permissions: write-all
|
||||
steps:
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v6
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/${{ secrets.AWS_ROLE_NAME }}
|
||||
role-duration-seconds: 3600
|
||||
aws-region: ${{ secrets.AWS_REGION }}
|
||||
|
||||
- name: Verify AWS identity
|
||||
run: aws sts get-caller-identity
|
||||
|
||||
- name: Start EC2 instances
|
||||
run: |
|
||||
aws ec2 start-instances --instance-ids ${{ secrets.EC2_INSTANCE_ID_ARM_RUNNER }}
|
||||
echo "EC2 instances started"
|
||||
|
||||
|
||||
release-arm:
|
||||
name: Build and Release (ARM64)
|
||||
runs-on: [self-hosted, linux, arm64, us-east-1]
|
||||
needs: [pre-run]
|
||||
if: >-
|
||||
${{
|
||||
needs.pre-run.result == 'success'
|
||||
}}
|
||||
# Job-level timeout to avoid runaway or stuck runs
|
||||
timeout-minutes: 120
|
||||
env:
|
||||
# Target images
|
||||
AWS_IMAGE: ${{ secrets.aws_account_id }}.dkr.ecr.us-east-1.amazonaws.com/${{ github.event.repository.name }}
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Download MaxMind GeoLite2 databases
|
||||
env:
|
||||
MAXMIND_LICENSE_KEY: ${{ secrets.MAXMIND_LICENSE_KEY }}
|
||||
run: |
|
||||
echo "Downloading MaxMind GeoLite2 databases..."
|
||||
|
||||
# Download GeoLite2-Country
|
||||
curl -L "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-Country&license_key=${MAXMIND_LICENSE_KEY}&suffix=tar.gz" \
|
||||
-o GeoLite2-Country.tar.gz
|
||||
|
||||
# Download GeoLite2-ASN
|
||||
curl -L "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-ASN&license_key=${MAXMIND_LICENSE_KEY}&suffix=tar.gz" \
|
||||
-o GeoLite2-ASN.tar.gz
|
||||
|
||||
# Extract the .mmdb files
|
||||
tar -xzf GeoLite2-Country.tar.gz --strip-components=1 --wildcards '*.mmdb'
|
||||
tar -xzf GeoLite2-ASN.tar.gz --strip-components=1 --wildcards '*.mmdb'
|
||||
|
||||
# Verify files exist
|
||||
if [ ! -f "GeoLite2-Country.mmdb" ]; then
|
||||
echo "ERROR: Failed to download GeoLite2-Country.mmdb"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "GeoLite2-ASN.mmdb" ]; then
|
||||
echo "ERROR: Failed to download GeoLite2-ASN.mmdb"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Clean up tar files
|
||||
rm -f GeoLite2-Country.tar.gz GeoLite2-ASN.tar.gz
|
||||
|
||||
echo "MaxMind databases downloaded successfully"
|
||||
ls -lh GeoLite2-*.mmdb
|
||||
|
||||
- name: Monitor storage space
|
||||
run: |
|
||||
THRESHOLD=75
|
||||
USED_SPACE=$(df / | grep / | awk '{ print $5 }' | sed 's/%//g')
|
||||
echo "Used space: $USED_SPACE%"
|
||||
if [ "$USED_SPACE" -ge "$THRESHOLD" ]; then
|
||||
echo "Used space is below the threshold of 75% free. Running Docker system prune."
|
||||
echo y | docker system prune -a
|
||||
else
|
||||
echo "Storage space is above the threshold. No action needed."
|
||||
fi
|
||||
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v6
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ secrets.aws_account_id }}:role/${{ secrets.AWS_ROLE_NAME }}
|
||||
role-duration-seconds: 3600
|
||||
aws-region: ${{ secrets.AWS_REGION }}
|
||||
|
||||
- name: Login to Amazon ECR
|
||||
id: login-ecr
|
||||
uses: aws-actions/amazon-ecr-login@v2
|
||||
|
||||
- name: Extract tag name
|
||||
id: get-tag
|
||||
run: echo "TAG=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV
|
||||
shell: bash
|
||||
|
||||
- name: Update version in package.json
|
||||
run: |
|
||||
TAG=${{ env.TAG }}
|
||||
sed -i "s/export const APP_VERSION = \".*\";/export const APP_VERSION = \"$TAG\";/" server/lib/consts.ts
|
||||
cat server/lib/consts.ts
|
||||
shell: bash
|
||||
|
||||
- name: Build and push Docker images (Docker Hub - ARM64)
|
||||
run: |
|
||||
TAG=${{ env.TAG }}
|
||||
make build-saas tag=$TAG
|
||||
echo "Built & pushed ARM64 images to: ${{ env.AWS_IMAGE }}:${TAG}"
|
||||
shell: bash
|
||||
|
||||
post-run:
|
||||
needs: [pre-run, release-arm]
|
||||
if: >-
|
||||
${{
|
||||
always() &&
|
||||
needs.pre-run.result == 'success' &&
|
||||
(needs.release-arm.result == 'success' || needs.release-arm.result == 'skipped' || needs.release-arm.result == 'failure')
|
||||
}}
|
||||
runs-on: ubuntu-latest
|
||||
permissions: write-all
|
||||
steps:
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@v6
|
||||
with:
|
||||
role-to-assume: arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/${{ secrets.AWS_ROLE_NAME }}
|
||||
role-duration-seconds: 3600
|
||||
aws-region: ${{ secrets.AWS_REGION }}
|
||||
|
||||
- name: Verify AWS identity
|
||||
run: aws sts get-caller-identity
|
||||
|
||||
- name: Stop EC2 instances
|
||||
run: |
|
||||
aws ec2 stop-instances --instance-ids ${{ secrets.EC2_INSTANCE_ID_ARM_RUNNER }}
|
||||
echo "EC2 instances stopped"
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
stale:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
|
||||
- uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0
|
||||
with:
|
||||
days-before-stale: 14
|
||||
days-before-close: 14
|
||||
|
||||
@@ -14,18 +14,12 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Install Node
|
||||
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
||||
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
|
||||
with:
|
||||
# Pin to the version the app actually ships on (Dockerfile /
|
||||
# Dockerfile.dev both use 24.18.1; .nvmrc pins the 24 major).
|
||||
# A bare '24' floats to the latest 24.x; Node >= 24.19.0 added
|
||||
# node::ObjectWrap cleanup hooks that crash better-sqlite3 on teardown
|
||||
# ("Assertion failed: (env) != nullptr" in RemoveEnvironmentCleanupHook),
|
||||
# which intermittently kills `npm run dev` in the step below.
|
||||
node-version: '24.18.1'
|
||||
node-version: '24'
|
||||
|
||||
- name: Copy config file
|
||||
run: cp config/config.example.yml config/config.yml
|
||||
@@ -49,41 +43,26 @@ jobs:
|
||||
run: npx tsc --noEmit
|
||||
|
||||
- name: Start app in background
|
||||
run: |
|
||||
nohup npm run dev > app.log 2>&1 &
|
||||
echo $! > app.pid
|
||||
run: nohup npm run dev &
|
||||
|
||||
- name: Wait for app availability
|
||||
run: |
|
||||
print_log() {
|
||||
if [ -f app.log ]; then cat app.log; else echo "(app.log not found)"; fi
|
||||
}
|
||||
for i in {1..5}; do
|
||||
if curl --silent --fail http://localhost:3002/auth/login; then
|
||||
echo "App is up"
|
||||
exit 0
|
||||
fi
|
||||
# Fail fast (and show why) if the process is gone. Read the PID
|
||||
# safely: if it's missing/empty (e.g. nohup never started), treat
|
||||
# that as "not running" instead of passing a bad arg to kill.
|
||||
pid="$(cat app.pid 2>/dev/null || true)"
|
||||
if [ -z "$pid" ] || ! kill -0 "$pid" 2>/dev/null; then
|
||||
echo "App process exited before becoming available. Output:"
|
||||
print_log
|
||||
exit 1
|
||||
fi
|
||||
echo "Waiting for the app... attempt $i"
|
||||
sleep 5
|
||||
done
|
||||
echo "App failed to start. Output:"
|
||||
print_log
|
||||
echo "App failed to start"
|
||||
exit 1
|
||||
|
||||
build-sqlite:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Build Docker image sqlite
|
||||
run: make dev-build-sqlite
|
||||
@@ -92,7 +71,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Build Docker image pg
|
||||
run: make dev-build-pg
|
||||
|
||||
+3
-4
@@ -17,9 +17,9 @@ yarn-error.log*
|
||||
*.tsbuildinfo
|
||||
next-env.d.ts
|
||||
*.db
|
||||
*.sqlite*
|
||||
*.sqlite
|
||||
!Dockerfile.sqlite
|
||||
*.sqlite3*
|
||||
*.sqlite3
|
||||
*.log
|
||||
.machinelogs*.json
|
||||
*-audit.json
|
||||
@@ -46,6 +46,7 @@ public/branding
|
||||
server/db/index.ts
|
||||
server/build.ts
|
||||
postgres/
|
||||
dynamic/
|
||||
*.mmdb
|
||||
scratch/
|
||||
tsconfig.json
|
||||
@@ -53,5 +54,3 @@ hydrateSaas.ts
|
||||
CLAUDE.md
|
||||
drizzle.config.ts
|
||||
server/setup/migrations.ts
|
||||
solo.yml
|
||||
allowedDevOrigins.json
|
||||
@@ -1,4 +0,0 @@
|
||||
# typescript-eslint@8.70.0 declares a peer range of typescript "<6.1.0" and
|
||||
# hasn't caught up to typescript@7.x yet, even though it works fine against it
|
||||
# in practice. Without this, `npm install`/`npm ci` fail with ERESOLVE.
|
||||
legacy-peer-deps=true
|
||||
Vendored
+1
-4
@@ -18,8 +18,5 @@
|
||||
"[json]": {
|
||||
"editor.defaultFormatter": "esbenp.prettier-vscode"
|
||||
},
|
||||
"editor.formatOnSave": true,
|
||||
"cSpell.words": [
|
||||
"nessicary"
|
||||
]
|
||||
"editor.formatOnSave": true
|
||||
}
|
||||
+5
-5
@@ -1,11 +1,11 @@
|
||||
# FROM node:24.18.1-slim AS base
|
||||
FROM public.ecr.aws/docker/library/node:24.18.1-slim AS base
|
||||
# FROM node:24-slim AS base
|
||||
FROM public.ecr.aws/docker/library/node:24-slim AS base
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apt-get update && apt-get install -y python3 make g++ && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY package*.json .npmrc ./
|
||||
COPY package*.json ./
|
||||
|
||||
FROM base AS builder-dev
|
||||
|
||||
@@ -32,8 +32,8 @@ FROM base AS builder
|
||||
|
||||
RUN npm ci --omit=dev
|
||||
|
||||
# FROM node:24.18.1-slim AS runner
|
||||
FROM public.ecr.aws/docker/library/node:24.18.1-slim AS runner
|
||||
# FROM node:24-slim AS runner
|
||||
FROM public.ecr.aws/docker/library/node:24-slim AS runner
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
+2
-2
@@ -1,10 +1,10 @@
|
||||
FROM node:24.18.1-alpine
|
||||
FROM node:24-alpine
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apk add --no-cache python3 make g++
|
||||
|
||||
COPY package*.json .npmrc ./
|
||||
COPY package*.json ./
|
||||
|
||||
# Install dependencies
|
||||
RUN npm ci
|
||||
|
||||
@@ -37,22 +37,11 @@
|
||||
|
||||
<p align="center">
|
||||
<strong>
|
||||
Get started with Pangolin Cloud at <a href="https://app.pangolin.net/auth/signup">app.pangolin.net</a>
|
||||
Get started with Pangolin at <a href="https://app.pangolin.net/auth/signup">app.pangolin.net</a>
|
||||
</strong>
|
||||
</p>
|
||||
|
||||
Pangolin is an open-source SASE platform, built on WireGuard®, with a simple mission: connect and protect your users, wherever they are. It brings networking and security together as one system including a zero-trust VPN, zero-trust reverse proxy, privileged access management, and an identity-aware AI gateway, all sharing one identity and policy model. It's the same idea behind platforms like Cloudflare One, Zscaler, and Prisma but open, self-hostable, and built to stay light and easy to deploy.
|
||||
|
||||
### Networking and security that's unified, open, and simple
|
||||
|
||||
Legacy SASE platforms got the idea right: connectivity and security belong together. But they delivered it as a heavyweight, closed, cloud-locked stack assembled from years of patchwork. Pangolin exists to do that unification differently, in the open, self-hostable, and simple enough that administrators actually enjoy running it.
|
||||
|
||||
* **Open source, not a black box**: the code is open and auditable, so you can see exactly how your traffic is handled and how access decisions get made, instead of trusting a closed cloud control plane.
|
||||
* **Networking and security as one platform**: sites, reverse proxy, client access, RBAC, and the AI gateway share one identity and policy model, so protecting users and connecting them are executed together.
|
||||
* **Lightweight by design**: the whole platform is built to stay small and fast: easy to self-host on a small server, with a lightweight, user-space connector that goes in your private networks.
|
||||
* **Enjoyable to use**: a clean, modern interface and a setup flow that gets out of your way, so managing access feels simple instead of like fighting a legacy admin console.
|
||||
* **Zero trust from day one**: access is granted per resource, not per network, with identity provider integration, role-based access control, and full audit logging.
|
||||
* **Run it your way**: self-host the Community Edition for free, step up to the Enterprise Edition for advanced features, or use Pangolin Cloud if you'd rather not manage infrastructure at all.
|
||||
Pangolin is an open-source, identity-based remote access platform built on WireGuard that enables secure, seamless connectivity to private and public resources. Pangolin combines reverse proxy and VPN capabilities into one platform, providing browser-based access to web applications and client-based access to any private resources with NAT traversal, all with granular access controls.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -64,9 +53,9 @@ Legacy SASE platforms got the idea right: connectivity and security belong toget
|
||||
|
||||
## Deployment Options
|
||||
|
||||
- **Pangolin Cloud** - Fully managed service with no infrastructure required.
|
||||
- **Self-Host: Community Edition** - Free, open-source, and licensed under AGPL-3.
|
||||
- **Self-Host: Enterprise Edition** - Open-core, and licensed under Fossorial Commercial License. Free for personal and hobbyist use, and for businesses making less than \$100K USD gross annual revenue.
|
||||
- **Pangolin Cloud** — Fully managed service - no infrastructure required.
|
||||
- **Self-Host: Community Edition** — Free, open source, and licensed under AGPL-3.
|
||||
- **Self-Host: Enterprise Edition** — Licensed under Fossorial Commercial License. Free for personal and hobbyist use, and for businesses making less than \$100K USD gross annual revenue.
|
||||
|
||||
## Key Features
|
||||
|
||||
@@ -74,26 +63,11 @@ Legacy SASE platforms got the idea right: connectivity and security belong toget
|
||||
|
||||
Pangolin's site connectors provide gateways into networks so you can access any networked resources. Sites use outbound tunnels and intelligent NAT traversal to make networks behind restrictive firewalls available for authorized access without public IPs or open ports. Easily deploy a site as a binary or container on any platform.
|
||||
|
||||
* Lightweight user-space connector runs anywhere
|
||||
* Punches through any firewall
|
||||
* Doesn't require open ports or a public IP
|
||||
* Strict network segmentation
|
||||
* WireGuard-based
|
||||
* Get alerts when a device or network resource goes down
|
||||
|
||||
<img src="public/screenshots/sites.png" alt="Sites" width="100%" />
|
||||
|
||||
### Browser-based reverse proxy access
|
||||
|
||||
Expose HTTPS web applications and connect to VNC, RDP, and SSH entirely in the browser through identity and context-aware tunneled reverse proxies. Users access resources with authentication and granular access control without installing a client. Pangolin handles routing, load balancing, health checking, and automatic SSL certificates without exposing your network directly to the internet.
|
||||
|
||||
* Expose a web panel anywhere
|
||||
* Access via any web browser
|
||||
* Single sign-on across all resources
|
||||
* HTTPS resources
|
||||
* Remote desktop in the browser with VNC and RDP
|
||||
* In-browser SSH terminal with privileged access management (PAM)
|
||||
* PIN codes, passcodes, email OTP, geoblocking, allow-lists, and more
|
||||
Expose web applications through identity and context-aware tunneled reverse proxies. Users access applications through any web browser with authentication and granular access control without installing a client. Pangolin handles routing, load balancing, health checking, and automatic SSL certificates without exposing your network directly to the internet.
|
||||
|
||||
<img src="public/clip.gif" alt="Reverse proxy access" width="100%" />
|
||||
|
||||
@@ -101,48 +75,14 @@ Expose HTTPS web applications and connect to VNC, RDP, and SSH entirely in the b
|
||||
|
||||
Access private resources like SSH servers, databases, RDP, and entire network ranges through Pangolin clients. Intelligent NAT traversal enables connections even through restrictive firewalls, while DNS aliases provide friendly names and fast connections to resources across all your sites. Add redundancy by routing traffic through multiple connectors in your network.
|
||||
|
||||
* Peer-to-peer with intelligent NAT traversal
|
||||
* Hosts/IPs and port ranges
|
||||
* Network ranges/CIDRs
|
||||
* Friendly DNS aliases for network addresses
|
||||
* Privileged access management (PAM) with SSH resources
|
||||
* Private HTTPS resources only accessible on the private network
|
||||
|
||||
<img src="public/screenshots/private-resources.png" alt="Private resources" width="100%" />
|
||||
|
||||
### Identity-aware AI gateway
|
||||
|
||||
Put an identity-aware proxy in front of public cloud (OpenAI, Anthropic, Gemini, etc.) and self-hosted model servers (Ollama, vLLM, Mistral, etc.) so coding agents and AI clients call a single Pangolin URL. Publish it as a public resource with personal API keys, or keep it private on a client tunnel where the connected client is the credential for keyless access. Budgets, session history, and usage analytics sit in front of every call.
|
||||
|
||||
* Access self-hosted models (vLLM, Ollama, etc) alongside cloud models (OpenAI, Anthropic, etc) in one place
|
||||
* Keyless access by authenticating users with the Pangolin desktop client
|
||||
* Or, provide users with personal API keys
|
||||
* Control costs and token usage by setting budgets
|
||||
* Audit with detailed session history and analytics
|
||||
* Integrate AI clients and coding agents (Claude Code, Codex, OpenCode, etc)
|
||||
|
||||
<img src="public/screenshots/expanded-session-logs.png" alt="AI Session Logs" width="100%" />
|
||||
|
||||
### Give users and roles access to resources
|
||||
|
||||
Use Pangolin's built-in users or bring your own identity provider and set up role-based access control (RBAC). Grant users access to specific resources, not entire networks. Unlike traditional VPNs that expose full network access, Pangolin's zero-trust model ensures users can only reach the applications, services, and routes you explicitly define.
|
||||
|
||||
* Bring your existing identity provider (IdP) or use Pangolin identities
|
||||
* Sync users and roles from your IdP
|
||||
* User- and role-based access control
|
||||
* Full network audit and access logs
|
||||
Use Pangolin's built in users or bring your own identity provider and set up role based access control (RBAC). Grant users access to specific resources, not entire networks. Unlike traditional VPNs that expose full network access, Pangolin's zero-trust model ensures users can only reach the applications, services, and routes you explicitly define.
|
||||
|
||||
<img src="public/screenshots/users.png" alt="Users from identity provider with roles" width="100%" />
|
||||
|
||||
### Find and launch resources from a personalized home page
|
||||
|
||||
Give users a landing page to quickly find and open the resources they can access. Resources are grouped by site or label, searchable, and filterable, with grid or list views. Saved views capture filters, grouping, and layout as personal or organization-wide defaults.
|
||||
|
||||
* Single place for admins and non-admins to see accessible resources
|
||||
* Create reusable views for common access patterns
|
||||
|
||||
<img src="public/screenshots/resource-launcher.png" alt="Resource Launcher" width="100%" />
|
||||
|
||||
## Download Clients
|
||||
|
||||
Download the Pangolin client for your platform:
|
||||
@@ -167,7 +107,7 @@ the docs to illustrate some basic ideas.
|
||||
|
||||
## Licensing
|
||||
|
||||
Pangolin is dual licensed under the AGPL-3 and the [Fossorial Commercial License](https://pangolin.net/fcl). For inquiries about commercial licensing, please contact us at [contact@pangolin.net](mailto:contact@pangolin.net).
|
||||
Pangolin is dual licensed under the AGPL-3 and the [Fossorial Commercial License](https://pangolin.net/fcl.html). For inquiries about commercial licensing, please contact us at [contact@pangolin.net](mailto:contact@pangolin.net).
|
||||
|
||||
## Contributions
|
||||
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: Create API Key
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
put {
|
||||
url: http://localhost:3000/api/v1/api-key
|
||||
body: json
|
||||
auth: inherit
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"isRoot": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Delete API Key
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
delete {
|
||||
url: http://localhost:3000/api/v1/api-key/dm47aacqxxn3ubj
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: List API Key Actions
|
||||
type: http
|
||||
seq: 6
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/api-key/ex0izu2c37fjz9x/actions
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: List Org API Keys
|
||||
type: http
|
||||
seq: 4
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/org/home-lab/api-keys
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: List Root API Keys
|
||||
type: http
|
||||
seq: 3
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/root/api-keys
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: Set API Key Actions
|
||||
type: http
|
||||
seq: 5
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/api-key/ex0izu2c37fjz9x/actions
|
||||
body: json
|
||||
auth: inherit
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"actionIds": ["listSites"]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: Set API Key Orgs
|
||||
type: http
|
||||
seq: 7
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/api-key/ex0izu2c37fjz9x/orgs
|
||||
body: json
|
||||
auth: inherit
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"orgIds": ["home-lab"]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
meta {
|
||||
name: API Keys
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
meta {
|
||||
name: 2fa-disable
|
||||
type: http
|
||||
seq: 6
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/2fa/disable
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"password": "aaaaa-1A",
|
||||
"code": "377289"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: 2fa-enable
|
||||
type: http
|
||||
seq: 4
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/2fa/enable
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"code": "374138"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: 2fa-request
|
||||
type: http
|
||||
seq: 5
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/2fa/request
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"password": "aaaaa-1A"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
meta {
|
||||
name: change-password
|
||||
type: http
|
||||
seq: 9
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/change-password
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"oldPassword": "",
|
||||
"newPassword": ""
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
meta {
|
||||
name: login
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/login
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"email": "admin@fosrl.io",
|
||||
"password": "Password123!"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: logout
|
||||
type: http
|
||||
seq: 3
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:4000/api/v1/auth/logout
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: reset-password-request
|
||||
type: http
|
||||
seq: 10
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/reset-password/request
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"email": "milo@pangolin.net"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
meta {
|
||||
name: reset-password
|
||||
type: http
|
||||
seq: 11
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/reset-password
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"token": "3uhsbom72dwdhboctwrtntyd6jrlg4jtf5oaxy4k",
|
||||
"newPassword": "aaaaa-1A",
|
||||
"code": "6irqCGR3"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
meta {
|
||||
name: signup
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
put {
|
||||
url: http://localhost:3000/api/v1/auth/signup
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"email": "numbat@pangolin.net",
|
||||
"password": "Password123!"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: verify-email-request
|
||||
type: http
|
||||
seq: 8
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/verify-email/request
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
meta {
|
||||
name: verify-email
|
||||
type: http
|
||||
seq: 7
|
||||
}
|
||||
|
||||
post {
|
||||
url: http://localhost:3000/api/v1/auth/verify-email
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"code": "50317187"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
meta {
|
||||
name: verify-user
|
||||
type: http
|
||||
seq: 4
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3001/api/v1/badger/verify-user?sessionId=mb52273jkb6t3oys2bx6ur5x7rcrkl26c7warg3e
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
|
||||
params:query {
|
||||
sessionId: mb52273jkb6t3oys2bx6ur5x7rcrkl26c7warg3e
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
meta {
|
||||
name: createClient
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
put {
|
||||
url: http://localhost:3000/api/v1/site/1/client
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"siteId": 1,
|
||||
"name": "test",
|
||||
"type": "olm",
|
||||
"subnet": "100.90.129.4/30",
|
||||
"olmId": "029yzunhx6nh3y5",
|
||||
"secret": "l0ymp075y3d4rccb25l6sqpgar52k09etunui970qq5gj7x6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: pickClientDefaults
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/site/1/pick-client-defaults
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
meta {
|
||||
name: Create OIDC Provider
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
put {
|
||||
url: http://localhost:3000/api/v1/org/home-lab/idp/oidc
|
||||
body: json
|
||||
auth: inherit
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"clientId": "JJoSvHCZcxnXT2sn6CObj6a21MuKNRXs3kN5wbys",
|
||||
"clientSecret": "2SlGL2wOGgMEWLI9yUuMAeFxre7qSNJVnXMzyepdNzH1qlxYnC4lKhhQ6a157YQEkYH3vm40KK4RCqbYiF8QIweuPGagPX3oGxEj2exwutoXFfOhtq4hHybQKoFq01Z3",
|
||||
"authUrl": "http://localhost:9000/application/o/authorize/",
|
||||
"tokenUrl": "http://localhost:9000/application/o/token/",
|
||||
"scopes": ["email", "openid", "profile"],
|
||||
"userIdentifier": "email"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Generate OIDC URL
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
meta {
|
||||
name: IDP
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Traefik Config
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3001/api/v1/traefik-config
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,3 @@
|
||||
meta {
|
||||
name: Internal
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Create Newt
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/newt
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
meta {
|
||||
name: Get Token
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/auth/newt/get-token
|
||||
body: json
|
||||
auth: none
|
||||
}
|
||||
|
||||
body:json {
|
||||
{
|
||||
"newtId": "o0d4rdxq3stnz7b",
|
||||
"secret": "sy7l09fnaesd03iwrfp9m3qf0ryn19g0zf3dqieaazb4k7vk"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
meta {
|
||||
name: createOlm
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
put {
|
||||
url: http://localhost:3000/api/v1/olm
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
|
||||
settings {
|
||||
encodeUrl: true
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
meta {
|
||||
name: Olm
|
||||
seq: 15
|
||||
}
|
||||
|
||||
auth {
|
||||
mode: inherit
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Check Id
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/org/checkId
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: listOrgs
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url:
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: createRemoteExitNode
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
put {
|
||||
url: http://localhost:4000/api/v1/org/org_i21aifypnlyxur2/remote-exit-node
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: listResourcesByOrg
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url:
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
meta {
|
||||
name: listResourcesBySite
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/site/1/resources?limit=10&offset=0
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
|
||||
params:query {
|
||||
limit: 10
|
||||
offset: 0
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Get Site
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/org/test/sites/mexican-mole-lizard-windy
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: listSites
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url:
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
meta {
|
||||
name: listTargets
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/resource/web.main.localhost/targets?limit=10&offset=0
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
|
||||
params:query {
|
||||
limit: 10
|
||||
offset: 0
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: Test
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1
|
||||
body: none
|
||||
auth: inherit
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: traefik-config
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3001/api/v1/traefik-config
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: adminListUsers
|
||||
type: http
|
||||
seq: 2
|
||||
}
|
||||
|
||||
get {
|
||||
url: http://localhost:3000/api/v1/users
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: adminRemoveUser
|
||||
type: http
|
||||
seq: 3
|
||||
}
|
||||
|
||||
delete {
|
||||
url: http://localhost:3000/api/v1/user/ky5r7ivqs8wc7u4
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
meta {
|
||||
name: getUser
|
||||
type: http
|
||||
seq: 1
|
||||
}
|
||||
|
||||
get {
|
||||
url:
|
||||
body: none
|
||||
auth: none
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"version": "1",
|
||||
"name": "Pangolin",
|
||||
"type": "collection",
|
||||
"ignore": [
|
||||
"node_modules",
|
||||
".git"
|
||||
],
|
||||
"presets": {
|
||||
"requestType": "http",
|
||||
"requestUrl": "http://localhost:3000/api/v1"
|
||||
}
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
import { CommandModule } from "yargs";
|
||||
import { db, certificates } from "@server/db";
|
||||
|
||||
type ClearCertificatesArgs = {};
|
||||
|
||||
export const clearCertificates: CommandModule<{}, ClearCertificatesArgs> = {
|
||||
command: "clear-certificates",
|
||||
describe: "Delete all entries from the certificates table",
|
||||
builder: (yargs) => {
|
||||
return yargs;
|
||||
},
|
||||
handler: async (argv: {}) => {
|
||||
try {
|
||||
console.log("Clearing all certificates from the database...");
|
||||
|
||||
const deleted = await db.delete(certificates).returning();
|
||||
|
||||
console.log(
|
||||
`Deleted ${deleted.length} certificate(s) from the database`
|
||||
);
|
||||
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error("Error:", error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -23,7 +23,7 @@ export const clearExitNodes: CommandModule<
|
||||
// Delete all exit nodes
|
||||
const deletedCount = await db
|
||||
.delete(exitNodes)
|
||||
.where(eq(exitNodes.exitNodeId, exitNodes.exitNodeId)).returning();; // delete all
|
||||
.where(eq(exitNodes.exitNodeId, exitNodes.exitNodeId)) .returning();; // delete all
|
||||
|
||||
console.log(`Deleted ${deletedCount.length} exit node(s) from the database`);
|
||||
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
import { CommandModule } from "yargs";
|
||||
import { db, users } from "@server/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
|
||||
/**
|
||||
* Disable 2FA for a user by email address.
|
||||
*/
|
||||
type DisableUser2faArgs = {
|
||||
email: string;
|
||||
};
|
||||
|
||||
export const disableUser2fa: CommandModule<{}, DisableUser2faArgs> = {
|
||||
command: "disable-user-2fa",
|
||||
describe: "Disable 2FA for a user (sets twoFactorEnabled=false, clears secret)",
|
||||
builder: (yargs) => {
|
||||
return yargs.option("email", {
|
||||
type: "string",
|
||||
demandOption: true,
|
||||
describe: "User email address"
|
||||
});
|
||||
},
|
||||
handler: async (argv: { email: string }) => {
|
||||
try {
|
||||
const { email } = argv;
|
||||
console.log(`Looking for user with email: ${email}`);
|
||||
|
||||
// Find the user by email
|
||||
const [user] = await db
|
||||
.select()
|
||||
.from(users)
|
||||
.where(eq(users.email, email))
|
||||
.limit(1);
|
||||
|
||||
if (!user) {
|
||||
console.error(`User with email '${email}' not found`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (!user.twoFactorEnabled) {
|
||||
console.log(`2FA is already disabled for user '${email}'.`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Update user: disable 2FA and clear secret
|
||||
await db.update(users)
|
||||
.set({
|
||||
twoFactorEnabled: false,
|
||||
twoFactorSecret: null,
|
||||
twoFactorSetupRequested: false
|
||||
})
|
||||
.where(eq(users.userId, user.userId));
|
||||
|
||||
console.log(`2FA disabled for user '${email}'.`);
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error("Error disabling 2FA:", error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -5,7 +5,7 @@ import { encrypt } from "@server/lib/crypto";
|
||||
import { configFilePath1, configFilePath2 } from "@server/lib/consts";
|
||||
import { generateCA } from "@server/lib/sshCA";
|
||||
import fs from "fs";
|
||||
import * as yaml from "js-yaml";
|
||||
import yaml from "js-yaml";
|
||||
|
||||
type GenerateOrgCaKeysArgs = {
|
||||
orgId: string;
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import { CommandModule } from "yargs";
|
||||
import { db, idpOidcConfig, licenseKey, certificates, eventStreamingDestinations, alertWebhookActions, aiProviders, virtualApiKeys } from "@server/db";
|
||||
import { db, idpOidcConfig, licenseKey } from "@server/db";
|
||||
import { encrypt, decrypt } from "@server/lib/crypto";
|
||||
import { configFilePath1, configFilePath2 } from "@server/lib/consts";
|
||||
import { eq } from "drizzle-orm";
|
||||
import fs from "fs";
|
||||
import * as yaml from "js-yaml";
|
||||
import yaml from "js-yaml";
|
||||
|
||||
type RotateServerSecretArgs = {
|
||||
"old-secret": string;
|
||||
@@ -129,19 +129,9 @@ export const rotateServerSecret: CommandModule<
|
||||
console.log("\nReading encrypted data from database...");
|
||||
const idpConfigs = await db.select().from(idpOidcConfig);
|
||||
const licenseKeys = await db.select().from(licenseKey);
|
||||
const certs = await db.select().from(certificates);
|
||||
const streamingDestinations = await db.select().from(eventStreamingDestinations);
|
||||
const webhookActions = await db.select().from(alertWebhookActions);
|
||||
const providers = await db.select().from(aiProviders);
|
||||
const virtualKeys = await db.select().from(virtualApiKeys);
|
||||
|
||||
console.log(`Found ${idpConfigs.length} OIDC IdP configuration(s)`);
|
||||
console.log(`Found ${licenseKeys.length} license key(s)`);
|
||||
console.log(`Found ${certs.length} certificate(s)`);
|
||||
console.log(`Found ${streamingDestinations.length} event streaming destination(s)`);
|
||||
console.log(`Found ${webhookActions.length} alert webhook action(s)`);
|
||||
console.log(`Found ${providers.length} AI provider(s)`);
|
||||
console.log(`Found ${virtualKeys.length} virtual API key(s)`);
|
||||
|
||||
// Prepare all decrypted and re-encrypted values
|
||||
console.log("\nDecrypting and re-encrypting values...");
|
||||
@@ -159,40 +149,8 @@ export const rotateServerSecret: CommandModule<
|
||||
encryptedInstanceId: string;
|
||||
};
|
||||
|
||||
type CertUpdate = {
|
||||
certId: number;
|
||||
encryptedCertFile: string | null;
|
||||
encryptedKeyFile: string | null;
|
||||
};
|
||||
|
||||
type StreamingDestinationUpdate = {
|
||||
destinationId: number;
|
||||
encryptedConfig: string;
|
||||
};
|
||||
|
||||
type WebhookActionUpdate = {
|
||||
webhookActionId: number;
|
||||
encryptedConfig: string;
|
||||
};
|
||||
|
||||
type AiProviderUpdate = {
|
||||
providerId: number;
|
||||
encryptedApiKey: string | null;
|
||||
encryptedHeaders: string | null;
|
||||
};
|
||||
|
||||
type VirtualApiKeyUpdate = {
|
||||
virtualApiKeyId: string;
|
||||
encryptedToken: string;
|
||||
};
|
||||
|
||||
const idpUpdates: IdpUpdate[] = [];
|
||||
const licenseKeyUpdates: LicenseKeyUpdate[] = [];
|
||||
const certUpdates: CertUpdate[] = [];
|
||||
const streamingDestinationUpdates: StreamingDestinationUpdate[] = [];
|
||||
const webhookActionUpdates: WebhookActionUpdate[] = [];
|
||||
const aiProviderUpdates: AiProviderUpdate[] = [];
|
||||
const virtualApiKeyUpdates: VirtualApiKeyUpdate[] = [];
|
||||
|
||||
// Process idpOidcConfig entries
|
||||
for (const idpConfig of idpConfigs) {
|
||||
@@ -259,124 +217,6 @@ export const rotateServerSecret: CommandModule<
|
||||
}
|
||||
}
|
||||
|
||||
// Process certificate entries
|
||||
for (const cert of certs) {
|
||||
try {
|
||||
const encryptedCertFile = cert.certFile
|
||||
? encrypt(decrypt(cert.certFile, oldSecret), newSecret)
|
||||
: null;
|
||||
const encryptedKeyFile = cert.keyFile
|
||||
? encrypt(decrypt(cert.keyFile, oldSecret), newSecret)
|
||||
: null;
|
||||
|
||||
certUpdates.push({
|
||||
certId: cert.certId,
|
||||
encryptedCertFile,
|
||||
encryptedKeyFile
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error processing certificate ${cert.certId} (${cert.domain}):`,
|
||||
error
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Process eventStreamingDestinations entries
|
||||
for (const dest of streamingDestinations) {
|
||||
try {
|
||||
const decryptedConfig = decrypt(dest.config, oldSecret);
|
||||
const encryptedConfig = encrypt(decryptedConfig, newSecret);
|
||||
|
||||
streamingDestinationUpdates.push({
|
||||
destinationId: dest.destinationId,
|
||||
encryptedConfig
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error processing event streaming destination ${dest.destinationId}:`,
|
||||
error
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Process alertWebhookActions entries
|
||||
for (const webhook of webhookActions) {
|
||||
try {
|
||||
if (webhook.config == null) continue;
|
||||
|
||||
const decryptedConfig = decrypt(webhook.config, oldSecret);
|
||||
const encryptedConfig = encrypt(decryptedConfig, newSecret);
|
||||
|
||||
webhookActionUpdates.push({
|
||||
webhookActionId: webhook.webhookActionId,
|
||||
encryptedConfig
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error processing alert webhook action ${webhook.webhookActionId}:`,
|
||||
error
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Process aiProviders entries (apiKey + headers)
|
||||
for (const provider of providers) {
|
||||
try {
|
||||
if (!provider.apiKey && !provider.headers) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const encryptedApiKey = provider.apiKey
|
||||
? encrypt(decrypt(provider.apiKey, oldSecret), newSecret)
|
||||
: null;
|
||||
const encryptedHeaders = provider.headers
|
||||
? encrypt(
|
||||
decrypt(provider.headers, oldSecret),
|
||||
newSecret
|
||||
)
|
||||
: null;
|
||||
|
||||
aiProviderUpdates.push({
|
||||
providerId: provider.providerId,
|
||||
encryptedApiKey,
|
||||
encryptedHeaders
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error processing AI provider ${provider.providerId}:`,
|
||||
error
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Process virtualApiKeys entries (token)
|
||||
for (const key of virtualKeys) {
|
||||
try {
|
||||
if (!key.token) {
|
||||
continue;
|
||||
}
|
||||
|
||||
virtualApiKeyUpdates.push({
|
||||
virtualApiKeyId: key.virtualApiKeyId,
|
||||
encryptedToken: encrypt(
|
||||
decrypt(key.token, oldSecret),
|
||||
newSecret
|
||||
)
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Error processing virtual API key ${key.virtualApiKeyId}:`,
|
||||
error
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Perform all database updates in a single transaction
|
||||
console.log("\nUpdating database in transaction...");
|
||||
await db.transaction(async (trx) => {
|
||||
@@ -410,78 +250,10 @@ export const rotateServerSecret: CommandModule<
|
||||
instanceId: update.encryptedInstanceId
|
||||
});
|
||||
}
|
||||
|
||||
// Update certificate entries
|
||||
for (const update of certUpdates) {
|
||||
await trx
|
||||
.update(certificates)
|
||||
.set({
|
||||
certFile: update.encryptedCertFile,
|
||||
keyFile: update.encryptedKeyFile
|
||||
})
|
||||
.where(eq(certificates.certId, update.certId));
|
||||
}
|
||||
|
||||
// Update event streaming destination entries
|
||||
for (const update of streamingDestinationUpdates) {
|
||||
await trx
|
||||
.update(eventStreamingDestinations)
|
||||
.set({ config: update.encryptedConfig })
|
||||
.where(
|
||||
eq(
|
||||
eventStreamingDestinations.destinationId,
|
||||
update.destinationId
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// Update alert webhook action entries
|
||||
for (const update of webhookActionUpdates) {
|
||||
await trx
|
||||
.update(alertWebhookActions)
|
||||
.set({ config: update.encryptedConfig })
|
||||
.where(
|
||||
eq(
|
||||
alertWebhookActions.webhookActionId,
|
||||
update.webhookActionId
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// Update AI provider entries
|
||||
for (const update of aiProviderUpdates) {
|
||||
await trx
|
||||
.update(aiProviders)
|
||||
.set({
|
||||
apiKey: update.encryptedApiKey,
|
||||
headers: update.encryptedHeaders
|
||||
})
|
||||
.where(eq(aiProviders.providerId, update.providerId));
|
||||
}
|
||||
|
||||
// Update virtual API key entries
|
||||
for (const update of virtualApiKeyUpdates) {
|
||||
await trx
|
||||
.update(virtualApiKeys)
|
||||
.set({
|
||||
token: update.encryptedToken
|
||||
})
|
||||
.where(
|
||||
eq(
|
||||
virtualApiKeys.virtualApiKeyId,
|
||||
update.virtualApiKeyId
|
||||
)
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
console.log(`Rotated ${idpUpdates.length} OIDC IdP configuration(s)`);
|
||||
console.log(`Rotated ${licenseKeyUpdates.length} license key(s)`);
|
||||
console.log(`Rotated ${certUpdates.length} certificate(s)`);
|
||||
console.log(`Rotated ${streamingDestinationUpdates.length} event streaming destination(s)`);
|
||||
console.log(`Rotated ${webhookActionUpdates.length} alert webhook action(s)`);
|
||||
console.log(`Rotated ${aiProviderUpdates.length} AI provider(s)`);
|
||||
console.log(`Rotated ${virtualApiKeyUpdates.length} virtual API key(s)`);
|
||||
|
||||
// Update config file with new secret
|
||||
console.log("\nUpdating config file...");
|
||||
@@ -498,10 +270,6 @@ export const rotateServerSecret: CommandModule<
|
||||
console.log(`\nSummary:`);
|
||||
console.log(` - OIDC IdP configurations: ${idpUpdates.length}`);
|
||||
console.log(` - License keys: ${licenseKeyUpdates.length}`);
|
||||
console.log(` - Certificates: ${certUpdates.length}`);
|
||||
console.log(` - Event streaming destinations: ${streamingDestinationUpdates.length}`);
|
||||
console.log(` - Alert webhook actions: ${webhookActionUpdates.length}`);
|
||||
console.log(` - AI providers: ${aiProviderUpdates.length}`);
|
||||
console.log(
|
||||
`\n IMPORTANT: Restart the server for the new secret to take effect.`
|
||||
);
|
||||
|
||||
@@ -1,85 +0,0 @@
|
||||
import { CommandModule } from "yargs";
|
||||
import { db, users } from "@server/db";
|
||||
import { eq } from "drizzle-orm";
|
||||
|
||||
type SetServerAdminArgs = {
|
||||
email: string;
|
||||
remove: boolean;
|
||||
};
|
||||
|
||||
export const setServerAdmin: CommandModule<{}, SetServerAdminArgs> = {
|
||||
command: "set-server-admin",
|
||||
describe: "Add or remove server admin by email address",
|
||||
builder: (yargs) => {
|
||||
return yargs
|
||||
.option("email", {
|
||||
type: "string",
|
||||
demandOption: true,
|
||||
describe: "User email address"
|
||||
})
|
||||
.option("remove", {
|
||||
type: "boolean",
|
||||
default: false,
|
||||
describe: "Remove server admin status from the user"
|
||||
});
|
||||
},
|
||||
handler: async (argv: SetServerAdminArgs) => {
|
||||
try {
|
||||
const email = argv.email.trim().toLowerCase();
|
||||
|
||||
const [user] = await db
|
||||
.select()
|
||||
.from(users)
|
||||
.where(eq(users.email, email))
|
||||
.limit(1);
|
||||
|
||||
if (!user) {
|
||||
console.error(`User with email '${email}' not found`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
if (argv.remove) {
|
||||
if (!user.serverAdmin) {
|
||||
console.log(`User '${email}' is not a server admin`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const serverAdmins = await db
|
||||
.select()
|
||||
.from(users)
|
||||
.where(eq(users.serverAdmin, true));
|
||||
|
||||
if (serverAdmins.length <= 1) {
|
||||
console.error(
|
||||
"Cannot remove server admin: at least one server admin must exist"
|
||||
);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
await db
|
||||
.update(users)
|
||||
.set({ serverAdmin: false })
|
||||
.where(eq(users.userId, user.userId));
|
||||
|
||||
console.log(`Server admin status removed from user '${email}'`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
if (user.serverAdmin) {
|
||||
console.log(`User '${email}' is already a server admin`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
await db
|
||||
.update(users)
|
||||
.set({ serverAdmin: true })
|
||||
.where(eq(users.userId, user.userId));
|
||||
|
||||
console.log(`User '${email}' has been marked as a server admin`);
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error("Error:", error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -9,9 +9,6 @@ import { rotateServerSecret } from "./commands/rotateServerSecret";
|
||||
import { clearLicenseKeys } from "./commands/clearLicenseKeys";
|
||||
import { deleteClient } from "./commands/deleteClient";
|
||||
import { generateOrgCaKeys } from "./commands/generateOrgCaKeys";
|
||||
import { clearCertificates } from "./commands/clearCertificates";
|
||||
import { disableUser2fa } from "./commands/disableUser2fa";
|
||||
import { setServerAdmin } from "./commands/setServerAdmin";
|
||||
|
||||
yargs(hideBin(process.argv))
|
||||
.scriptName("pangctl")
|
||||
@@ -22,8 +19,5 @@ yargs(hideBin(process.argv))
|
||||
.command(clearLicenseKeys)
|
||||
.command(deleteClient)
|
||||
.command(generateOrgCaKeys)
|
||||
.command(clearCertificates)
|
||||
.command(disableUser2fa)
|
||||
.command(setServerAdmin)
|
||||
.demandCommand()
|
||||
.help().argv;
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
services:
|
||||
mailer:
|
||||
image: axllent/mailpit
|
||||
ports:
|
||||
- 8025:8025
|
||||
- 1025:1025
|
||||
volumes:
|
||||
- mailpit-storage:/data
|
||||
environment:
|
||||
- MP_DATABASE=/data/mailpit.db
|
||||
volumes:
|
||||
mailpit-storage:
|
||||
@@ -1,3 +0,0 @@
|
||||
## Example Docker Reference HA Deployment
|
||||
|
||||
This directory contains a basic reference config for a highly available deployment of Pangolin with two nodes. For more information [refer to the docs](https://docs.pangolin.net/self-host/clustering/understanding-clustering).
|
||||
@@ -1,23 +0,0 @@
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17
|
||||
container_name: postgres
|
||||
environment:
|
||||
POSTGRES_DB: postgres # Default database name
|
||||
POSTGRES_USER: postgres # Default user
|
||||
POSTGRES_PASSWORD: password # Default password (change for production!)
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "5432:5432"
|
||||
restart: always
|
||||
|
||||
redis:
|
||||
image: redis:latest
|
||||
container_name: redis
|
||||
ports:
|
||||
- "6379:6379"
|
||||
restart: always
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
@@ -1,38 +0,0 @@
|
||||
# To see all available options, please visit the docs:
|
||||
# https://docs.pangolin.net/
|
||||
|
||||
gerbil:
|
||||
start_port: 51820
|
||||
base_endpoint: "<THIS_NODE_EXTERNAL_IP>"
|
||||
exit_node_name: "node1"
|
||||
|
||||
app:
|
||||
dashboard_url: "https://pangolin.example.com"
|
||||
log_level: "info"
|
||||
|
||||
postgres:
|
||||
connection_string: postgresql://<POSTGRES_USERNAME>:<POSTGRES_PASSWORD>@<POSTGRES_INTERNAL_HOST>:5432/postgres
|
||||
|
||||
traefik:
|
||||
site_types: ["newt"] # Wireguard and local sites are not support in clustering
|
||||
file_mode: true # Pangolin will generate and save yaml files in a shared volume
|
||||
|
||||
server:
|
||||
secret: "<SECRET>"
|
||||
cors:
|
||||
origins: ["https://pangolin.example.com"]
|
||||
methods: ["GET", "POST", "PUT", "DELETE", "PATCH"]
|
||||
allowed_headers: ["X-CSRF-Token", "Content-Type"]
|
||||
credentials: false
|
||||
maxmind_db_path: "./config/GeoLite2-Country.mmdb" # Make sure to download and place into the config dir
|
||||
maxmind_asn_path: "./config/GeoLite2-ASN.mmdb"
|
||||
|
||||
flags:
|
||||
require_email_verification: false
|
||||
disable_signup_without_invite: true
|
||||
disable_user_create_org: false
|
||||
allow_raw_resources: false
|
||||
enable_acme_cert_sync: false
|
||||
disable_local_sites: true
|
||||
disable_basic_wireguard_sites: true
|
||||
disable_config_managed_domains: true
|
||||
@@ -1,67 +0,0 @@
|
||||
http:
|
||||
middlewares:
|
||||
badger:
|
||||
plugin:
|
||||
badger:
|
||||
disableForwardAuth: true
|
||||
|
||||
routers:
|
||||
# Next.js router (handles everything except API and WebSocket paths)
|
||||
next-router:
|
||||
rule: "!PathPrefix(`/api/v1`)"
|
||||
service: next-service
|
||||
entryPoints:
|
||||
- dashboard
|
||||
middlewares:
|
||||
- badger
|
||||
|
||||
# API router (handles /api/v1 paths)
|
||||
api-router:
|
||||
rule: "PathPrefix(`/api/v1`)"
|
||||
service: api-service
|
||||
entryPoints:
|
||||
- dashboard
|
||||
middlewares:
|
||||
- badger
|
||||
|
||||
# WebSocket router
|
||||
ws-router:
|
||||
rule: "PathPrefix(`/`)"
|
||||
service: api-service
|
||||
entryPoints:
|
||||
- dashboard
|
||||
middlewares:
|
||||
- badger
|
||||
|
||||
services:
|
||||
next-service:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: "http://pangolin:3002" # Next.js server
|
||||
|
||||
api-service:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: "http://pangolin:3000" # API/WebSocket server
|
||||
|
||||
tcp:
|
||||
serversTransports:
|
||||
pp-transport-v1:
|
||||
proxyProtocol:
|
||||
version: 1
|
||||
pp-transport-v2:
|
||||
proxyProtocol:
|
||||
version: 2
|
||||
|
||||
udp:
|
||||
routers:
|
||||
dns-router:
|
||||
entryPoints:
|
||||
- dns
|
||||
service: dns-service
|
||||
|
||||
services:
|
||||
dns-service:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- address: "pangolin:53"
|
||||
@@ -1,18 +0,0 @@
|
||||
app:
|
||||
region: "region1"
|
||||
identity_provider_mode: "org"
|
||||
redis:
|
||||
host: "<REDIS_INTERNAL_HOST>"
|
||||
port: 6379
|
||||
flags:
|
||||
enable_redis: true
|
||||
use_pangolin_dns: true
|
||||
acme:
|
||||
cert_mode: "pangolin"
|
||||
contact_email: "<CONTACT_EMAIL>"
|
||||
enable_acme_client: true
|
||||
dns:
|
||||
enabled: true
|
||||
nameserver_name: "ns.example.com"
|
||||
cname_extension: "cname.example.com"
|
||||
site_extension: "site.example.com" # Optional
|
||||
@@ -1,45 +0,0 @@
|
||||
providers:
|
||||
file:
|
||||
directory: "/var/dynamic"
|
||||
watch: true
|
||||
|
||||
experimental:
|
||||
plugins:
|
||||
badger:
|
||||
moduleName: "github.com/fosrl/badger"
|
||||
version: "v1.7.0"
|
||||
|
||||
log:
|
||||
level: "INFO"
|
||||
format: "common"
|
||||
maxSize: 100
|
||||
maxBackups: 3
|
||||
maxAge: 3
|
||||
compress: true
|
||||
|
||||
entryPoints:
|
||||
web:
|
||||
address: ":80"
|
||||
websecure:
|
||||
address: ":443"
|
||||
proxyProtocol: # We trust gerbil upstream
|
||||
trustedIPs:
|
||||
- 0.0.0.0/0
|
||||
- ::1/128
|
||||
transport:
|
||||
respondingTimeouts:
|
||||
readTimeout: "30m"
|
||||
http:
|
||||
encodedCharacters:
|
||||
allowEncodedSlash: true
|
||||
allowEncodedQuestionMark: true
|
||||
dashboard:
|
||||
address: ":3000"
|
||||
dns:
|
||||
address: ":53/udp"
|
||||
|
||||
serversTransport:
|
||||
insecureSkipVerify: true
|
||||
|
||||
ping:
|
||||
entryPoint: "web"
|
||||
@@ -1,62 +0,0 @@
|
||||
name: pangolin
|
||||
services:
|
||||
pangolin:
|
||||
image: docker.io/fosrl/pangolin:ee-latest
|
||||
container_name: pangolin
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./config:/app/config
|
||||
- ./config/certificates:/var/certificates
|
||||
- ./config/dynamic:/var/dynamic
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"]
|
||||
interval: "10s"
|
||||
timeout: "10s"
|
||||
retries: 15
|
||||
|
||||
gerbil:
|
||||
image: docker.io/fosrl/gerbil:latest
|
||||
container_name: gerbil
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
pangolin:
|
||||
condition: service_healthy
|
||||
command:
|
||||
- --reachableAt=http://<NODE1_INTERNAL_IP>:3004
|
||||
- --generateAndSaveKeyTo=/var/config/key
|
||||
- --remoteConfig=http://pangolin:3001/api/v1/
|
||||
- --trusted-upstreams=<NODE1_EXTERNAL_IP>,<NODE2_EXTERNAL_IP>
|
||||
volumes:
|
||||
- ./config/:/var/config
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
- SYS_MODULE
|
||||
ports:
|
||||
- 51820:51820/udp # wireguard
|
||||
- 21820:21820/udp # relay
|
||||
- 53:53/udp # DNS
|
||||
- 443:8443 # resources
|
||||
- 80:80 # web
|
||||
- 3004:3004 # gerbil api
|
||||
- 3000:3000 # Pangolin UI
|
||||
|
||||
traefik:
|
||||
image: docker.io/traefik:v3.7.11
|
||||
container_name: traefik
|
||||
restart: unless-stopped
|
||||
network_mode: service:gerbil # Ports appear on the gerbil service
|
||||
depends_on:
|
||||
pangolin:
|
||||
condition: service_healthy
|
||||
command:
|
||||
- --configFile=/etc/traefik/traefik_config.yml
|
||||
volumes:
|
||||
- ./config/traefik:/etc/traefik:ro
|
||||
- ./config/traefik/logs:/var/log/traefik
|
||||
- ./config/certificates:/var/certificates:ro
|
||||
- ./config/dynamic:/var/dynamic:ro
|
||||
|
||||
networks:
|
||||
default:
|
||||
driver: bridge
|
||||
name: pangolin
|
||||
@@ -1,38 +0,0 @@
|
||||
# To see all available options, please visit the docs:
|
||||
# https://docs.pangolin.net/
|
||||
|
||||
gerbil:
|
||||
start_port: 51820
|
||||
base_endpoint: "<THIS_NODE_EXTERNAL_IP>"
|
||||
exit_node_name: "node2"
|
||||
|
||||
app:
|
||||
dashboard_url: "https://pangolin.example.com"
|
||||
log_level: "info"
|
||||
|
||||
postgres:
|
||||
connection_string: postgresql://<POSTGRES_USERNAME>:<POSTGRES_PASSWORD>@<POSTGRES_INTERNAL_HOST>:5432/postgres
|
||||
|
||||
traefik:
|
||||
site_types: ["newt"] # Wireguard and local sites are not support in clustering
|
||||
file_mode: true # Pangolin will generate and save yaml files in a shared volume
|
||||
|
||||
server:
|
||||
secret: "<SECRET>"
|
||||
cors:
|
||||
origins: ["https://pangolin.example.com"]
|
||||
methods: ["GET", "POST", "PUT", "DELETE", "PATCH"]
|
||||
allowed_headers: ["X-CSRF-Token", "Content-Type"]
|
||||
credentials: false
|
||||
maxmind_db_path: "./config/GeoLite2-Country.mmdb" # Make sure to download and place into the config dir
|
||||
maxmind_asn_path: "./config/GeoLite2-ASN.mmdb"
|
||||
|
||||
flags:
|
||||
require_email_verification: false
|
||||
disable_signup_without_invite: true
|
||||
disable_user_create_org: false
|
||||
allow_raw_resources: false
|
||||
enable_acme_cert_sync: false
|
||||
disable_local_sites: true
|
||||
disable_basic_wireguard_sites: true
|
||||
disable_config_managed_domains: true
|
||||
@@ -1,67 +0,0 @@
|
||||
http:
|
||||
middlewares:
|
||||
badger:
|
||||
plugin:
|
||||
badger:
|
||||
disableForwardAuth: true
|
||||
|
||||
routers:
|
||||
# Next.js router (handles everything except API and WebSocket paths)
|
||||
next-router:
|
||||
rule: "!PathPrefix(`/api/v1`)"
|
||||
service: next-service
|
||||
entryPoints:
|
||||
- dashboard
|
||||
middlewares:
|
||||
- badger
|
||||
|
||||
# API router (handles /api/v1 paths)
|
||||
api-router:
|
||||
rule: "PathPrefix(`/api/v1`)"
|
||||
service: api-service
|
||||
entryPoints:
|
||||
- dashboard
|
||||
middlewares:
|
||||
- badger
|
||||
|
||||
# WebSocket router
|
||||
ws-router:
|
||||
rule: "PathPrefix(`/`)"
|
||||
service: api-service
|
||||
entryPoints:
|
||||
- dashboard
|
||||
middlewares:
|
||||
- badger
|
||||
|
||||
services:
|
||||
next-service:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: "http://pangolin:3002" # Next.js server
|
||||
|
||||
api-service:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: "http://pangolin:3000" # API/WebSocket server
|
||||
|
||||
tcp:
|
||||
serversTransports:
|
||||
pp-transport-v1:
|
||||
proxyProtocol:
|
||||
version: 1
|
||||
pp-transport-v2:
|
||||
proxyProtocol:
|
||||
version: 2
|
||||
|
||||
udp:
|
||||
routers:
|
||||
dns-router:
|
||||
entryPoints:
|
||||
- dns
|
||||
service: dns-service
|
||||
|
||||
services:
|
||||
dns-service:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- address: "pangolin:53"
|
||||
@@ -1,16 +0,0 @@
|
||||
app:
|
||||
region: "region1"
|
||||
identity_provider_mode: "org"
|
||||
redis:
|
||||
host: "<REDIS_INTERNAL_HOST>"
|
||||
port: 6379
|
||||
flags:
|
||||
enable_redis: true
|
||||
use_pangolin_dns: true
|
||||
acme:
|
||||
cert_mode: "pangolin"
|
||||
dns:
|
||||
enabled: true
|
||||
nameserver_name: "ns.example.com"
|
||||
cname_extension: "cname.example.com"
|
||||
site_extension: "site.example.com" # Optional
|
||||
@@ -1,45 +0,0 @@
|
||||
providers:
|
||||
file:
|
||||
directory: "/var/dynamic"
|
||||
watch: true
|
||||
|
||||
experimental:
|
||||
plugins:
|
||||
badger:
|
||||
moduleName: "github.com/fosrl/badger"
|
||||
version: "v1.7.0"
|
||||
|
||||
log:
|
||||
level: "INFO"
|
||||
format: "common"
|
||||
maxSize: 100
|
||||
maxBackups: 3
|
||||
maxAge: 3
|
||||
compress: true
|
||||
|
||||
entryPoints:
|
||||
web:
|
||||
address: ":80"
|
||||
websecure:
|
||||
address: ":443"
|
||||
proxyProtocol: # We trust gerbil upstream
|
||||
trustedIPs:
|
||||
- 0.0.0.0/0
|
||||
- ::1/128
|
||||
transport:
|
||||
respondingTimeouts:
|
||||
readTimeout: "30m"
|
||||
http:
|
||||
encodedCharacters:
|
||||
allowEncodedSlash: true
|
||||
allowEncodedQuestionMark: true
|
||||
dashboard:
|
||||
address: ":3000"
|
||||
dns:
|
||||
address: ":53/udp"
|
||||
|
||||
serversTransport:
|
||||
insecureSkipVerify: true
|
||||
|
||||
ping:
|
||||
entryPoint: "web"
|
||||
@@ -1,62 +0,0 @@
|
||||
name: pangolin
|
||||
services:
|
||||
pangolin:
|
||||
image: docker.io/fosrl/pangolin:ee-latest
|
||||
container_name: pangolin
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ./config:/app/config
|
||||
- ./config/certificates:/var/certificates
|
||||
- ./config/dynamic:/var/dynamic
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:3001/api/v1/"]
|
||||
interval: "10s"
|
||||
timeout: "10s"
|
||||
retries: 15
|
||||
|
||||
gerbil:
|
||||
image: docker.io/fosrl/gerbil:latest
|
||||
container_name: gerbil
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
pangolin:
|
||||
condition: service_healthy
|
||||
command:
|
||||
- --reachableAt=http://<NODE1_INTERNAL_IP>:3004
|
||||
- --generateAndSaveKeyTo=/var/config/key
|
||||
- --remoteConfig=http://pangolin:3001/api/v1/
|
||||
- --trusted-upstreams=<NODE1_EXTERNAL_IP>,<NODE2_EXTERNAL_IP>
|
||||
volumes:
|
||||
- ./config/:/var/config
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
- SYS_MODULE
|
||||
ports:
|
||||
- 51820:51820/udp # wireguard
|
||||
- 21820:21820/udp # relay
|
||||
- 53:53/udp # DNS
|
||||
- 443:8443 # resources
|
||||
- 80:80 # web
|
||||
- 3004:3004 # gerbil api
|
||||
- 3000:3000 # Pangolin UI
|
||||
|
||||
traefik:
|
||||
image: docker.io/traefik:v3.7.11
|
||||
container_name: traefik
|
||||
restart: unless-stopped
|
||||
network_mode: service:gerbil # Ports appear on the gerbil service
|
||||
depends_on:
|
||||
pangolin:
|
||||
condition: service_healthy
|
||||
command:
|
||||
- --configFile=/etc/traefik/traefik_config.yml
|
||||
volumes:
|
||||
- ./config/traefik:/etc/traefik:ro
|
||||
- ./config/traefik/logs:/var/log/traefik
|
||||
- ./config/certificates:/var/certificates:ro
|
||||
- ./config/dynamic:/var/dynamic:ro
|
||||
|
||||
networks:
|
||||
default:
|
||||
driver: bridge
|
||||
name: pangolin
|
||||
@@ -1,47 +1,54 @@
|
||||
api:
|
||||
insecure: true
|
||||
dashboard: true
|
||||
|
||||
providers:
|
||||
http:
|
||||
endpoint: http://pangolin:3001/api/v1/traefik-config
|
||||
pollInterval: 5s
|
||||
endpoint: "http://pangolin:3001/api/v1/traefik-config"
|
||||
pollInterval: "5s"
|
||||
file:
|
||||
filename: /etc/traefik/dynamic_config.yml
|
||||
filename: "/etc/traefik/dynamic_config.yml"
|
||||
|
||||
experimental:
|
||||
plugins:
|
||||
badger:
|
||||
moduleName: github.com/fosrl/badger
|
||||
version: v1.4.1
|
||||
moduleName: "github.com/fosrl/badger"
|
||||
version: "{{.BadgerVersion}}"
|
||||
|
||||
log:
|
||||
level: INFO
|
||||
format: common
|
||||
level: "INFO"
|
||||
format: "common"
|
||||
maxSize: 100
|
||||
maxBackups: 3
|
||||
maxAge: 3
|
||||
compress: true
|
||||
|
||||
certificatesResolvers:
|
||||
letsencrypt:
|
||||
acme:
|
||||
httpChallenge:
|
||||
entryPoint: web
|
||||
email: '{{.LetsEncryptEmail}}'
|
||||
storage: /letsencrypt/acme.json
|
||||
caServer: https://acme-v02.api.letsencrypt.org/directory
|
||||
email: "{{.LetsEncryptEmail}}"
|
||||
storage: "/letsencrypt/acme.json"
|
||||
caServer: "https://acme-v02.api.letsencrypt.org/directory"
|
||||
|
||||
entryPoints:
|
||||
web:
|
||||
address: ':80'
|
||||
address: ":80"
|
||||
websecure:
|
||||
address: ':443'
|
||||
address: ":443"
|
||||
transport:
|
||||
respondingTimeouts:
|
||||
readTimeout: 30m
|
||||
readTimeout: "30m"
|
||||
http:
|
||||
tls:
|
||||
certResolver: letsencrypt
|
||||
certResolver: "letsencrypt"
|
||||
encodedCharacters:
|
||||
allowEncodedSlash: true
|
||||
allowEncodedQuestionMark: true
|
||||
|
||||
serversTransport:
|
||||
insecureSkipVerify: true
|
||||
|
||||
ping:
|
||||
entryPoint: web
|
||||
entryPoint: "web"
|
||||
|
||||
@@ -41,7 +41,7 @@ services:
|
||||
- 80:80 # Port for traefik because of the network_mode
|
||||
|
||||
traefik:
|
||||
image: traefik:v3.7
|
||||
image: traefik:v3.6
|
||||
container_name: traefik
|
||||
restart: unless-stopped
|
||||
network_mode: service:gerbil # Ports appear on the gerbil service
|
||||
@@ -7,8 +7,8 @@ services:
|
||||
POSTGRES_DB: postgres # Default database name
|
||||
POSTGRES_USER: postgres # Default user
|
||||
POSTGRES_PASSWORD: password # Default password (change for production!)
|
||||
volumes:
|
||||
- ${HOME}/.local/share/pangolin-dev/postgres:/var/lib/postgresql/data
|
||||
# volumes:
|
||||
# - ./config/postgres:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "5432:5432" # Map host port 5432 to container port 5432
|
||||
restart: no
|
||||
@@ -1,350 +0,0 @@
|
||||
# AI Gateway Provider Selection
|
||||
|
||||
How the AI gateway picks which attached provider handles a request when an
|
||||
inference resource has more than one AI provider.
|
||||
|
||||
**Code:**
|
||||
|
||||
- Route → capability binding: `server/routers/aiGateway/createAiGatewayRouter.ts`
|
||||
- Request pipeline: `server/routers/aiGateway/pipeline.ts` (`selectProvider`)
|
||||
- Model discovery: `server/routers/aiGateway/v1Models.ts` and
|
||||
`server/lib/aiModelDiscovery.ts`
|
||||
- Tie-break scoring: `server/lib/aiProviderSelection.ts`
|
||||
- Allow/block matching: `server/lib/aiModelKeyMatch.ts`
|
||||
- Model catalog: `server/lib/aiModelCatalog.ts`
|
||||
- Default capabilities per provider type: `server/lib/aiProviderDefaults.ts`
|
||||
|
||||
Overlapping model allows are permitted at save time. Selection happens at
|
||||
request time. If the algorithm cannot confidently pick one provider, the
|
||||
gateway returns `403` with an ambiguous-provider error.
|
||||
|
||||
## Selection Pipeline
|
||||
|
||||
Every gateway request runs through these steps in order. Each step narrows
|
||||
the candidate set. Later steps only run when more than one provider remains.
|
||||
|
||||
```
|
||||
1. Capability filter
|
||||
2. Allow / block lists
|
||||
3. Most specific allow pattern
|
||||
4. Catalog ownership
|
||||
5. Provider class preference
|
||||
6. Ambiguous → error
|
||||
```
|
||||
|
||||
### 1. Capability Filter
|
||||
|
||||
The incoming path selects a capability before any provider logic runs.
|
||||
|
||||
| Path | Capability |
|
||||
|------|------------|
|
||||
| `POST /v1/chat/completions` | `openai_chat` |
|
||||
| `POST /v1/responses` | `openai_responses` |
|
||||
| `POST /v1/messages` | `anthropic_messages` |
|
||||
| `GET /v1/models`, `GET /v1/models/{id}` | `v1_models` |
|
||||
| Gemini / Vertex / Bedrock routes | their respective capability ids |
|
||||
|
||||
Only attached providers that advertise that capability stay in the candidate
|
||||
set. Default capabilities do not overlap for native OpenAI vs Anthropic:
|
||||
|
||||
| Provider type | Default capabilities |
|
||||
|---------------|----------------------|
|
||||
| `openai` | `openai_chat`, `openai_responses` |
|
||||
| `anthropic` | `anthropic_messages`, `v1_models` |
|
||||
| `openRouter` | `openai_chat` |
|
||||
| `vercelAiGateway` | `openai_chat`, `openai_responses` |
|
||||
| `microsoftFoundry` | `openai_chat`, `openai_responses`, `anthropic_messages`, `v1_models` |
|
||||
| `custom` | whatever was configured |
|
||||
|
||||
### 2. Allow / Block Lists
|
||||
|
||||
For each remaining provider, the gateway resolves the effective allow and
|
||||
block patterns:
|
||||
|
||||
- **`inherit`**: use the provider's own model lists
|
||||
- **`select`**: use the resource-selected subset of those lists
|
||||
|
||||
A candidate is kept only if `isAllowedByLists(requestedModel, allows, blocks)`
|
||||
passes:
|
||||
|
||||
1. At least one allow pattern must match
|
||||
2. No block pattern may match
|
||||
|
||||
Patterns support `*` and `?` globs (`gpt-*`, `claude-3-5-sonnet-?`).
|
||||
|
||||
### 3. Most Specific Allow Pattern
|
||||
|
||||
Among providers that allow the model, keep those whose matching allow
|
||||
pattern is most specific:
|
||||
|
||||
1. Exact keys beat patterns
|
||||
2. Fewer wildcard characters win
|
||||
3. Longer literal length wins
|
||||
|
||||
Example: `gpt-4o` beats `gpt-*` beats `*`.
|
||||
|
||||
### 4. Catalog Ownership
|
||||
|
||||
When specificity is tied (common with multiple `*` allows), score each
|
||||
provider against the known model catalog:
|
||||
|
||||
| Score | Meaning |
|
||||
|------:|---------|
|
||||
| 2 | Typed provider whose catalog contains the model (`openai` → openai catalog, `anthropic` → anthropic, etc.) |
|
||||
| 1 | Aggregator or custom (`openRouter`, `vercelAiGateway`, `custom`) and the model exists somewhere in the catalog |
|
||||
| 0 | No ownership signal (typed catalog miss, or unknown model on aggregator/custom) |
|
||||
|
||||
Model id lookup tries the raw id, then a stripped `vendor/model` form
|
||||
(e.g. `openai/gpt-4o` → also try `gpt-4o`).
|
||||
|
||||
Typed providers map to catalog providers as:
|
||||
|
||||
| Provider type | Catalog |
|
||||
|---------------|---------|
|
||||
| `openai` | `openai` |
|
||||
| `anthropic` | `anthropic` |
|
||||
| `googleGemini` | `gemini` |
|
||||
| `vertexAi` | `vertex` |
|
||||
| `bedrock` | `bedrock` |
|
||||
| `microsoftFoundry` | `azure` |
|
||||
| `openRouter` / `vercelAiGateway` / `custom` | none (aggregator/custom path) |
|
||||
|
||||
### 5. Provider Class Preference
|
||||
|
||||
If catalog ownership is still tied, prefer:
|
||||
|
||||
| Rank | Class |
|
||||
|-----:|-------|
|
||||
| 2 | Native typed provider (`openai`, `anthropic`, `googleGemini`, …) |
|
||||
| 1 | Aggregator (`openRouter`, `vercelAiGateway`) |
|
||||
| 0 | `custom` |
|
||||
|
||||
### 6. Ambiguous Error
|
||||
|
||||
If more than one distinct provider remains after all steps, the gateway
|
||||
rejects the request:
|
||||
|
||||
```
|
||||
Model "<id>" is ambiguous across multiple AI providers on this resource
|
||||
```
|
||||
|
||||
Typical remaining ties: two OpenAI-type providers both with `*`, or two
|
||||
customs advertising the same capability for an unknown model.
|
||||
|
||||
## Model Discovery Is Not Selection
|
||||
|
||||
`GET /v1/models` and `GET /v1/models/{id}` (`v1_models`) skip steps 3-6
|
||||
entirely. There is no requested model to disambiguate on, so the gateway does
|
||||
not pick one provider - it returns the **union** of what every attached
|
||||
provider advertising `v1_models` would accept, deduplicated by model id
|
||||
(lowest `providerId` wins a collision).
|
||||
|
||||
Discovery is answered from the gateway's own view of the allow/block lists,
|
||||
never proxied upstream. Providers that expose no `/v1/models` endpoint of their
|
||||
own still get a working listing, and a model an allow/block list forbids is
|
||||
never advertised.
|
||||
|
||||
Each provider's candidate ids come from two places:
|
||||
|
||||
| Source | Contributes |
|
||||
|--------|-------------|
|
||||
| Exact (non-wildcard) allow entries | the model key itself |
|
||||
| The model catalog for the provider's type | every catalog id matching an allow pattern |
|
||||
|
||||
Both sources are then filtered through the same
|
||||
`isAllowedByLists(id, allows, blocks)` check step 2 applies, so a block pattern
|
||||
hides a model from discovery exactly as it would reject it at request time.
|
||||
|
||||
The catalog source is what makes a wildcard allow such as `claude-*`
|
||||
enumerable. Provider types with no catalog mapping (`openRouter`,
|
||||
`vercelAiGateway`, `custom`) have nothing to expand against, so a wildcard
|
||||
allow on those types lists nothing - **add exact allow entries to make their
|
||||
models discoverable.**
|
||||
|
||||
### Where each field comes from
|
||||
|
||||
Token limits and capability flags can't be derived from an allow/block list.
|
||||
They come from the model catalog (`server/lib/aiModelCatalog.ts`), which the
|
||||
Fossorial API builds from LiteLLM:
|
||||
|
||||
| Field | Source |
|
||||
|-------|--------|
|
||||
| `max_input_tokens` | catalog `limits.input` |
|
||||
| `max_tokens` | catalog `limits.output` |
|
||||
| `capabilities` | catalog flags, mapped to the Models API shape by `capabilitiesFromCatalog` |
|
||||
| `display_name` | the configured model row's name, else the model id |
|
||||
| `created_at` | the configured model row's timestamp, else the epoch |
|
||||
|
||||
A model the catalog doesn't know (an exact allow entry for a fine-tune, say)
|
||||
reports `null` for all three metadata fields. The Models API declares them
|
||||
nullable, so that is a valid answer rather than a broken one.
|
||||
|
||||
The catalog's flags are coarser than the Models API describes: it carries a
|
||||
single `reasoning` flag with no way to distinguish adaptive from
|
||||
`budget_tokens`-style thinking, and nothing at all for batch, citations, code
|
||||
execution, PDF input, or context management. Anything it reports as unknown
|
||||
(`null`) is surfaced as unsupported rather than invented, so `capabilities`
|
||||
understates rather than overstates what a model can do.
|
||||
|
||||
The gateway does **not** query the provider's own `/v1/models`. Discovery is
|
||||
answered entirely from local state.
|
||||
|
||||
Results are ordered newest-first with the id as tie-break, and paginated with
|
||||
Anthropic's `limit` / `after_id` / `before_id` semantics (default 20, max
|
||||
1000).
|
||||
|
||||
## Examples
|
||||
|
||||
Assume each provider below is attached and enabled on the same inference
|
||||
resource.
|
||||
|
||||
### Example A: OpenAI + Anthropic, Both `*`
|
||||
|
||||
| Provider | Allow | Capabilities |
|
||||
|----------|-------|--------------|
|
||||
| OpenAI | `*` | `openai_chat`, `openai_responses` |
|
||||
| Anthropic | `*` | `anthropic_messages` |
|
||||
|
||||
**Request:** `POST /v1/chat/completions` with `model: "gpt-4o"`
|
||||
|
||||
1. Capability → only OpenAI remains
|
||||
2. Allow → OpenAI matches `*`
|
||||
3. Result → **OpenAI**
|
||||
|
||||
Anthropic never reaches pattern or catalog scoring. Capability alone decides.
|
||||
|
||||
**Request:** `POST /v1/messages` with `model: "claude-3-5-sonnet-latest"`
|
||||
|
||||
1. Capability → only Anthropic remains
|
||||
2. Result → **Anthropic**
|
||||
|
||||
### Example B: OpenAI + OpenRouter, Both `*`
|
||||
|
||||
| Provider | Allow | Capabilities |
|
||||
|----------|-------|--------------|
|
||||
| OpenAI | `*` | `openai_chat`, … |
|
||||
| OpenRouter | `*` | `openai_chat` |
|
||||
|
||||
**Request:** `POST /v1/chat/completions` with `model: "gpt-4o"`
|
||||
|
||||
1. Capability → both remain (`openai_chat`)
|
||||
2. Allow → both match `*`
|
||||
3. Specificity → tie (`*` vs `*`)
|
||||
4. Catalog → OpenAI scores `2` (owns `gpt-4o`); OpenRouter scores `1`
|
||||
5. Result → **OpenAI**
|
||||
|
||||
### Example C: OpenRouter Only Serving a Claude Model Over OpenAI Chat
|
||||
|
||||
| Provider | Allow | Capabilities |
|
||||
|----------|-------|--------------|
|
||||
| OpenRouter | `*` | `openai_chat` |
|
||||
|
||||
**Request:** `POST /v1/chat/completions` with `model: "anthropic/claude-3.5-sonnet"`
|
||||
|
||||
1. Capability → OpenRouter remains
|
||||
2. Only one candidate → **OpenRouter**
|
||||
|
||||
No tie-breaking needed.
|
||||
|
||||
### Example D: OpenAI (`gpt-*`) + OpenRouter (`*`)
|
||||
|
||||
| Provider | Allow |
|
||||
|----------|-------|
|
||||
| OpenAI | `gpt-*` |
|
||||
| OpenRouter | `*` |
|
||||
|
||||
**Request:** `model: "gpt-4o"` on `openai_chat`
|
||||
|
||||
1. Capability → both
|
||||
2. Allow → both match
|
||||
3. Specificity → OpenAI's `gpt-*` beats OpenRouter's `*`
|
||||
4. Result → **OpenAI**
|
||||
|
||||
Catalog scoring is not needed because specificity already unique'd the set.
|
||||
|
||||
### Example E: OpenAI + Anthropic With Overlapping Custom Capabilities
|
||||
|
||||
Someone grants Anthropic `openai_chat` as well (non-default).
|
||||
|
||||
| Provider | Allow | Capabilities |
|
||||
|----------|-------|--------------|
|
||||
| OpenAI | `*` | `openai_chat`, … |
|
||||
| Anthropic | `*` | `anthropic_messages`, `openai_chat` |
|
||||
|
||||
**Request:** `POST /v1/chat/completions` with `model: "gpt-4o"`
|
||||
|
||||
1. Capability → both remain
|
||||
2. Allow → both match `*`
|
||||
3. Specificity → tie
|
||||
4. Catalog → OpenAI `2`, Anthropic `0` (`gpt-4o` is not in the anthropic catalog)
|
||||
5. Result → **OpenAI**
|
||||
|
||||
### Example F: Two Aggregators, Known Model
|
||||
|
||||
| Provider | Allow |
|
||||
|----------|-------|
|
||||
| OpenRouter | `*` |
|
||||
| Vercel AI Gateway | `*` |
|
||||
|
||||
**Request:** `model: "gpt-4o"` on `openai_chat`
|
||||
|
||||
1. Capability → both
|
||||
2. Allow / specificity → tie
|
||||
3. Catalog → both score `1` (known model, no typed owner in the set)
|
||||
4. Class → both aggregators (rank `1`) → still tied
|
||||
5. Result → **ambiguous error**
|
||||
|
||||
Attach a native OpenAI provider (or narrow one aggregator's allow list) to
|
||||
make this determinable.
|
||||
|
||||
### Example G: Two OpenAI Providers, Both `*`
|
||||
|
||||
| Provider | Type | Allow |
|
||||
|----------|------|-------|
|
||||
| OpenAI Prod | `openai` | `*` |
|
||||
| OpenAI Staging | `openai` | `*` |
|
||||
|
||||
**Request:** `model: "gpt-4o"`
|
||||
|
||||
1–5 all leave both candidates (same capability, same specificity, same
|
||||
catalog ownership, same class).
|
||||
|
||||
Result → **ambiguous error**
|
||||
|
||||
Disambiguate with different allow patterns, disable one attachment, or
|
||||
split across resources.
|
||||
|
||||
### Example H: Unknown Model Across Native + Aggregator
|
||||
|
||||
| Provider | Allow |
|
||||
|----------|-------|
|
||||
| OpenAI | `*` |
|
||||
| OpenRouter | `*` |
|
||||
|
||||
**Request:** `model: "my-fine-tune-v3"` (not in catalog)
|
||||
|
||||
1. Capability → both
|
||||
2. Allow / specificity → tie
|
||||
3. Catalog → both score `0` (typed miss + unknown aggregator model)
|
||||
4. Class → OpenAI (`2`) beats OpenRouter (`1`)
|
||||
5. Result → **OpenAI**
|
||||
|
||||
## Practical Guidance
|
||||
|
||||
- Native OpenAI + Anthropic with `*` is safe. Different default APIs never
|
||||
collide.
|
||||
- OpenAI + OpenRouter with `*` is usually fine for catalog-known OpenAI
|
||||
models. Native wins.
|
||||
- Prefer specific allow patterns (`gpt-4o`, `gpt-*`) when two providers share
|
||||
a capability.
|
||||
- Two providers of the same type both using `*` will stay ambiguous. Narrow
|
||||
at least one allow list.
|
||||
- Custom providers only win ties when no stronger native/aggregator signal
|
||||
remains.
|
||||
|
||||
## Related Behavior
|
||||
|
||||
- **Saving providers on a resource does not reject overlapping allows.**
|
||||
Collisions are resolved (or rejected) per request.
|
||||
- Budgets, auth, and upstream URL / target routing run after a single
|
||||
provider has been selected.
|
||||
@@ -1,347 +0,0 @@
|
||||
# How to build a CRUD endpoint in this repo
|
||||
|
||||
Reference for adding a new CRUD entity to the server. Based on two real
|
||||
examples already in the codebase — read them side by side with this doc:
|
||||
|
||||
- **Public / open-source (Community Edition) pattern**: `server/routers/aiProvider/`
|
||||
- **Enterprise-only pattern**: `server/private/routers/alertRule/`
|
||||
|
||||
The two are structurally identical. The only difference is *where the files
|
||||
live* and *which router they get wired into*.
|
||||
|
||||
## 1. Decide: public or private?
|
||||
|
||||
- `server/routers/<entity>/` — ships in the open-source Community Edition.
|
||||
Anyone running Pangolin gets this.
|
||||
- `server/private/routers/<entity>/` — Enterprise/SaaS only. Gated behind
|
||||
`verifyValidLicense` (and often `verifyValidSubscription(tierMatrix.x)`).
|
||||
Every file here starts with the Fossorial Commercial License header block
|
||||
(copy it verbatim from an existing private file).
|
||||
|
||||
Everything below applies to both — swap `@server/...` for `#private/...`
|
||||
import paths and add license headers when building the private version.
|
||||
|
||||
## 2. Directory layout
|
||||
|
||||
One folder per entity, one file per operation, a barrel `index.ts`:
|
||||
|
||||
```
|
||||
server/routers/<entity>/
|
||||
index.ts # export * from each operation file + ./types
|
||||
types.ts # response payload types + row->public mapper
|
||||
validation.ts # zod schemas/refinements shared by create + update (optional)
|
||||
create<Entity>.ts
|
||||
list<Entities>.ts
|
||||
get<Entity>.ts
|
||||
update<Entity>.ts
|
||||
delete<Entity>.ts
|
||||
```
|
||||
|
||||
`index.ts` is a flat barrel:
|
||||
|
||||
```ts
|
||||
export * from "./createAiProvider";
|
||||
export * from "./listAiProviders";
|
||||
export * from "./getAiProvider";
|
||||
export * from "./updateAiProvider";
|
||||
export * from "./deleteAiProvider";
|
||||
export * from "./types";
|
||||
```
|
||||
|
||||
## 3. Anatomy of a single handler
|
||||
|
||||
Every handler file (`create<Entity>.ts`, etc.) follows the same shape:
|
||||
|
||||
```ts
|
||||
import { Request, Response, NextFunction } from "express";
|
||||
import { z } from "zod";
|
||||
import { <table>, db } from "@server/db";
|
||||
import response from "@server/lib/response";
|
||||
import HttpCode from "@server/types/HttpCode";
|
||||
import createHttpError from "http-errors";
|
||||
import logger from "@server/logger";
|
||||
import { fromError } from "zod-validation-error";
|
||||
import { OpenAPITags, registry } from "@server/openApi";
|
||||
import { eq } from "drizzle-orm";
|
||||
import type { GetXResponse } from "@server/routers/<entity>/types";
|
||||
|
||||
const paramsSchema = z.strictObject({
|
||||
orgId: z.string().nonempty() // or entityId: z.coerce.number().int().positive()
|
||||
});
|
||||
|
||||
const bodySchema = z.strictObject({ /* ... */ }); // create/update only
|
||||
|
||||
registry.registerPath({
|
||||
method: "get", // put | post | delete
|
||||
path: "/org/{orgId}/x",
|
||||
description: "...",
|
||||
tags: [OpenAPITags.<Entity>],
|
||||
request: { params: paramsSchema, /* body: {...} for write ops, query: for list */ },
|
||||
responses: { 200: { description: "Successful response" } }
|
||||
});
|
||||
|
||||
export async function getX(req: Request, res: Response, next: NextFunction): Promise<any> {
|
||||
try {
|
||||
const parsedParams = paramsSchema.safeParse(req.params);
|
||||
if (!parsedParams.success) {
|
||||
return next(createHttpError(HttpCode.BAD_REQUEST, fromError(parsedParams.error).toString()));
|
||||
}
|
||||
// parse body too, if present, same pattern
|
||||
|
||||
// ...business logic against db...
|
||||
|
||||
if (!row) {
|
||||
return next(createHttpError(HttpCode.NOT_FOUND, `X with ID ${id} not found`));
|
||||
}
|
||||
|
||||
return response<GetXResponse>(res, {
|
||||
data: { /* ... */ },
|
||||
success: true,
|
||||
error: false,
|
||||
message: "X retrieved successfully",
|
||||
status: HttpCode.OK
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error(error);
|
||||
return next(createHttpError(HttpCode.INTERNAL_SERVER_ERROR, "An error occurred"));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Rules to keep consistent with the rest of the codebase:
|
||||
|
||||
- `z.strictObject` for params/body — rejects unknown keys.
|
||||
- Params parsed first, then body; each on its own `safeParse` + early
|
||||
`next(createHttpError(...))` — never throw raw errors.
|
||||
- Every handler registers itself with the OpenAPI `registry` even if nobody
|
||||
reads the spec directly — it's how `/api/v1/docs` stays accurate.
|
||||
- Catch-all `try/catch` at the bottom: `logger.error(error)` +
|
||||
generic `500` message. Never leak internal error details to the client.
|
||||
- Use `response<T>(res, { data, success, error, message, status })` from
|
||||
`@server/lib/response` for every response, success or otherwise (errors go
|
||||
through `next(createHttpError(...))` instead, not through `response`).
|
||||
- If the route already ran an access-control middleware that fetched the row
|
||||
(see §5), reuse it instead of re-querying:
|
||||
`req.aiProvider && req.aiProvider.providerId === providerId ? [req.aiProvider] : await db.select()...`
|
||||
|
||||
### List handler specifics
|
||||
|
||||
Pagination is a fixed shape (`page`, `pageSize`, optional `query` for
|
||||
search). See `listAiProviders.ts`:
|
||||
|
||||
```ts
|
||||
const listSchema = z.object({
|
||||
pageSize: z.coerce.number<string>().int().positive().optional().catch(20).default(20),
|
||||
page: z.coerce.number<string>().int().min(0).optional().catch(1).default(1),
|
||||
query: z.string().optional()
|
||||
});
|
||||
```
|
||||
|
||||
Run the count query and the page query in `Promise.all`, and return
|
||||
`PaginatedResponse<{ items: T[] }>` (`@server/types/Pagination`) with
|
||||
`{ total, pageSize, page }`.
|
||||
|
||||
### types.ts specifics
|
||||
|
||||
- Define one response type per operation: `List<Entities>Response`,
|
||||
`Get<Entity>Response`, `CreateOrEdit<Entity>Response` (create and update
|
||||
commonly share a response shape).
|
||||
- If the raw DB row needs to be shaped for clients (decrypting secrets,
|
||||
parsing a serialized column, hiding a column), put a `toPublic<Entity>()`
|
||||
mapper here — see `toPublicAiProvider` for the pattern of stripping
|
||||
`apiKey`/serialized columns and re-adding decrypted/parsed versions.
|
||||
|
||||
### validation.ts specifics
|
||||
|
||||
Only needed when create and update share non-trivial zod pieces (enums,
|
||||
`superRefine` cross-field rules). Export the raw schemas (`z.enum([...])`)
|
||||
and refinement functions, and import them into both `createX.ts` and
|
||||
`updateX.ts` — see `aiProvider/validation.ts`'s
|
||||
`refineProviderUpstreamFields`.
|
||||
|
||||
## 4. Wire up an access-control middleware (for id-scoped routes)
|
||||
|
||||
For routes scoped to a single row (`/x/:xId`, as opposed to
|
||||
`/org/:orgId/x` create/list), add a `verify<Entity>Access` middleware in
|
||||
`server/middlewares/` (or `server/private/middlewares/` for enterprise-only
|
||||
entities) and export it from that directory's `index.ts`.
|
||||
|
||||
Pattern (`verifyAiProviderAccess.ts`):
|
||||
|
||||
1. Read the id param, `Number.parseInt`/validate it.
|
||||
2. Load the row by id.
|
||||
3. `404` if it doesn't exist.
|
||||
4. Resolve the row's `orgId`, then check/attach `req.userOrg` (query
|
||||
`userOrgs` if not already on the request), `403` if the user isn't in
|
||||
that org.
|
||||
5. Run `checkOrgAccessPolicy` if `req.orgPolicyAllowed` hasn't been resolved
|
||||
yet.
|
||||
6. Set `req.userOrgId`, `req.userOrgRoleIds`, and stash the row on the
|
||||
request (e.g. `req.aiProvider = provider`) so downstream handlers and
|
||||
`verifyUserHasAction` don't have to refetch it.
|
||||
|
||||
Org-scoped create/list routes (`/org/:orgId/x`) don't need a bespoke
|
||||
middleware — they use the existing generic `verifyOrgAccess` from
|
||||
`@server/middlewares`.
|
||||
|
||||
## 5. Register an action + permission check
|
||||
|
||||
Add one `ActionsEnum` entry per operation in `server/auth/actions.ts`,
|
||||
grouped near the entity's other actions, named `create<Entity>`,
|
||||
`get<Entity>`, `update<Entity>`, `delete<Entity>`, `list<Entities>`:
|
||||
|
||||
```ts
|
||||
createAiProvider = "createAiProvider",
|
||||
deleteAiProvider = "deleteAiProvider",
|
||||
getAiProvider = "getAiProvider",
|
||||
listAiProviders = "listAiProviders",
|
||||
updateAiProvider = "updateAiProvider",
|
||||
```
|
||||
|
||||
Every route uses `verifyUserHasAction(ActionsEnum.x)` to check the caller's
|
||||
role/permissions for that action, and mutating routes (create/update/delete)
|
||||
follow it with `logActionAudit(ActionsEnum.x)` to record the action in the
|
||||
audit log.
|
||||
|
||||
## 6. Register the routes
|
||||
|
||||
There are four router files; which one(s) you touch depends on public vs.
|
||||
private and user-facing vs. service-to-service:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `server/routers/external.ts` | Public, user-facing API. Exports `authenticated`, `unauthenticated`, `authRouter` Express routers. |
|
||||
| `server/routers/internal.ts` | Public, internal service-to-service API (gerbil, badger, traefik-config) — no user auth, exports `internalRouter`. |
|
||||
| `server/private/routers/external.ts` | Enterprise-only, user-facing. Imports `authenticated`/`unauthenticated`/`authRouter` **from the public `external.ts`** and re-exports them, then adds more routes on top. |
|
||||
| `server/private/routers/internal.ts` | Enterprise-only, service-to-service. Same re-export trick with `internalRouter`. |
|
||||
|
||||
Private router files always start:
|
||||
|
||||
```ts
|
||||
import {
|
||||
unauthenticated as ua,
|
||||
authenticated as a,
|
||||
authRouter as aa
|
||||
} from "@server/routers/external";
|
||||
|
||||
export const authenticated = a;
|
||||
export const unauthenticated = ua;
|
||||
export const authRouter = aa;
|
||||
```
|
||||
|
||||
...and then call `authenticated.get/put/post/delete(...)` to bolt on
|
||||
additional, enterprise-only routes on the *same* router instances the public
|
||||
build uses. This is why the private build has strictly more routes than the
|
||||
public build, not a divergent copy.
|
||||
|
||||
### Route registration order (mutating vs read)
|
||||
|
||||
Standard middleware chain per verb, using `alertRule`'s registrations as the
|
||||
template:
|
||||
|
||||
```ts
|
||||
// Create — org-scoped, no row exists yet
|
||||
authenticated.put(
|
||||
"/org/:orgId/x",
|
||||
verifyValidLicense, // private/enterprise routes only
|
||||
verifyOrgAccess,
|
||||
verifyLimits, // if the entity counts against a plan limit
|
||||
verifyUserHasAction(ActionsEnum.createX),
|
||||
logActionAudit(ActionsEnum.createX),
|
||||
x.createX
|
||||
);
|
||||
|
||||
// Update — row-scoped
|
||||
authenticated.post(
|
||||
"/org/:orgId/x/:xId", // or "/x/:xId" if id is globally unique
|
||||
verifyValidLicense,
|
||||
verifyOrgAccess, // or verifyXAccess if globally-keyed
|
||||
verifyUserHasAction(ActionsEnum.updateX),
|
||||
logActionAudit(ActionsEnum.updateX),
|
||||
x.updateX
|
||||
);
|
||||
|
||||
// Delete — row-scoped
|
||||
authenticated.delete(
|
||||
"/org/:orgId/x/:xId",
|
||||
verifyValidLicense,
|
||||
verifyOrgAccess,
|
||||
verifyUserHasAction(ActionsEnum.deleteX),
|
||||
logActionAudit(ActionsEnum.deleteX),
|
||||
x.deleteX
|
||||
);
|
||||
|
||||
// List — org-scoped, read-only, no audit log
|
||||
authenticated.get(
|
||||
"/org/:orgId/xs",
|
||||
verifyValidLicense,
|
||||
verifyOrgAccess,
|
||||
verifyUserHasAction(ActionsEnum.listXs),
|
||||
x.listXs
|
||||
);
|
||||
|
||||
// Get one — row-scoped, read-only, no audit log
|
||||
authenticated.get(
|
||||
"/org/:orgId/x/:xId",
|
||||
verifyValidLicense,
|
||||
verifyOrgAccess,
|
||||
verifyUserHasAction(ActionsEnum.getX),
|
||||
x.getX
|
||||
);
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- HTTP verbs: `PUT` = create, `POST` = update, `GET` = read, `DELETE` =
|
||||
delete. This repo does not use `PATCH` for entity updates (site
|
||||
provisioning keys are the one exception, using `PATCH`).
|
||||
- `verifyValidLicense` is only needed on private/enterprise routes; public
|
||||
OSS routes skip it.
|
||||
- Use `verifyValidSubscription(tierMatrix.someFeature)` right after
|
||||
`verifyValidLicense` when a feature is gated to specific SaaS tiers (see
|
||||
`tierMatrix` usages in `server/private/routers/external.ts`).
|
||||
- `verifyLimits` goes on create routes for entities that count against a
|
||||
plan/seat limit.
|
||||
- For entities keyed by a globally-unique id (not nested under `/org/:orgId`),
|
||||
use the dedicated `verify<Entity>Access` middleware from §4 instead of
|
||||
`verifyOrgAccess` on the row-scoped routes (see how `/ai-provider/:providerId`
|
||||
uses `verifyAiProviderAccess`, while `/org/:orgId/ai-provider` create/list
|
||||
use plain `verifyOrgAccess`).
|
||||
- Read-only routes (`get`, `list`) skip `logActionAudit` — only mutations are
|
||||
audited.
|
||||
- `internal*.ts` routes are for trusted internal callers (gerbil/badger
|
||||
sidecars) and generally skip user-facing auth entirely, using
|
||||
`verifySessionUserMiddleware` / `verifyUserFromResourceSessionMiddleware`
|
||||
instead of `verifyOrgAccess`/`verifyUserHasAction`. CRUD entities almost
|
||||
never need internal router entries — only add one if a sidecar process
|
||||
needs direct access to the resource.
|
||||
|
||||
## 7. The `#dynamic` alias (advanced — most CRUD work can ignore this)
|
||||
|
||||
Some middleware (e.g. `logActionAudit`) needs a real implementation in the
|
||||
enterprise/SaaS build but a no-op stub in the open-source build, while
|
||||
being imported by identical code in `server/routers/external.ts` in both
|
||||
builds. That's done via the `#dynamic/*` import alias, which
|
||||
`tsconfig.oss.json` points at `./server/*` and `tsconfig.enterprise.json` /
|
||||
`tsconfig.saas.json` point at `./server/private/*`. You only need this
|
||||
pattern if you're adding a genuinely dual-implementation hook; a normal
|
||||
private-only CRUD entity (like `alertRule`) never touches `#dynamic` — it
|
||||
just lives entirely under `server/private/` and is imported with `#private/*`
|
||||
directly from `server/private/routers/external.ts`.
|
||||
|
||||
## 8. Checklist for a new entity
|
||||
|
||||
1. Add the DB table to `server/db/pg/schema/schema.ts` (and sqlite schema if
|
||||
applicable).
|
||||
2. Add `ActionsEnum` entries in `server/auth/actions.ts`.
|
||||
3. Create `server/routers/<entity>/` (or `server/private/routers/<entity>/`):
|
||||
`types.ts`, optional `validation.ts`, one file per operation, `index.ts`
|
||||
barrel.
|
||||
4. If routes are row-scoped by a global id, add
|
||||
`verify<Entity>Access.ts` to `server/middlewares/` or
|
||||
`server/private/middlewares/`, and export it from that directory's
|
||||
`index.ts`.
|
||||
5. Wire routes into `external.ts` (public or private) following the verb/
|
||||
middleware table in §6. Add to `internal.ts` only if a sidecar needs
|
||||
direct access.
|
||||
6. Add license header block to every new file if it's under `server/private/`.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user