ci: add secrets scanning, staticcheck, gosec, coverage, env docs, compose/nginx validation, a11y
CI / Nginx config check (push) Failing after 7s
CI / Docker compose check (push) Failing after 7s
CI / Secrets scan (push) Failing after 7s
CI / Env docs check (push) Failing after 8s
CI / Frontend deps check (push) Failing after 23s
CI / Knip (push) Has been skipped
CI / Frontend a11y check (push) Has been skipped
CI / Go build (push) Successful in 37s
CI / Go vulnerabilities (push) Successful in 37s
CI / Frontend build (push) Successful in 1m1s
CI / go mod tidy (push) Successful in 24s
CI / Svelte strict check (push) Has been skipped
CI / Frontend QC (audit) (push) Has been skipped
CI / Frontend QC (typecheck) (push) Has been skipped
CI / Frontend QC (lint) (push) Has been skipped
CI / Go vet (push) Successful in 1m31s
CI / Staticcheck (push) Failing after 1m50s
CI / golangci-lint (push) Successful in 2m26s
CI / Security scan (gosec) (push) Failing after 2m38s
CI / Tests (prod) (push) Has been skipped
CI / Tests (dev) (push) Has been skipped
CI / Race (prod) (push) Has been skipped
CI / Race (dev) (push) Has been skipped
CI / Nginx config check (push) Failing after 7s
CI / Docker compose check (push) Failing after 7s
CI / Secrets scan (push) Failing after 7s
CI / Env docs check (push) Failing after 8s
CI / Frontend deps check (push) Failing after 23s
CI / Knip (push) Has been skipped
CI / Frontend a11y check (push) Has been skipped
CI / Go build (push) Successful in 37s
CI / Go vulnerabilities (push) Successful in 37s
CI / Frontend build (push) Successful in 1m1s
CI / go mod tidy (push) Successful in 24s
CI / Svelte strict check (push) Has been skipped
CI / Frontend QC (audit) (push) Has been skipped
CI / Frontend QC (typecheck) (push) Has been skipped
CI / Frontend QC (lint) (push) Has been skipped
CI / Go vet (push) Successful in 1m31s
CI / Staticcheck (push) Failing after 1m50s
CI / golangci-lint (push) Successful in 2m26s
CI / Security scan (gosec) (push) Failing after 2m38s
CI / Tests (prod) (push) Has been skipped
CI / Tests (dev) (push) Has been skipped
CI / Race (prod) (push) Has been skipped
CI / Race (dev) (push) Has been skipped
New jobs in pipeline: secrets-scan: gitleaks detection go-staticcheck: static analysis (complement to golangci-lint) go-gosec: Go security linter test: coverage profiling with 50% threshold gate env-docs-check: verifies all env vars are documented in .env.example docker-compose-check: validates compose.yml syntax nginx-check: validates nginx config frontend-a11y: Svelte a11y accessibility checks Also: remove orphaned Makefile, update .env.example with 11 missing vars, create .gitleaks.toml with allowlist, add check-env-docs.py script. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
+180
-3
@@ -17,6 +17,38 @@ env:
|
||||
POSTGRES_DB: mydb
|
||||
|
||||
jobs:
|
||||
secrets-scan:
|
||||
name: Secrets scan
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Fix node toolcache path for Post-step cleanup
|
||||
run: |
|
||||
mkdir -p /opt/hostedtoolcache/node/22.23.1/x64/bin
|
||||
ln -sf /usr/local/bin/node /opt/hostedtoolcache/node/22.23.1/x64/bin/node
|
||||
|
||||
- name: Install gitleaks
|
||||
run: go install github.com/gitleaks/gitleaks/v8@latest
|
||||
|
||||
- name: Detect secrets
|
||||
run: gitleaks detect --source . --verbose --no-banner
|
||||
|
||||
env-docs-check:
|
||||
name: Env docs check
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Check env var documentation
|
||||
run: python3 scripts/check-env-docs.py
|
||||
|
||||
go-build:
|
||||
name: Go build
|
||||
runs-on: ubuntu-latest
|
||||
@@ -124,6 +156,78 @@ jobs:
|
||||
golangci-lint run ./...
|
||||
working-directory: backend
|
||||
|
||||
go-staticcheck:
|
||||
name: Staticcheck
|
||||
needs: [go-build]
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: "1.26"
|
||||
cache: false
|
||||
|
||||
- name: Cache Go modules
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/go/pkg/mod
|
||||
~/.cache/go-build
|
||||
key: ${{ runner.os }}-go-${{ hashFiles('backend/go.sum') }}-staticcheck
|
||||
restore-keys: |
|
||||
${{ runner.os }}-go-
|
||||
|
||||
- name: Fix node toolcache path for Post-step cleanup
|
||||
run: |
|
||||
mkdir -p /opt/hostedtoolcache/node/22.23.1/x64/bin
|
||||
ln -sf /usr/local/bin/node /opt/hostedtoolcache/node/22.23.1/x64/bin/node
|
||||
|
||||
- name: Staticcheck
|
||||
run: |
|
||||
go install honnef.co/go/tools/cmd/staticcheck@latest
|
||||
staticcheck -tags "test,dev" ./...
|
||||
working-directory: backend
|
||||
|
||||
go-gosec:
|
||||
name: Security scan (gosec)
|
||||
needs: [go-build]
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: "1.26"
|
||||
cache: false
|
||||
|
||||
- name: Cache Go modules
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/go/pkg/mod
|
||||
~/.cache/go-build
|
||||
key: ${{ runner.os }}-go-${{ hashFiles('backend/go.sum') }}-gosec
|
||||
restore-keys: |
|
||||
${{ runner.os }}-go-
|
||||
|
||||
- name: Fix node toolcache path for Post-step cleanup
|
||||
run: |
|
||||
mkdir -p /opt/hostedtoolcache/node/22.23.1/x64/bin
|
||||
ln -sf /usr/local/bin/node /opt/hostedtoolcache/node/22.23.1/x64/bin/node
|
||||
|
||||
- name: gosec
|
||||
run: |
|
||||
go install github.com/securego/gosec/v2/cmd/gosec@latest
|
||||
gosec -severity medium -tags "test,dev" ./...
|
||||
working-directory: backend
|
||||
|
||||
go-mod-tidy:
|
||||
name: go mod tidy
|
||||
needs: [go-build]
|
||||
@@ -165,7 +269,7 @@ jobs:
|
||||
|
||||
test:
|
||||
name: Tests (${{ matrix.label }})
|
||||
needs: [go-vet, go-lint, go-mod-tidy, vulns]
|
||||
needs: [go-vet, go-lint, go-staticcheck, go-gosec, go-mod-tidy, vulns]
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
@@ -188,9 +292,11 @@ jobs:
|
||||
- label: dev
|
||||
gotags: test,dev
|
||||
verbose: -v
|
||||
coverflags: -coverprofile=coverage.out -covermode=atomic
|
||||
- label: prod
|
||||
gotags: test,!dev
|
||||
verbose: ""
|
||||
coverflags: ""
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -230,14 +336,29 @@ jobs:
|
||||
|
||||
- name: Run tests
|
||||
working-directory: backend
|
||||
run: go test -tags "${{ matrix.gotags }}" -count=1 ${{ matrix.verbose }} ./...
|
||||
run: go test -tags "${{ matrix.gotags }}" -count=1 ${{ matrix.verbose }} ${{ matrix.coverflags }} ./...
|
||||
env:
|
||||
POSTGRES_HOST: postgres
|
||||
TEST_DB_HOST: postgres
|
||||
|
||||
- name: Check coverage threshold
|
||||
working-directory: backend
|
||||
run: |
|
||||
if [ -f coverage.out ]; then
|
||||
COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')
|
||||
echo "Total coverage: $COVERAGE%"
|
||||
if [ "$(echo "$COVERAGE < 50" | bc)" -eq 1 ]; then
|
||||
echo "FAIL: Coverage $COVERAGE% is below 50% threshold"
|
||||
exit 1
|
||||
fi
|
||||
echo "PASS: Coverage $COVERAGE% meets 50% threshold"
|
||||
else
|
||||
echo "No coverage file generated (prod build)"
|
||||
fi
|
||||
|
||||
race:
|
||||
name: Race (${{ matrix.label }})
|
||||
needs: [go-vet, go-lint, go-mod-tidy, vulns]
|
||||
needs: [go-vet, go-lint, go-staticcheck, go-gosec, go-mod-tidy, vulns]
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
@@ -461,6 +582,37 @@ jobs:
|
||||
- name: Run knip
|
||||
run: cd frontend && npx knip
|
||||
|
||||
frontend-a11y:
|
||||
name: Frontend a11y check
|
||||
needs: [frontend-deps]
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Fix node toolcache path for Post-step cleanup
|
||||
run: |
|
||||
mkdir -p /opt/hostedtoolcache/node/22.23.1/x64/bin
|
||||
ln -sf /usr/local/bin/node /opt/hostedtoolcache/node/22.23.1/x64/bin/node
|
||||
|
||||
- name: Restore npm dependencies
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.npm
|
||||
frontend/node_modules
|
||||
key: ${{ runner.os }}-npm-${{ hashFiles('frontend/package-lock.json') }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-npm-
|
||||
|
||||
- name: Install dependencies
|
||||
run: cd frontend && npm ci
|
||||
|
||||
- name: Check a11y
|
||||
run: cd frontend && npm run lint:a11y
|
||||
|
||||
frontend-build:
|
||||
name: Frontend build
|
||||
runs-on: ubuntu-latest
|
||||
@@ -564,3 +716,28 @@ jobs:
|
||||
|
||||
- name: Run QC task
|
||||
run: cd frontend && ${{ matrix.cmd }}
|
||||
|
||||
docker-compose-check:
|
||||
name: Docker compose check
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Validate compose.yml
|
||||
run: docker compose -f compose.yml config --quiet
|
||||
|
||||
nginx-check:
|
||||
name: Nginx config check
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
shell: sh
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Validate nginx config
|
||||
run: |
|
||||
docker run --rm -v $(pwd)/nginx/conf.d/default.conf:/etc/nginx/conf.d/default.conf:ro nginx:stable-alpine nginx -t 2>&1
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# Gitleaks configuration for Crussell
|
||||
title = "Crussell secrets scanning"
|
||||
|
||||
[extend]
|
||||
# Use the default gitleaks rules
|
||||
useDefault = true
|
||||
|
||||
# Allowlist paths and regexes that are known safe
|
||||
[allowlist]
|
||||
description = "Known false positives"
|
||||
paths = [
|
||||
# Test fixtures with mock data
|
||||
"backend/testutils/",
|
||||
"backend/handlers/.*_test.go",
|
||||
# Example env file with placeholder values
|
||||
".env.example",
|
||||
# SabreDAV dependency files
|
||||
"sabredav/composer.json",
|
||||
# Frontend env examples
|
||||
"frontend/.env.production",
|
||||
]
|
||||
@@ -1,58 +0,0 @@
|
||||
# Pipeline targets for Crussell backend
|
||||
# Use: make <target> (runs with default tags: test,dev)
|
||||
# make TAGS="test,!dev" <target>
|
||||
|
||||
TAGS ?= test,dev
|
||||
RACE_FLAGS ?= -race -timeout 480s
|
||||
|
||||
.PHONY: test test-all test-race vet lint ci help
|
||||
|
||||
# --- Test matrix ---
|
||||
|
||||
test: ## Run tests with default build tags (test,dev)
|
||||
go test -tags "$(TAGS)" -count=1 -parallel 8 ./...
|
||||
|
||||
test-prod: ## Run tests with production build tags (test,!dev)
|
||||
go test -tags "test,!dev" -count=1 -parallel 8 ./...
|
||||
|
||||
test-dev: ## Run tests with dev build tags (test,dev)
|
||||
go test -tags "test,dev" -count=1 -parallel 8 ./...
|
||||
|
||||
test-all: test-dev test-prod ## Run tests under both dev and prod build tags
|
||||
|
||||
test-race: ## Run tests with race detector enabled
|
||||
go test -tags "$(TAGS)" $(RACE_FLAGS) -count=1 ./...
|
||||
|
||||
test-count: ## Run each test 10 times to detect flakiness
|
||||
go test -tags "$(TAGS)" -count=10 -parallel 8 ./...
|
||||
|
||||
# --- Static analysis ---
|
||||
|
||||
vet: ## go vet under current build tags
|
||||
go vet -tags "$(TAGS)" ./...
|
||||
|
||||
vet-all: ## go vet under both dev and prod build tags
|
||||
@echo "=== vet: dev ==="
|
||||
go vet -tags "test,dev" ./...
|
||||
@echo "=== vet: prod ==="
|
||||
go vet -tags "test,!dev" ./...
|
||||
|
||||
lint: ## staticcheck under current build tags
|
||||
staticcheck -tags "$(TAGS)" ./...
|
||||
|
||||
lint-all: ## staticcheck under both build tags
|
||||
@echo "=== staticcheck: dev ==="
|
||||
staticcheck -tags "test,dev" ./...
|
||||
@echo "=== staticcheck: prod ==="
|
||||
staticcheck -tags "test,!dev" ./...
|
||||
|
||||
# --- Combined ---
|
||||
|
||||
ci: test-all vet-all lint-all ## Full CI pipeline: test + vet + lint under both tag sets
|
||||
|
||||
ci-race: test-race vet lint ## CI with race detection
|
||||
|
||||
# --- Utilities ---
|
||||
|
||||
help: ## Show this help
|
||||
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-15s\033[0m %s\n", $$1, $$2}'
|
||||
@@ -0,0 +1,38 @@
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { includeIgnoreFile } from '@eslint/compat';
|
||||
import js from '@eslint/js';
|
||||
import svelte from 'eslint-plugin-svelte';
|
||||
import { defineConfig } from 'eslint/config';
|
||||
import globals from 'globals';
|
||||
import ts from 'typescript-eslint';
|
||||
import svelteConfig from './svelte.config.js';
|
||||
|
||||
const gitignorePath = fileURLToPath(new URL('./.gitignore', import.meta.url));
|
||||
|
||||
export default defineConfig(
|
||||
includeIgnoreFile(gitignorePath),
|
||||
js.configs.recommended,
|
||||
...ts.configs.recommended,
|
||||
...svelte.configs.recommended,
|
||||
{
|
||||
languageOptions: {
|
||||
globals: { ...globals.browser, ...globals.node }
|
||||
}
|
||||
},
|
||||
{
|
||||
files: ['**/*.svelte', '**/*.svelte.ts', '**/*.svelte.js'],
|
||||
languageOptions: {
|
||||
parserOptions: {
|
||||
projectService: true,
|
||||
extraFileExtensions: ['.svelte'],
|
||||
parser: ts.parser,
|
||||
svelteConfig
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
rules: {
|
||||
'svelte/valid-compile': 'error'
|
||||
}
|
||||
}
|
||||
);
|
||||
@@ -14,7 +14,8 @@
|
||||
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json",
|
||||
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
|
||||
"format": "prettier --write .",
|
||||
"lint": "prettier --check . && eslint ."
|
||||
"lint": "prettier --check . && eslint .",
|
||||
"lint:a11y": "eslint 'src/**/*.svelte' --config eslint.a11y.config.js"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/compat": "^1.2.5",
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check that all env vars used in the codebase are documented in .env.example."""
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
def find_env_vars_in_code():
|
||||
"""Find all os.Getenv() and import.meta.env.VITE_* calls in the codebase."""
|
||||
env_vars = set()
|
||||
|
||||
# Search backend Go files for os.Getenv("VAR_NAME")
|
||||
go_dir = os.path.join(REPO_ROOT, 'backend')
|
||||
for root, dirs, files in os.walk(go_dir):
|
||||
dirs[:] = [d for d in dirs if d not in ('vendor', '.git', 'node_modules')]
|
||||
for f in files:
|
||||
if f.endswith('.go'):
|
||||
path = os.path.join(root, f)
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
m = re.findall(r'os\.Getenv\(["\']([A-Z_][A-Z0-9_]*)["\']', line)
|
||||
env_vars.update(m)
|
||||
|
||||
# Search frontend files for import.meta.env.VITE_*
|
||||
frontend_dir = os.path.join(REPO_ROOT, 'frontend')
|
||||
for root, dirs, files in os.walk(frontend_dir):
|
||||
dirs[:] = [d for d in dirs if d not in ('.git', 'node_modules', 'build')]
|
||||
for f in files:
|
||||
if f.endswith(('.ts', '.svelte', '.js')):
|
||||
path = os.path.join(root, f)
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
m = re.findall(r'import\.meta\.env\.([A-Z_][A-Z0-9_]*)', line)
|
||||
env_vars.update(m)
|
||||
|
||||
return sorted(env_vars)
|
||||
|
||||
|
||||
def find_documented_vars():
|
||||
"""Find all env vars documented in .env.example."""
|
||||
env_path = os.path.join(REPO_ROOT, '.env.example')
|
||||
documented = set()
|
||||
if os.path.exists(env_path):
|
||||
with open(env_path) as fh:
|
||||
for line in fh:
|
||||
line = line.strip()
|
||||
if line and not line.startswith('#') and '=' in line:
|
||||
var_name = line.split('=')[0].strip()
|
||||
if var_name:
|
||||
documented.add(var_name)
|
||||
return documented
|
||||
|
||||
|
||||
def main():
|
||||
code_vars = find_env_vars_in_code()
|
||||
documented = find_documented_vars()
|
||||
|
||||
undocumented = [v for v in code_vars if v not in documented]
|
||||
|
||||
if undocumented:
|
||||
print("ERROR: The following env vars are used in code but not documented in .env.example:\n")
|
||||
for v in undocumented:
|
||||
print(f" {v}")
|
||||
print(f"\nTotal: {len(undocumented)} undocumented vars out of {len(code_vars)} total")
|
||||
sys.exit(1)
|
||||
else:
|
||||
print(f"OK: All {len(code_vars)} env vars used in code are documented in .env.example")
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user