Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.pangolin.net/llms.txt

Use this file to discover all available pages before exploring further.

Try free on Pangolin Cloud

Fastest way to get started with Pangolin using the hosted control plane. No credit card required.
This page covers troubleshooting Pangolin Kubernetes deployments across Helm, Kustomize, Argo CD, and Flux workflows. Start with the core checks, then use the section that matches the symptom.

​
Core diagnostics

Set the namespace and release name used by your installation: 
export PANGOLIN_NAMESPACE=pangolin
export PANGOLIN_RELEASE=pangolin

​
Helm diagnostics

Check the release: 
helm status "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE"
helm history "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE"
helm get values "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" --all
Render the chart locally with your values file: 
helm repo update fossorial

helm template "$PANGOLIN_RELEASE" fossorial/pangolin \
  --namespace "$PANGOLIN_NAMESPACE" \
  --values values-pangolin.yaml
Preview an upgrade: 
helm upgrade "$PANGOLIN_RELEASE" fossorial/pangolin \
  --namespace "$PANGOLIN_NAMESPACE" \
  --values values-pangolin.yaml \
  --dry-run
helm lint charts/pangolin is only useful when you are working inside the Helm chart repository. For normal installs, use helm template and helm upgrade --dry-run.

​
Kubernetes diagnostics

Check workloads and events: 
kubectl get pods --namespace "$PANGOLIN_NAMESPACE"
kubectl get deploy,statefulset,job,cronjob --namespace "$PANGOLIN_NAMESPACE"
kubectl get events --namespace "$PANGOLIN_NAMESPACE" --sort-by=.lastTimestamp
Inspect a pod: 
kubectl describe pod <pod-name> --namespace "$PANGOLIN_NAMESPACE"
kubectl logs <pod-name> --namespace "$PANGOLIN_NAMESPACE" --all-containers --tail=200
Check services, PVCs, and policies: 
kubectl get svc,pvc,secret,configmap --namespace "$PANGOLIN_NAMESPACE"
kubectl get networkpolicy --namespace "$PANGOLIN_NAMESPACE"

​
Traefik diagnostics

In controller mode, the chart uses Traefik CRDs such as IngressRoute. Check whether Traefik CRDs are installed: 
kubectl get crd | grep traefik
Check rendered or applied Traefik resources: 
kubectl get ingressroute --namespace "$PANGOLIN_NAMESPACE"
kubectl describe ingressroute <name> --namespace "$PANGOLIN_NAMESPACE"
Depending on your Traefik setup, also check: 
kubectl get middleware,tlsoption,traefikservice --namespace "$PANGOLIN_NAMESPACE"
kubectl get ingress is only useful if your selected deployment mode renders standard Kubernetes Ingress resources. In controller mode, use IngressRoute.

​
Database diagnostics

If you use CloudNativePG, first check that the CRD exists: 
kubectl get crd | grep postgresql.cnpg.io
Then check CNPG resources: 
kubectl get cluster --namespace "$PANGOLIN_NAMESPACE"
kubectl describe cluster <cluster-name> --namespace "$PANGOLIN_NAMESPACE"
kubectl get pods --namespace "$PANGOLIN_NAMESPACE" | grep -E 'pangolin-db|postgres'
kubectl get secret --namespace "$PANGOLIN_NAMESPACE" | grep -E 'pangolin-db|postgres'
If you use external PostgreSQL, verify the connection Secret: 
kubectl get secret <connection-secret-name> --namespace "$PANGOLIN_NAMESPACE"
kubectl describe secret <connection-secret-name> --namespace "$PANGOLIN_NAMESPACE"
Do not decode and paste database credentials into logs, screenshots, or issue reports.

​
Common issues and solutions

Symptoms
  • Gerbil pod crashes during a fresh install.
  • Logs mention missing setup data, missing exit node, or tunnel configuration not being ready.
  • Pangolin itself is not initialized yet.
CauseOn first install, Gerbil may start before Pangolin has completed the initial setup. The chart supports gerbil.startupMode for this case.ResolutionUse delayed startup for the first install:
gerbil:
  startupMode: delayed
Install or upgrade with the values file:
helm upgrade --install "$PANGOLIN_RELEASE" fossorial/pangolin \
  --namespace "$PANGOLIN_NAMESPACE" \
  --values values-pangolin.yaml
After Pangolin setup is complete, switch Gerbil to normal startup:
helm upgrade "$PANGOLIN_RELEASE" fossorial/pangolin \
  --namespace "$PANGOLIN_NAMESPACE" \
  --reuse-values \
  --set gerbil.startupMode=normal
Check Gerbil resources:
kubectl get pods,svc,pvc --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=gerbil
Symptoms
  • Gerbil pod does not start.
  • Events mention Pod Security Admission.
  • Events mention forbidden capabilities.
  • Logs or events mention NET_ADMIN.
CauseGerbil requires the NET_ADMIN Linux capability for WireGuard interface management. A namespace using a restricted Pod Security profile can block this.ResolutionCheck namespace labels:
kubectl get namespace "$PANGOLIN_NAMESPACE" --show-labels
For a namespace running Gerbil, use a policy profile that allows the required capability. Example:
kubectl label namespace "$PANGOLIN_NAMESPACE" \
  pod-security.kubernetes.io/enforce=privileged \
  pod-security.kubernetes.io/warn=baseline \
  pod-security.kubernetes.io/audit=restricted \
  --overwrite
Then restart the affected pods:
kubectl rollout restart deploy --namespace "$PANGOLIN_NAMESPACE"
Do not use a restricted Pod Security profile for Gerbil unless you have validated the selected chart mode and security context. Removing NET_ADMIN breaks WireGuard management.
Symptoms
  • The dashboard URL does not load.
  • Browser shows timeout, bad gateway, 404, or TLS error.
  • API path /api/v1 fails while the dashboard path works, or the reverse.
Common causes
  • DNS points to the wrong load balancer or ingress endpoint.
  • Traefik CRDs are missing.
  • Traefik controller is not watching the namespace or selector labels.
  • IngressRoute host does not match the dashboard URL.
  • API route was changed and no longer matches PathPrefix(/api/v1).
  • TLS resolver or TLS Secret is misconfigured.
ChecksCheck DNS:
nslookup pangolin.example.com
Check Traefik CRDs:
kubectl get crd | grep traefik
Check IngressRoute resources:
kubectl get ingressroute --namespace "$PANGOLIN_NAMESPACE"
kubectl describe ingressroute <name> --namespace "$PANGOLIN_NAMESPACE"
Check the rendered values:
helm get values "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" --all | grep -A30 ingressRoute
Check Traefik logs. Adjust the namespace and label selector to your Traefik installation:
kubectl logs --namespace traefik -l app.kubernetes.io/name=traefik --tail=100
Temporary local check for the dashboard/UI service port:
kubectl port-forward --namespace "$PANGOLIN_NAMESPACE" svc/pangolin 8080:3002
Then open:
http://localhost:8080
The dashboard/UI port is 3002. The API/external port is 3000. Port-forward 3002 when checking the dashboard locally.
Symptoms
  • IngressRoute is created but TLS does not work.
  • Traefik logs mention TLS configuration problems.
  • Certificate is not issued or the TLS Secret is not found.
CauseThe dashboard IngressRoute TLS configuration should use either a Traefik certificate resolver or an existing TLS Secret.ResolutionUse Traefik ACME certificate resolver:
pangolin:
  ingressRoute:
    dashboard:
      tls:
        enabled: true
        certResolver: letsencrypt
        secretName: ""
Or use an existing TLS Secret:
pangolin:
  ingressRoute:
    dashboard:
      tls:
        enabled: true
        certResolver: ""
        secretName: pangolin-dashboard-tls
Verify the Secret if using secretName:
kubectl get secret pangolin-dashboard-tls --namespace "$PANGOLIN_NAMESPACE"
certResolver is a Traefik ACME resolver setting. It is not a cert-manager issuer reference.
Symptoms
  • Newt shows repeated connection or tunnel errors.
  • Tunnel traffic does not pass.
  • WireGuard UDP ports are unreachable from the Newt location.
Common causes
  • pangolin.config.gerbil.base_endpoint points to the wrong host.
  • Gerbil Service is not exposed as expected.
  • External firewall blocks UDP traffic.
  • NetworkPolicy blocks the required traffic.
  • pangolin.config.gerbil.start_port and gerbil.ports.wg1 are not aligned.
  • pangolin.config.gerbil.clients_start_port and gerbil.ports.wg2 are not aligned.
ChecksCheck Gerbil Service:
kubectl get svc --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=gerbil

kubectl describe svc <gerbil-service-name> --namespace "$PANGOLIN_NAMESPACE"
Check Gerbil values:
helm get values "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" --all | grep -A30 gerbil
Check NetworkPolicies:
kubectl get networkpolicy --namespace "$PANGOLIN_NAMESPACE"
kubectl describe networkpolicy --namespace "$PANGOLIN_NAMESPACE"
Verify external firewall rules for the configured UDP ports.
Symptoms
  • Newt peers do not establish stable handshakes.
  • Tunnel traffic drops even though Gerbil pods are healthy.
  • Logs show connection resets or malformed upstream traffic.
CauseProxy protocol handling is inconsistent between the upstream hop and Gerbil.
If Gerbil is exposed through a reverse proxy or UDP gateway, keep proxy protocol settings aligned end-to-end. Do not enable proxy protocol on the upstream hop unless Gerbil is configured to accept it.
ChecksCheck endpoint and port alignment:
helm get values "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" --all | grep -A40 gerbil
Check Gerbil logs:
kubectl logs --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=gerbil \
  --tail=200
Check Service exposure:
kubectl get svc --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=gerbil -o wide
Symptoms
  • Pangolin pod crashes.
  • Logs mention database connection errors.
  • Events mention missing Secret or missing Secret key.
Causedatabase.mode=external needs a valid database connection Secret unless the chart is configured to generate one from values.ResolutionCreate a connection Secret:
kubectl create secret generic pangolin-db-connection \
  --namespace "$PANGOLIN_NAMESPACE" \
  --from-literal=connectionString='postgresql://pangolin:password@postgres.example.com:5432/pangolin?sslmode=require'
Reference it in values:
database:
  mode: external
  connection:
    existingSecretName: pangolin-db-connection
    existingSecretKey: connectionString
Check the Secret:
kubectl describe secret pangolin-db-connection --namespace "$PANGOLIN_NAMESPACE"
Do not put database passwords directly in values files for production. Use an existing Secret or your normal secret-management workflow.
Symptoms
  • CNPG Cluster resource is missing.
  • CNPG pods do not start.
  • Pangolin cannot connect to the generated CNPG database.
  • Secret such as pangolin-db-app is missing.
Common causes
  • CloudNativePG CRDs/operator are not installed.
  • cnpg-cluster.enabled is false when you expected the chart to create a cluster.
  • cnpg-operator.enabled is false and no operator exists.
  • database.cloudnativepg.cluster.name does not match the CNPG cluster name.
  • StorageClass or PVC provisioning fails.
ChecksCheck CRDs:
kubectl get crd | grep postgresql.cnpg.io
Check CNPG operator pods:
kubectl get pods --all-namespaces | grep -i cnpg
Check CNPG Cluster:
kubectl get cluster --namespace "$PANGOLIN_NAMESPACE"
kubectl describe cluster pangolin-db --namespace "$PANGOLIN_NAMESPACE"
Check PVCs and Secrets:
kubectl get pvc --namespace "$PANGOLIN_NAMESPACE"
kubectl get secret --namespace "$PANGOLIN_NAMESPACE" | grep pangolin-db
Expected naming when using the default example:
database:
  cloudnativepg:
    cluster:
      name: pangolin-db

cnpg-cluster:
  enabled: true
  fullnameOverride: pangolin-db
Symptoms
  • DNS lookups fail.
  • Pangolin cannot connect to the database.
  • Controller cannot reach the Kubernetes API.
  • Gerbil or Newt traffic does not work.
  • External services such as SMTP, OIDC, or webhooks time out.
CauseThe chart can render NetworkPolicies. If your CNI enforces them, missing egress or ingress rules can break required paths.Checks
kubectl get networkpolicy --namespace "$PANGOLIN_NAMESPACE"
kubectl describe networkpolicy --namespace "$PANGOLIN_NAMESPACE"
Check whether DNS is allowed:
networkPolicy:
  dns:
    enabled: true
Check database egress:
networkPolicy:
  database:
    enabled: true
    port: 5432
Check controller API access:
networkPolicy:
  controller:
    egress:
      enabled: true
      kubernetesApi:
        enabled: true
        port: 443
For external integrations, add scoped egress rules for the required services instead of allowing broad egress.For a temporary isolation test, disable NetworkPolicy and re-apply:
networkPolicy:
  enabled: false
If this fixes the issue, re-enable policies and add the missing rules.
Symptoms
  • Pangolin pod restarts repeatedly.
  • Pod stays Pending.
  • Readiness never becomes true.
ChecksFind the pod:
kubectl get pods --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=pangolin
Inspect it:
kubectl describe pod <pod-name> --namespace "$PANGOLIN_NAMESPACE"
kubectl logs <pod-name> --namespace "$PANGOLIN_NAMESPACE" --tail=200
kubectl logs <pod-name> --namespace "$PANGOLIN_NAMESPACE" --previous --tail=200
Check PVCs:
kubectl get pvc --namespace "$PANGOLIN_NAMESPACE"
kubectl describe pvc <pvc-name> --namespace "$PANGOLIN_NAMESPACE"
Common causes:
StatusCommon causes
CrashLoopBackOffDatabase connection issue, missing Secret, invalid config, startup dependency not ready
PendingPVC not bound, insufficient resources, node selector/affinity mismatch, Pod Security policy rejection
ImagePullBackOffWrong image override, registry access issue, missing imagePullSecret
Do not assume tools such as psql, curl, or dig are available inside the Pangolin container. Use logs, Events, or a temporary debug pod when needed.
Run a temporary debug pod for network tests:
kubectl run net-debug \
  --namespace "$PANGOLIN_NAMESPACE" \
  --rm -it \
  --image=curlimages/curl:latest \
  --restart=Never \
  -- sh
Symptoms
  • Helm template or install succeeds, but Traefik resources are not reconciled.
  • kubectl get ingressroute fails with unknown resource type.
  • Argo CD or Flux reports missing kind IngressRoute.
CauseController mode expects Traefik CRDs and a Traefik controller. They must be installed separately or through the bundled dependency when enabled.Checks
kubectl get crd | grep traefik
kubectl get pods --all-namespaces | grep -i traefik
If you want the chart to install the bundled Traefik controller, enable it:
deployment:
  type: controller
  installTraefikController: true
If Traefik is already installed elsewhere, keep it disabled and make sure the controller watches the namespace and labels used by the Pangolin IngressRoute.
Symptoms
  • helm upgrade fails.
  • Rendered resources changed unexpectedly.
  • Existing resources conflict with chart-managed resources.
  • GitOps reports immutable field changes or ownership conflicts.
ChecksRender before upgrading:
helm template "$PANGOLIN_RELEASE" fossorial/pangolin \
  --namespace "$PANGOLIN_NAMESPACE" \
  --values values-pangolin.yaml > rendered.yaml
Run a server-side dry run:
kubectl apply -f rendered.yaml --dry-run=server
Compare the current live release:
helm get manifest "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" > live-release.yaml
diff -u live-release.yaml rendered.yaml
Check ownership conflicts:
kubectl get all --namespace "$PANGOLIN_NAMESPACE" -o yaml | grep -E "meta.helm.sh|app.kubernetes.io/managed-by"
Avoid --force unless you understand which resources will be recreated.
helm upgrade --force can delete and recreate resources. That can interrupt traffic and may affect persistent workloads depending on the resource type.
Symptoms
  • Kustomize build succeeds but changes are missing.
  • Patch target does not match any resource.
  • Patch breaks after chart upgrade.
ChecksList generated resource names:
kustomize build base | grep -E "^(kind:|  name:)"
Validate the overlay:
kustomize build overlays/prod
Run a server-side dry run:
kustomize build overlays/prod | kubectl apply -f - --dry-run=server
Preview live changes:
kustomize build overlays/prod | kubectl diff -f -
Use modern Kustomize patches syntax:
patches:
  - path: patches/pangolin-resources.patch.yaml
    target:
      group: apps
      version: v1
      kind: Deployment
      name: pangolin
For Helm-rendered bases, do not assume resource names. Check the rendered manifests after each chart upgrade.
Symptoms
  • Argo CD Application is OutOfSync or Degraded.
  • Flux HelmRelease or Kustomization is not Ready.
  • Resources are missing or constantly reverted.
Argo CD checks
kubectl describe application pangolin --namespace argocd
kubectl logs --namespace argocd deployment/argocd-application-controller --tail=100
argocd app diff pangolin
argocd app sync pangolin
Flux checks
flux get sources all --all-namespaces
flux get helmreleases --all-namespaces
flux get kustomizations --all-namespaces
flux logs --all-namespaces --follow
Reconcile manually:
flux reconcile helmrelease pangolin --namespace "$PANGOLIN_NAMESPACE"
flux reconcile kustomization pangolin --namespace flux-system
Common causes:
  • chart repository or OCI source not reachable
  • wrong chart version
  • missing CRDs
  • invalid values
  • rendered resource ownership conflict
  • Secret not available in the expected namespace

​
Routing issues to the right repository

Use the repository that matches the failing area:
AreaRepository
Chart templates, values, examples, rendered manifestsfosrl/helm-charts
Pangolin runtime, API, UI, auth, application behaviorfosrl/pangolin
Newt client behavior or connectivityfosrl/newt
Documentationfosrl/docs-v2

​
Before opening an issue, collect

Collect this information before opening an issue:
  • chart version
  • Pangolin app version
  • Kubernetes version
  • Helm version
  • deployment method: Helm, Kustomize, Argo CD, or Flux
  • sanitized values file
  • pod logs
  • namespace events
  • Traefik logs, if routing is involved
  • rendered manifests from helm template or kustomize build
  • Helm release status or GitOps sync status
  • reproduction steps
Collect basic diagnostics: 
kubectl version
helm version

helm status "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE"
helm get values "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" --all > pangolin-values.yaml
helm get manifest "$PANGOLIN_RELEASE" --namespace "$PANGOLIN_NAMESPACE" > pangolin-manifest.yaml

kubectl get pods --namespace "$PANGOLIN_NAMESPACE" -o wide > pangolin-pods.txt
kubectl get events --namespace "$PANGOLIN_NAMESPACE" --sort-by=.lastTimestamp > pangolin-events.txt
Before sharing diagnostics, remove:
  • database passwords
  • SERVER_SECRET
  • API keys
  • OAuth/OIDC client secrets
  • TLS private keys
  • internal hostnames, if sensitive

​
Useful command reference

# General cluster info
kubectl cluster-info
kubectl version

# Namespace overview
kubectl get all --namespace "$PANGOLIN_NAMESPACE"
kubectl get pvc,secret,configmap --namespace "$PANGOLIN_NAMESPACE"
kubectl get events --namespace "$PANGOLIN_NAMESPACE" --sort-by=.lastTimestamp

# Logs
kubectl logs --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=pangolin \
  --tail=200

kubectl logs --namespace "$PANGOLIN_NAMESPACE" \
  -l app.kubernetes.io/name=gerbil \
  --tail=200

# Dashboard local test
kubectl port-forward --namespace "$PANGOLIN_NAMESPACE" svc/pangolin 8080:3002

# Traefik resources
kubectl get ingressroute --namespace "$PANGOLIN_NAMESPACE"

# Resource usage
kubectl top pod --namespace "$PANGOLIN_NAMESPACE"
kubectl top node

​
Next steps

Pangolin Configuration

Review Pangolin chart options.

Helm Quick-Start

Install Pangolin with Helm.

Kustomize Quick-Start

Install Pangolin with rendered manifests and Kustomize overlays.

GitOps Overview

Deploy Pangolin with Argo CD or Flux.