fix(server): gracefully shut down API server

Description of the Change

Problem

The API HTTP server was started in a goroutine but was not explicitly shut down before telemetry and shared database resources were cleaned up. An unexpected server error used a fatal exit path that could skip deferred cleanup.

Solution

• Keep the HTTP server available to the shutdown path and stop it with a fresh, bounded 15-second context.
• Force-close remaining connections if graceful shutdown fails or times out, and report shutdown errors.
• Wait for the serve goroutine so a real server error racing with a termination signal is not lost.
• Drain telemetry on a fresh 5-second context after HTTP shutdown.
• Return an exit code so deferred cleanup can run before the process exits.
• Add tests for in-flight requests, timeout/forced close, unexpected server errors, telemetry ordering, and non-zero exit behavior.

Closes #480

Type of Change

• 🐛 Bug fix (non-breaking change which fixes an issue)
• 🧪 Test update

Areas Affected

• Core Services (Frontend UI/Backend API)
• Monitoring Stack (Grafana/OpenTelemetry)

Testing and Verification

Test Configuration

PentAGI Version: PR head 3fcf33f
Docker Version: N/A (unit tests only)
Host OS: Linux (CI runner ubuntu-latest)
LLM Provider: Not applicable to the shutdown tests
Enabled Features: HTTP server shutdown and telemetry-drain paths

Test Steps

1. Run go test -race ./cmd/pentagi/ from the backend directory.
2. Run go vet ./cmd/pentagi/ from the backend directory.
3. Review the PR CI checks after GitHub Actions completes.

Test Results

All 12 tests in backend/cmd/pentagi/main_test.go pass with -race enabled. The test suite covers: graceful shutdown with in-flight requests, deadline expiry with forced close, ErrServerClosed after requested vs. unrequested shutdown, server error racing a signal, unexpected failure cleanup ordering, telemetry drain ordering and failure tolerance, listen failure on a real server, and non-zero exit status via a helper process.

Security Considerations

No authentication, authorization, or network exposure changes are intended. Shutdown uses bounded contexts to avoid hanging indefinitely.

Performance Impact

No steady-state request-path changes are intended. Shutdown can wait up to 15 seconds for active HTTP requests before forcing close.

Documentation Updates

No user-facing API or configuration changes are intended.

Deployment Notes

No new environment variables, migrations, or deployment steps are required.

Checklist

Code Quality

• My code follows the project's coding standards
• I have added/updated necessary documentation
• I have added tests to cover my changes
• All new and existing tests pass
• I have run go fmt and go vet (for Go code)
• I have run pnpm run lint (for TypeScript/JavaScript code)

Security

• I have considered security implications
• Changes maintain or improve the security model
• Sensitive information has been properly handled

Compatibility

• Changes are backward compatible
• Breaking changes are clearly marked and documented
• Dependencies are properly updated

Documentation

• Documentation is clear and complete
• Comments are added for non-obvious code
• API changes are documented

Additional Notes

• This PR only modifies Go backend files (backend/cmd/pentagi/main.go and backend/cmd/pentagi/main_test.go). No frontend or TypeScript changes are involved; the pnpm run lint checkbox is checked as N/A.
• The code follows the project's coding standards per CLAUDE.md: comments carry rationale not restated from code, test names follow Test<Component>_<Unit>_<Behaviour>, and tests use blocking select with timeout guards rather than timing-dependent assertions.