GCP Deployment Deep Dive
Complete guide to deploying Bedsheet AI agents on Google Cloud Platform - architecture, authentication, templates, troubleshooting, and battle-tested lessons from production.
๐๏ธ Architecture Overview
High-Level System Architecture
Bedsheet's GCP deployment target creates a complete serverless infrastructure using Cloud Run, Cloud Build, and Vertex AI. The generated artifacts include Terraform for infrastructure and a Makefile for developer experience.
Component Responsibilities
| Component | Responsibility |
|---|---|
bedsheet CLI |
Generates deployment artifacts from templates |
Jinja2 Templates |
Define infrastructure-as-code and runtime configuration |
Cloud Build |
CI/CD pipeline for building and deploying containers |
Artifact Registry |
Docker image storage |
Cloud Run |
Serverless container hosting with auto-scaling |
ADK Web Server |
Google's Agent Development Kit runtime with Dev UI |
Vertex AI |
Gemini model API (global endpoint for Gemini 3) |
Multi-Agent Architecture
Bedsheet supports multi-agent systems with a Supervisor pattern. The Investment Advisor example demonstrates this with three specialized collaborators:
Supervisor] MA[MarketAnalyst] NR[NewsResearcher] RA[RiskAnalyst] end USER[User] -->|HTTP| ROOT ROOT -->|delegate| MA ROOT -->|delegate| NR ROOT -->|delegate| RA MA --> TOOLS1[get_stock_data
get_technical_analysis] NR --> TOOLS2[search_news
analyze_sentiment] RA --> TOOLS3[analyze_volatility
get_position_recommendation]
๐ Authentication Deep Dive
The Authentication Stack
The Google Cloud SDK and Python libraries check credentials in a specific priority order. Understanding this order is crucial for debugging.
(Environment Variable)"] P2["2๏ธโฃ Application Default Credentials
(ADC from gcloud auth)"] P3["3๏ธโฃ Metadata Server
(GCE/Cloud Run SA)"] end P1 --> CHECK1{Set?} CHECK1 -->|Yes| USE1[Use Service Account from file] CHECK1 -->|No| P2 P2 --> CHECK2{Exists?} CHECK2 -->|Yes| USE2[Use ADC credentials] CHECK2 -->|No| P3 P3 --> CHECK3{On GCP?} CHECK3 -->|Yes| USE3[Use instance SA] CHECK3 -->|No| FAIL[No credentials!] style P1 fill:#ff6b6b,color:white style USE1 fill:#ff6b6b,color:white
The January 2026 Bug
This diagram explains exactly what happened during our debugging session:
Authentication Checklist
Before deploying or debugging, always verify:
# 1. Check if GOOGLE_APPLICATION_CREDENTIALS is set
echo $GOOGLE_APPLICATION_CREDENTIALS
# If set to wrong project's SA, unset it:
unset GOOGLE_APPLICATION_CREDENTIALS
# 2. Check current gcloud auth
gcloud auth list
# 3. Check ADC configuration
cat ~/.config/gcloud/application_default_credentials.json | jq '.quota_project_id'
# 4. Test direct API access (bypasses Python SDK)
curl -s -X POST \
"https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT/locations/global/publishers/google/models/gemini-3-flash-preview:generateContent" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
-d '{"contents":[{"role":"user","parts":[{"text":"Hi"}]}]}'
curl works but Python SDK fails, you likely have a
GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to the wrong
service account.
๐ค The ADK Integration
What is ADK?
Agent Development Kit (ADK) is Google's framework for building AI agents. Bedsheet generates ADK-compatible agents for GCP deployment, leveraging ADK's runtime and development tools.
ADK Server Modes
| Mode | Command | Features | Use Case |
|---|---|---|---|
api_server |
adk api_server |
/run, /run_sse, /apps/* | API-only production |
web โ
|
adk web |
All API + /dev-ui/ | Development & debugging |
web mode
We chose web mode for all deployments because:
- Includes all API endpoints from
api_server - Adds interactive Dev UI at
/dev-ui/ - Enables trace visualization for debugging
- No performance penalty
ADK Directory Structure
deploy/gcp/
โโโ agent/ # ADK agent directory
โ โโโ __init__.py # Exports root_agent
โ โโโ agent.py # Agent definition
โ โโโ tools.py # Tool implementations
โโโ Dockerfile # Container definition
โโโ pyproject.toml # Dependencies
โโโ Makefile # Deployment commands
โโโ cloudbuild.yaml # CI/CD pipeline
โโโ terraform/ # Infrastructure as code
โโโ main.tf
โโโ variables.tf
โโโ outputs.tf
The root_agent Pattern
ADK discovers agents by looking for root_agent in the agent module:
# agent/__init__.py
from .agent import root_agent
# agent/agent.py
from google.adk.agents import LlmAgent
root_agent = LlmAgent(
name="InvestmentAdvisor",
model="gemini-3-flash-preview",
instruction="You are an investment advisor...",
sub_agents=[market_analyst, news_researcher, risk_analyst]
)
๐ Template System
How Templates Work
Key Template: Dockerfile.j2
# {{ config.name }} - Cloud Run Container
# Generated by: bedsheet generate --target gcp
FROM python:3.11-slim
# Install uv (fast Python package manager)
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
# Install dependencies
COPY pyproject.toml .
RUN uv pip install --system -r pyproject.toml
# Copy agent code
COPY agent/ ./agent/
# Cloud Run expects PORT env var
ENV PORT=8080
# ADK serves the agent with Dev UI
CMD ["python", "-m", "google.adk.cli", "web", "--host", "0.0.0.0", "--port", "8080", "."]
Template Variables
| Variable | Source | Example |
|---|---|---|
config.name |
bedsheet.yaml | investment-advisor |
gcp.project |
bedsheet.yaml targets.gcp | my-gcp-project |
gcp.region |
bedsheet.yaml targets.gcp | europe-west1 |
gcp.model |
bedsheet.yaml targets.gcp | gemini-3-flash-preview |
๐ Deployment Flow
Complete Sequence
Make Targets
# Quick reference
make help # Show all commands
make setup # One-time setup (auth, project, APIs)
make deploy # Deploy to Cloud Run via Cloud Build
make dev # Run locally with ADK Dev UI
make logs # Stream Cloud Run logs
make url # Get service URL
make destroy # Remove all resources
๐ง Troubleshooting Guide
Decision Tree
Common Issues
1. 403 PERMISSION_DENIED on Vertex AI
# Quick fix
unset GOOGLE_APPLICATION_CREDENTIALS
gcloud auth application-default login --scopes="https://www.googleapis.com/auth/cloud-platform"
gcloud auth application-default set-quota-project YOUR_PROJECT
2. Gemini 3 Model Not Found
# Gemini 3 requires global endpoint
export GOOGLE_CLOUD_LOCATION=global
# Check SDK version (need >= 1.51.0)
pip show google-genai | grep Version
3. ADK Dev UI Returns 404
# Wrong - api_server has no UI
CMD ["python", "-m", "google.adk.cli", "api_server", ...]
# Correct - web mode includes Dev UI
CMD ["python", "-m", "google.adk.cli", "web", ...]
๐ The Great Debugging of January 2026
This is the story of how we spent hours debugging a 403 error that turned out to be a single environment variable. It's now documented here so no one else has to suffer.
The Mystery
| Test | Result |
|---|---|
curl with gcloud auth print-access-token |
โ Worked |
Python SDK with project other-project-id |
โ Worked |
Python SDK with project my-gcp-project |
โ 403 Error |
| Both projects had same APIs enabled | โ Verified |
| User has Owner role on both | โ Verified |
The Investigation Timeline
Investment Advisor returns 403 PERMISSION_DENIED when deployed to my-gcp-project
Checked - both projects have aiplatform.googleapis.com enabled โ
User has Owner role, includes all Vertex AI permissions โ
Tried curl directly - it worked! So the API is accessible โ
curl works, SDK fails โ Different authentication paths!
echo $GOOGLE_APPLICATION_CREDENTIALSOutput:
/path/to/other-project-service-account.json
GOOGLE_APPLICATION_CREDENTIALS pointed to another project's service account!
SDK used other-project SA โ No access to bedsheet project โ 403
unset GOOGLE_APPLICATION_CREDENTIALSNow SDK uses ADC โ Correct project โ Success!
GOOGLE_APPLICATION_CREDENTIALS takes priority over everything else.
๐ Lessons Learned
-
Check environment variables FIRST when debugging auth issues.
GOOGLE_APPLICATION_CREDENTIALSoverrides ADC. - Credential priority matters: ENV > ADC > Metadata Server. Know the order.
- curl working but SDK failing = different auth sources. This is the key diagnostic.
- Trust your instincts: "It works on another project" means the API works - the issue is auth.
- Multiple GCP projects require careful credential management. Service accounts are project-specific.
โก Quick Reference
Deploy a New Agent to GCP
# 1. Create agent
bedsheet init my-agent && cd my-agent
# 2. Configure bedsheet.yaml
# Set your GCP project, region, model
# 3. Generate deployment artifacts
bedsheet generate --target gcp
# 4. Deploy
cd deploy/gcp
make setup # One-time
make deploy # Deploy to Cloud Run
# 5. Access Dev UI
open $(make url)/dev-ui/
Environment Variables
| Variable | Purpose | Required |
|---|---|---|
GOOGLE_CLOUD_PROJECT |
Target GCP project | Yes |
GOOGLE_CLOUD_LOCATION |
Model location (global for Gemini 3) |
Yes |
GOOGLE_GENAI_USE_VERTEXAI |
Use Vertex AI (not AI Studio) | Yes (True) |
GOOGLE_APPLICATION_CREDENTIALS |
Service account key path | No (use ADC) |
๐ฎ Future Considerations
Potential Improvements
1. Credential Validation in Makefile
_validate_credentials:
@if [ -n "$$GOOGLE_APPLICATION_CREDENTIALS" ]; then \
echo "โ ๏ธ WARNING: GOOGLE_APPLICATION_CREDENTIALS is set"; \
echo " Value: $$GOOGLE_APPLICATION_CREDENTIALS"; \
echo " This may cause auth issues with different projects."; \
fi
2. Project-Specific Service Account
resource "google_service_account" "agent_sa" {
account_id = "${var.service_name}-sa"
display_name = "Service Account for ${var.service_name}"
project = var.project_id
}
resource "google_project_iam_member" "vertex_ai_user" {
project = var.project_id
role = "roles/aiplatform.user"
member = "serviceAccount:${google_service_account.agent_sa.email}"
}
Roadmap
| Version | Feature | Status |
|---|---|---|
| v0.4.2 | GCP E2E Testing | โ Done |
| v0.4.2 | ADK Dev UI in Cloud Run | โ Done |
| v0.5.0 | Knowledge Bases / RAG | Planned |
| v0.7.0 | Agent Engine (managed) | Planned |