GCP Discovery

The Infracast GCP plugin uses Google Cloud APIs and the Cloud Asset Inventory to enumerate resources across your projects. It authenticates using a service account with read-only IAM roles.

How It Works

1. Infracast authenticates using a service account key or Workload Identity Federation
2. The Cloud Asset Inventory API provides a comprehensive snapshot of all resources
3. Individual service APIs are queried for resource-specific details
4. Results are normalized and written to the Infracast graph

tip

The Cloud Asset Inventory API is the preferred discovery mechanism — it returns all resources across all services in a single call, rather than requiring per-service API calls. Infracast uses this by default.

Service Account Setup

Step 1: Create the Service Account

# Set your project ID
export PROJECT_ID="your-project-id"

# Create the service account
gcloud iam service-accounts create infracast-discovery \
  --display-name "Infracast Discovery" \
  --description "Service account for Infracast infrastructure discovery" \
  --project "$PROJECT_ID"

# Note the service account email
export SA_EMAIL="infracast-discovery@${PROJECT_ID}.iam.gserviceaccount.com"
echo "Service account: $SA_EMAIL"

Step 2: Assign IAM Roles

Infracast requires the following roles on each project (or at the organization level for multi-project discovery):

# Assign roles at the project level
for ROLE in \
  "roles/viewer" \
  "roles/cloudasset.viewer" \
  "roles/iam.securityReviewer"; do
  gcloud projects add-iam-policy-binding "$PROJECT_ID" \
    --member "serviceAccount:$SA_EMAIL" \
    --role "$ROLE"
done

Required IAM roles:

Role	Purpose
roles/viewer	Read all project resources (Primitive role)
roles/cloudasset.viewer	Access Cloud Asset Inventory
roles/iam.securityReviewer	Read IAM policies and service accounts
roles/container.clusterViewer	Read GKE cluster details (optional)
roles/cloudsql.viewer	Read Cloud SQL instances (optional, covered by viewer)

warning

roles/viewer is a primitive role that grants broad read access. If your security policy prohibits primitive roles, you can replace it with a combination of predefined roles per service. See the Least Privilege Alternative section.

Step 3: Generate a Service Account Key

# Create and download the key
gcloud iam service-accounts keys create infracast-discovery-key.json \
  --iam-account "$SA_EMAIL" \
  --project "$PROJECT_ID"

# Restrict file permissions
chmod 600 infracast-discovery-key.json

warning

Store the service account key securely. Do not commit it to source control. Consider using Workload Identity Federation instead of key files for production deployments.

Least Privilege Alternative

Instead of roles/viewer, use these predefined roles for a narrower permission set:

ROLES=(
  "roles/cloudasset.viewer"
  "roles/compute.viewer"
  "roles/container.clusterViewer"
  "roles/cloudsql.viewer"
  "roles/storage.objectViewer"
  "roles/run.viewer"
  "roles/cloudfunctions.viewer"
  "roles/bigquery.metadataViewer"
  "roles/pubsub.viewer"
  "roles/iam.securityReviewer"
  "roles/dns.reader"
  "roles/monitoring.viewer"
  "roles/logging.viewer"
)

for ROLE in "${ROLES[@]}"; do
  gcloud projects add-iam-policy-binding "$PROJECT_ID" \
    --member "serviceAccount:$SA_EMAIL" \
    --role "$ROLE"
done

Workload Identity Federation

For production deployments, use Workload Identity Federation instead of service account key files:

# Create a Workload Identity Pool
gcloud iam workload-identity-pools create "infracast-pool" \
  --location "global" \
  --display-name "Infracast Workload Identity Pool"

# Create a provider for AWS (if Infracast runs on AWS)
gcloud iam workload-identity-pools providers create-aws "infracast-aws-provider" \
  --location "global" \
  --workload-identity-pool "infracast-pool" \
  --account-id "INFRACAST_AWS_ACCOUNT_ID"

# Bind the service account
gcloud iam service-accounts add-iam-policy-binding "$SA_EMAIL" \
  --role "roles/iam.workloadIdentityUser" \
  --member "principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/infracast-pool/attribute.aws_role/arn:aws:sts::INFRACAST_AWS_ACCOUNT_ID:assumed-role/infracast-collector"

What Gets Discovered

Service	Resource Types
Compute Engine	VM instances, Instance templates, Instance groups, Disks, Snapshots, Images, Machine types
VPC	Networks, Subnets, Firewall rules, Routes, Cloud NAT, VPN tunnels, Interconnects, Peering
GKE	Clusters, Node pools, Namespaces (via k8s API)
Cloud SQL	Instances, Databases, Users (metadata), Backups
Cloud Storage	Buckets, Bucket IAM policies, Public access settings
Cloud Run	Services, Revisions
Cloud Functions	Functions (v1 and v2), Triggers
BigQuery	Datasets, Tables (metadata), Views
Pub/Sub	Topics, Subscriptions
IAM	Service accounts, IAM policies, Roles (custom), Workload Identity Pools
DNS	Managed zones, DNS records
KMS	Key rings, Keys, Key versions
Secret Manager	Secrets (metadata only, no values)
Artifact Registry	Repositories, Images (metadata)
Organization	Organization, Folders, Projects, IAM policies at each level

Registering the Credential in Infracast

# Register using service account key file
infracast creds add \
  --plugin gcp \
  --name "my-gcp-project" \
  --type service-account-key \
  --key-file /run/secrets/infracast-discovery-key.json

# Register using Workload Identity Federation
infracast creds add \
  --plugin gcp \
  --name "my-gcp-wif" \
  --type workload-identity \
  --credential-config-file /run/secrets/gcp-wif-config.json

Configuring the Discovery Job

infracast.yaml

discovery:
  jobs:
    - name: gcp-myproject
      plugin: gcp
      credential: my-gcp-project
      schedule: "0 */4 * * *"
      config:
        project_id: "your-project-id"
        # Use Cloud Asset Inventory for efficient bulk discovery (recommended)
        use_asset_inventory: true
        # Optional: restrict to specific asset types
        asset_types:
          - "compute.googleapis.com/Instance"
          - "compute.googleapis.com/Network"
          - "storage.googleapis.com/Bucket"
          - "iam.googleapis.com/ServiceAccount"
        # Optional: include Cloud Run and Functions
        discover_serverless: true

Multi-Project Setup

Option 1: Organization-Level Discovery

Grant the service account roles at the organization level to discover all projects automatically:

# Get your organization ID
gcloud organizations list

export ORG_ID="your-org-id"

# Assign roles at org level
for ROLE in \
  "roles/viewer" \
  "roles/cloudasset.viewer" \
  "roles/iam.securityReviewer"; do
  gcloud organizations add-iam-policy-binding "$ORG_ID" \
    --member "serviceAccount:$SA_EMAIL" \
    --role "$ROLE"
done

discovery:
  jobs:
    - name: gcp-org-discovery
      plugin: gcp
      credential: my-gcp-project
      schedule: "0 2 * * *"
      config:
        mode: organization
        organization_id: "your-org-id"
        use_asset_inventory: true
        # Optional: exclude specific projects
        exclude_projects:
          - "sandbox-project-id"
          - "test-project-id"

Option 2: Folder-Level Discovery

Scope discovery to a specific folder (e.g., production workloads only):

export FOLDER_ID="your-folder-id"

gcloud resource-manager folders add-iam-policy-binding "$FOLDER_ID" \
  --member "serviceAccount:$SA_EMAIL" \
  --role "roles/viewer"

config:
  mode: folder
  folder_id: "your-folder-id"

Option 3: Explicit Project List

List specific project IDs when you don't want org-level access:

config:
  projects:
    - id: "prod-project-123"
      display_name: "Production"
    - id: "staging-project-456"
      display_name: "Staging"

Troubleshooting

PERMISSION_DENIED on Cloud Asset Inventory

Symptom: Error: PERMISSION_DENIED: The caller does not have permission

Fix: Enable the Cloud Asset Inventory API and verify the role:

# Enable the API
gcloud services enable cloudasset.googleapis.com --project "$PROJECT_ID"

# Verify the service account has the role
gcloud projects get-iam-policy "$PROJECT_ID" \
  --flatten="bindings[].members" \
  --filter="bindings.members:$SA_EMAIL" \
  --format="table(bindings.role)"

API not enabled errors

Symptom: Error: API [compute.googleapis.com] not enabled on project

Fix: Enable required APIs:

gcloud services enable \
  compute.googleapis.com \
  container.googleapis.com \
  sqladmin.googleapis.com \
  storage.googleapis.com \
  cloudfunctions.googleapis.com \
  run.googleapis.com \
  cloudasset.googleapis.com \
  cloudkms.googleapis.com \
  secretmanager.googleapis.com \
  --project "$PROJECT_ID"

Service account key rejected

Symptom: Error: invalid_grant: Invalid JWT Signature

Cause: Key file corruption or clock skew.

Fix:

# Delete and recreate the key
gcloud iam service-accounts keys delete KEY_ID \
  --iam-account "$SA_EMAIL"

gcloud iam service-accounts keys create infracast-discovery-key-new.json \
  --iam-account "$SA_EMAIL"

Quota exceeded errors

Symptom: Error: RESOURCE_EXHAUSTED: Quota exceeded for quota metric

Fix: Reduce API call frequency or request quota increase:

config:
  api_rate_limit_rps: 5
  use_asset_inventory: true  # reduces individual API calls significantly

Organization-level discovery misses some projects

Symptom: Some projects are not discovered even with org-level roles

Cause: Projects in certain folders may have org-policy constraints preventing IAM inheritance.

Fix: Check for org policy overrides:

gcloud resource-manager org-policies describe \
  iam.disableServiceAccountKeyCreation \
  --organization "$ORG_ID"

Add explicit project-level bindings for affected projects, or exclude them from discovery if they're out of scope.