GCP

​

Components and Sizing Recommendations

​

Prerequisites

• A running GKE cluster with at least 2 worker nodes. ( Best Practice: Use 2 nodes, with 1 node in each Availability Zone, to ensure high availability.)
• gcloud CLI
• Kubectl
• Helm (v3 or above)

​

Create a Portkey Account

• Go to the Portkey website.
• Sign up for a Portkey account.
• Once logged in, locate and save your Organisation ID for future reference. You can find it in the browser URL: https://app.portkey.ai/organisation/<organisation_id>/
• Contact the Portkey AI team and provide your Organisation ID and the email address used during signup.
• The Portkey team will share the following information with you:
  • Docker credentials for the Gateway images (username and password).
  • License: Client Auth Key.

​

Setup Project Environment

CLUSTER_NAME=<GKE_CLUSTER_NAME>               # Specify the name of the GKE cluster where the gateway will be deployed.
NAMESPACE=<NAMESPACE>                         # Specify the namespace where the gateway should be deployed (for example, portkeyai).
KSA=<KSA>                                     # Provide a name for the Service Account to be associated with Gateway Pod (for example, gateway-sa)

mkdir portkey-gateway
cd portkey-gateway
touch values.yaml

​

Image Credentials Configuration

# Update the values.yaml file
imageCredentials:
  - name: portkey-enterprise-registry-credentials
    create: true
    registry: https://index.docker.io/v1/
    username: <PROVIDED BY PORTKEY>
    password: <PROVIDED BY PORTKEY>

  gatewayImage:
    repository: "docker.io/portkeyai/gateway_enterprise"
    pullPolicy: Always
    tag: "latest"
  dataserviceImage:
    repository: "docker.io/portkeyai/data-service"
    pullPolicy: Always
    tag: "latest"
  redisImage:
    repository: "docker.io/redis"
    pullPolicy: IfNotPresent
    tag: "7.2-alpine"
environment:
  create: true
  secret: true
  data:
    ANALYTICS_STORE: control_plane
    SERVICE_NAME: <SERVICE_NAME>                       # Specify a name for the service
    PORTKEY_CLIENT_AUTH: <PROVIDED BY PORTKEY>
    ORGANISATIONS_TO_SYNC: <ORGANISATION_ID>           # This is obtained after signing up for a Portkey account.  

​

Configure Components

​

MCP Gateway (Optional)

environment:
  data:
    SERVER_MODE: "mcp/all"
    MCP_PORT: "8788"
    MCP_GATEWAY_BASE_URL: "<This must be set to MCP LoadBalancer URL or Hostname pointing to MCP Service>"

1. "" (empty or not provided): Deploys only the AI Gateway. This is the default configuration.
2. "mcp": Deploys only the MCP Gateway.
3. "all": Deploys both the AI Gateway and MCP Gateway.

​

Cache Store

​

Built-in Redis

## To use the built-in Redis, add the following configuration to the values.yaml file.
environment:
  data:
    CACHE_STORE: redis
    REDIS_URL: "redis://redis:6379"
    REDIS_TLS_ENABLED: "false"

​

Google Memorystore

## To use Google Memorystore Redis or Valkey, add the following configuration in the values.yaml file.
environment:
  data:
    CACHE_STORE: memory-store
    REDIS_URL: "redis://<GCP_MEMORY_STORE_IP>:<Port>" 
    REDIS_TLS_ENABLED: "false"                            ## "true"/"false"
    REDIS_MODE: cluster                                   ## Add this parameter only if cluster mode is enabled on Memorystore

## To use IAM based authentication with Google Memorystore Redis or Valkey, add the following configuration in the values.yaml file.
environment:
  data:
    CACHE_STORE: memory-store
    GCP_REDIS_AUTH_MODE: workload
    REDIS_URL: "redis://<MEMORY_STORE_IP>:<Port>" 
    REDIS_TLS_ENABLED: "false"                             ## "true"/"false"
    REDIS_MODE: cluster                                   ## Add this parameter only if cluster mode is enabled on Memorystore
    REDIS_PASSWORD: <MEMORY_STORE_AUTH_STRING>

1. Download the certificate file server-ca.pem from your GCP Memorystore Redis cluster.
2. Create a Kubernetes secret to store the Memorystore certificate:
   Copy

   Ask AI

   kubectl create secret generic memorystore-tls-certs  --from-file=server-ca.pem -n $NAMESPACE
   
3. Add the following configuration to values.yaml:
   Copy

   Ask AI

   environment:
     data:
       REDIS_TLS_CERTS: /etc/ssl/certs/server-ca.pem
       REDIS_TLS_ENABLED: "true"
   volumes:
     - name: memorystore-tls-certs
       secret:
         secretName: memorystore-tls-certs
   
   volumeMounts:
     - name: memorystore-tls-certs
       mountPath: /etc/ssl/certs/server-ca.pem
       subPath: server-ca.pem
   

​

Log Store

​

Google Cloud Storage

1. Create a GCS bucket for storing LLM access logs.
2. Set up access to the log store. The Gateway supports the following methods for connecting to GCS bucket for log storage:
   • Workload Identity Federation
   • HMAC
   Depending on the chosen GCS access method, update values.yaml with the following configuration.
   • Workload Identity Federation
   • HMAC Keys

   To set up IAM-based authentication for Portkey Gateway to GCP bucket, follow the steps and add following configuration in values.yaml.

   Copy

   Ask AI

   ## To enable `Workload Identity Federation` update values.yaml with the following details:-
   serviceAccount:
     create: true
     automount: true
     # Provide the name of service account. Must be same as the name you provide while binding GSA and KSA during workload identity permission setup.
     name: <KSA>             
     annotations:
     # Replace <GSA> and <PROJECT_ID_A> with Google Service Account name and Project ID of service account respectively.
       iam.gke.io/gcp-service-account: <GSA>@<PROJECT_ID_A>.iam.gserviceaccount.com      
   
   environment:
     data:
       LOG_STORE: gcs_assume
       GCP_AUTH_MODE: workload
       LOG_STORE_REGION: <GCS_BUCKET_REGION>                     # Specify the GCP region where the GCS log bucket resides (e.g., us-east1).
       LOG_STORE_GENERATIONS_BUCKET: <GCS_BUCKET_NAME>           # Specify the name of GCS log bucket.
   

   Copy

   Ask AI

   ## To enable HMAC based access update values.yaml with following details:-
   serviceAccount:
     create: true
     automount: true
     name: <KSA>                                  
   
   environment:
     data:
       LOG_STORE: gcs
       LOG_STORE_REGION: <GCS_BUCKET_REGION>                     # Specify the GCP region where the GCS log bucket resides (e.g., us-east1).
       LOG_STORE_GENERATIONS_BUCKET: <GCS_BUCKET_NAME>           # Specify the name of GCS log bucket.
       LOG_STORE_ACCESS_KEY: <HMAC_ACCESS_KEY>                   # Specify the HMAC access key of service account.
       LOG_STORE_SECRET_KEY: <HMAC_SECRET_KEY>                   # Specify the HMAC secret key of service account.
   
3. (Optional) Configure log path format using LOG_STORE_FILE_PATH_FORMAT. See Log Object Path Format for details.

​

Data Service (Optional)

dataservice:
  name: "dataservice"
  enabled: true
  env:
    DEBUG_ENABLED: false
    SERVICE_NAME: "portkeyenterprise-dataservice"
  serviceAccount:
    create: true
    name: <KSA>

​

Network Configuration

​

Set Up External Access

• GCS Application Load Balancer with Kubernetes Ingress
• GCS Network Load Balancer with Kubernetes Service

• GKE cluster must have HTTP Load Balancing add-on enabled.

​

GCP Load Balancer Ingress

service:
  type: ClusterIP
  port: 8787

ingress:
  enabled: true
  # hostname: "<AI Gateway Hostname>"
  # hostBased: false
  # mcpHostname: "<MCP Gateway Hostname>"
  annotations: 
    kubernetes.io/ingress.class : gce
    ingress.gcp.kubernetes.io/healthcheck-path: /v1/health

​

GCP Load Balancer Service

service:
  type: LoadBalancer
  port: 8787                                              
  annotations:
    cloud.google.com/l4-rbs: "enabled"                              # Use this annotation for creating external Load Balancer. 
    # networking.gke.io/load-balancer-type: "Internal"              # Use this annotation for creating internal Load Balancer.                      
    spec.loadBalancerSourceRanges: "0.0.0.0/0"         

​

Ensure Outbound Network Access

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all-egress
  namespace: portkeyai
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0

​

Deploying Portkey Gateway

# Add the Portkey AI Gateway helm repository
helm repo add portkey-ai https://portkey-ai.github.io/helm
helm repo update

# Install the chart
helm upgrade --install portkey-ai portkey-ai/gateway -f ./values.yaml -n ${NAMESPACE} --create-namespace

​

Verify the deployment

• Verify that all pods are running correctly.

# 
kubectl get pods -n ${NAMESPACE}
# You should see all pods with a 'STATUS' of 'Running'.

• Test Gateway by sending a cURL request.
  1. Port-forward the Gateway pod
  Copy

  Ask AI

    kubectl port-forward  <POD_NAME> -n ${NAMESPACE} 9000:8787       # Replace <POD_NAME> with your Gateway pod's actual name.
  
  2. Once port forwarding is active, open a new terminal window or tab and send a test request by running:
  Copy

  Ask AI

  # Specify LLM provider and Portkey API keys
  OPENAI_API_KEY=<OPENAI_API_KEY>                           # Replace <OPENAI_API_KEY> with an actual API key
  PORTKEY_API_KEY=<PORTKEY_API_KEY>                         # Replace <PORTKEY_API_KEY> with Portkey API key which can be created from Portkey website(https://app.portkey.ai/api-keys).
  
  # Configure and send the curl request
  curl 'http://localhost:9000/v1/chat/completions'`\
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY"  \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY"  \
  -d '{ 
      "model": "gpt-4o-mini", 
      "messages": [{"role": "user","content": "What is a fractal?"}]  
  }'
  
  3. Test gateway service integration with Load Balancer.
  Copy

  Ask AI

  # Replace <LOAD_BALANCER_IP> and <LB_LISTENER_PORT_NUMBER> with the DNS name and listener port of the created load balancer, respectively.
  curl 'http://<LOAD_BALANCER_IP>:8787/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY"  \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY"  \
  -d '{
      "model": "gpt-4o-mini",
      "messages": [{"role": "user","content": "What is a fractal?"}]
  }'
  

​

Integrating Gateway with Control Plane

• IP Whitelisting

​

IP Whitelisting

​

Verifying Gateway Integration with the Control Plane

• Send a test request to Gateway using curl.
• Go to Portkey website -> Logs.
• Verify that the test request appears in the logs and that you can view its full details by selecting the log entry.

​

Uninstalling Portkey Gateway

helm uninstall portkey-ai -n ${NAMESPACE}

​

Setting up IAM Permission