Google Gemini Best Practices Guide

crefft > Guides > AI > Gemini

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Table of Contents

• Introduction
• Model Selection
  • Model Availability
  • Gemini 3.8 Flash
  • Gemini 3.1 Pro Preview
  • Stable Fallbacks and Retired Models
  • Model Comparison Matrix
  • Pricing Structure
  • Selection Guidelines
• API Access Patterns
  • Google AI Studio
  • Vertex AI
  • Platform Comparison
  • Authentication and Setup
• System Instructions for Coding
  • Effective System Instructions
  • Code Review Instructions
  • Documentation Generation
  • Testing and Quality Assurance
  • System Instruction Best Practices
• Function Calling
  • Function Declaration
  • Function Call Flow
  • Parallel Function Calling
  • Error Handling
  • Function Calling Best Practices
• Grounding with Google Search
  • Search Grounding Configuration
  • Dynamic Retrieval
  • Grounding Metadata
  • Use Cases for Grounding
• Large Context Window Usage
  • Context Window Capabilities
  • Effective Context Loading
  • Context Caching
  • Long Context Strategies
  • Context Window Limitations
• Cost Optimization
  • Model Selection for Cost
  • Context Caching for Savings
  • Prompt Engineering for Efficiency
  • Batch Processing
  • Rate Limiting and Quotas
• Common Pitfalls
  • Safety Settings Issues
  • Context Window Misuse
  • Function Calling Errors
  • JSON Mode Mistakes
  • Streaming Complications
• Do and Don’t Examples
  • Prompt Engineering
  • API Usage
  • Function Calling
  • Context Management
  • Error Handling
• Advanced Techniques
  • Multi-Turn Conversations
  • Code Execution
  • Multimodal Capabilities
  • Controlled Generation
• Security and Privacy
  • Data Handling
  • API Key Management
  • Content Filtering
  • Compliance Considerations
• Monitoring and Debugging
  • Response Quality Metrics
  • Performance Monitoring
  • Debugging Techniques
• References

Introduction

Google Gemini¹ is a family of large language models designed for multimodal understanding and generation. This guide establishes best practices for integrating Gemini models into development workflows, with a focus on coding assistance, function calling, and leveraging Gemini’s unique capabilities like massive context windows and grounding with Google Search.

Gemini models are available through two primary platforms:

1. Google AI Studio² - Simplified API access for prototyping and small-scale applications
2. Vertex AI³ - Enterprise-grade deployment with advanced features and SLA guarantees

This guide covers both platforms and provides prescriptive guidance on model selection, API patterns, cost optimization, and common pitfalls.

Who Should Use This Guide

This guide is REQUIRED reading for:

• Developers integrating Gemini into applications
• Teams building AI-powered coding tools
• Engineers optimizing LLM costs
• Architects designing AI-assisted workflows

Prerequisites

Readers SHOULD have:

• Basic understanding of REST APIs or gRPC
• Familiarity with JSON data structures
• Experience with API authentication mechanisms
• Knowledge of Python or Node.js (for code examples)

Model Selection

Model Availability

Model IDs are not stable over time: once a model is shut down its endpoint is removed and requests fail outright rather than degrading. You MUST check the Gemini API deprecations page⁴ and the Gemini Enterprise Agent Platform model versions page⁵ before pinning a model, and you MUST record the shutdown or retirement date next to every pinned ID in your own configuration.

Why: The two platforms run separate lifecycles. A model that the Gemini API still serves can already have a retirement date on Vertex AI, so a single “current model” list is wrong for at least one of the two audiences.

The following table was verified on 8 September 2026 against the Gemini API deprecations page⁴ and the Gemini Enterprise Agent Platform model versions page⁵. “GEAP” is the Gemini Enterprise Agent Platform, the Vertex AI surface for Gemini models.

You MUST NOT send traffic to gemini-2.0-flash, gemini-1.5-pro, gemini-1.5-flash, or any -001/-002 variant of those families: those endpoints no longer exist on either platform⁴⁵.

On the Gemini Enterprise Agent Platform, the 3.6, 3.7, and 3.8 Flash models sit in the short-availability tier: they retire 45 days after a replacement model ships⁵. Models in the 12-month tier, such as gemini-3.5-flash, are guaranteed for at least a year from release⁵. Choose the tier deliberately if a migration costs you more than 45 days of notice.

You MUST pin an exact stable ID such as gemini-3.8-flash in production rather than a -latest alias. Aliases are hot-swapped on each release of that model variation, with only two weeks’ notice of breaking changes¹.

Why: An alias silently changes the model behind your evaluation baseline, so regressions appear as unexplained quality drift rather than as a deploy.

Gemini 3.8 Flash⁶

Model ID: gemini-3.8-flash — GA, released 2 September 2026, no shutdown date announced⁶⁴

Context Window: 1,048,576 tokens (1M input), 65,536 tokens (64K output)⁶

Default thinking level: medium, tunable to low or high. The minimal level is not supported on this model⁶.

Key Features⁶:

• Long-horizon software engineering across multi-file refactors
• Resilient multi-step planning and tool orchestration for autonomous agents
• Deterministic tool execution with the full built-in tool suite
• Default model for the Antigravity managed agent
• Flash-class latency and pricing

Use Cases:

• Interactive coding assistance and development environments
• Agentic workflows that call tools iteratively
• High-volume API calls with cost constraints
• Multimodal applications (code plus diagrams)

Limitations:

• Spends more tokens by design on long, complex tasks, because it takes smaller reasoning steps and verifies its own work⁶
• Preview-only capabilities (for example Live audio) require a different model

You SHOULD use Gemini 3.8 Flash as the default for new work, and you SHOULD lower thinking_level to low for latency-critical or high-throughput paths rather than switching to an older family⁶.

Gemini 3.1 Pro Preview⁷

Model ID: gemini-3.1-pro-preview — preview, released 19 February 2026, no shutdown date announced⁷⁴

Context Window: 1,048,576 tokens (1M input), 65,536 tokens (64K output); knowledge cutoff January 2025⁷

Default thinking level: high (dynamic); minimal is not supported⁷

Key Features⁷:

• Strongest reasoning in the Gemini 3 family for broad world knowledge
• Improved token efficiency and factual consistency over Gemini 3 Pro
• Optimized for agentic workflows needing precise tool usage
• A companion gemini-3.1-pro-preview-customtools endpoint that prioritizes caller-defined tools such as view_file or search_code⁷

Use Cases:

• Cross-repository refactoring and architectural decision support
• Multi-file code review where reasoning quality dominates cost
• Long-form documentation generation from large sources

Limitations:

• Preview stage: deprecation needs only two weeks’ notice¹
• Roughly 2.8x the cost of Gemini 3.8 Flash on a typical review workload (see Cost-Performance Trade-offs)
• Higher latency, because thinking defaults to high

You SHOULD use Gemini 3.1 Pro Preview when reasoning quality dominates cost, and you MUST treat it as a preview dependency: pin the exact ID, subscribe to the deprecations page⁴, and keep a tested fallback to gemini-3.8-flash.

Why: Preview models can be withdrawn on two weeks’ notice¹. Without a tested fallback that notice period becomes an unplanned migration.

Stable Fallbacks and Retired Models

gemini-2.5-pro and gemini-2.5-flash are still served by the Gemini API with no announced shutdown date⁴, but both retire on the Gemini Enterprise Agent Platform on 20 October 2026⁵.

If you run on Vertex AI you MUST migrate off the 2.5 series before 20 October 2026. If you run on the Gemini API you MAY keep 2.5 models for workloads with a frozen evaluation baseline, and you MUST re-check the deprecations page⁴ each quarter.

Why: Retirement dates may be extended but are never moved earlier⁵, so a dated check is cheap insurance and an undated “current” list is not.

Model Comparison Matrix

Verified 8 September 2026. Blank cells mean the figure is not published on the cited pages; check the model page before relying on one.

Gemini 3.8 Flash prices are the introductory rates that apply through 31 December 2026; see Pricing Structure.

Pricing Structure

Verified 8 September 2026⁸⁷ (Google AI Studio and Gemini Enterprise Agent Platform), per 1M tokens in USD, paid tier:

Gemini 3.8 Flash

• Input: $0.75 through 31 December 2026, then $1.50 from 1 January 2027
• Output (including thinking tokens): $3.75, then $7.50 from 1 January 2027
• Context caching: $0.075, then $0.15 from 1 January 2027
• Cache storage: $0.50 per 1M tokens per hour, then $1.00 from 1 January 2027
• Batch and Flex tiers: half the standard input and output rates
• Priority tier: $1.35 input, $6.75 output

Introductory pricing also covers Gemini 3.7 Flash and Gemini 3.6 Flash through 31 December 2026⁸.

Gemini 3.1 Pro Preview

• Input: $2.00 for requests under 200K tokens, $4.00 above 200K tokens
• Output: $12.00 under 200K tokens, $18.00 above 200K tokens

Grounding with Google Search

• 5,000 free search requests per month, shared across all Gemini 3.x models
• $14 per 1,000 requests thereafter⁸

Note: Free-tier usage of the Gemini API MAY be used to improve Google’s products; paid-tier usage MAY NOT⁸. See Data Handling before sending any proprietary source code.

Selection Guidelines

Decision Tree

START
├─ Running a 2.5-series model on Vertex AI?
│  └─ YES → migrate before 20 Oct 2026 → gemini-3.8-flash
│  └─ NO → Continue
│
├─ Prompt larger than 1M tokens?
│  └─ YES → split or retrieve; no current model exceeds 1M input
│  └─ NO → Continue
│
├─ Reasoning quality dominates cost (architecture, cross-repo refactor)?
│  └─ YES → gemini-3.1-pro-preview (pin it; keep a Flash fallback)
│  └─ NO → Continue
│
├─ Latency- or throughput-critical, well-defined task?
│  └─ YES → gemini-3.8-flash with thinking_level="low"
│  └─ NO → Continue
│
└─ Default → gemini-3.8-flash (thinking_level="medium")

Cost-Performance Trade-offs

Example Scenario: Code review service processing 1,000 pull requests/day

Each PR analysis requires:

• Average input: 50,000 tokens
• Average output: 2,000 tokens

That is 1,500M input tokens and 60M output tokens per 30-day month.

Monthly Costs (introductory rates, through 31 December 2026):

Gemini 3.8 Flash:

• Input: 1,500M tokens × $0.75/1M = $1,125.00
• Output: 60M tokens × $3.75/1M = $225.00
• Total: $1,350.00/month (doubles to $2,700.00 from 1 January 2027)

Gemini 3.1 Pro Preview (requests under 200K tokens):

• Input: 1,500M tokens × $2.00/1M = $3,000.00
• Output: 60M tokens × $12.00/1M = $720.00
• Total: $3,720.00/month

Cost Difference: 2.8x more expensive for Pro

When Pro is Worth It:

• Critical path reviews (security, compliance)
• Complex architectural changes
• Cross-repository refactoring
• Quality > cost in business requirements

API Access Patterns

Google AI Studio

Overview: Google AI Studio² provides a simplified REST API for accessing Gemini models with minimal setup.

Best For:

• Rapid prototyping
• Individual developers
• Small applications (< 100K requests/month)
• Educational projects
• Quick experiments

Limitations:

• No SLA guarantees
• Limited enterprise features
• Fewer regional deployment options
• No VPC integration
• Basic quota management

Authentication

The Gemini API accepts two kinds of API key and they are not interchangeable⁹:

• Authorization (auth) keys are bound to a Google Cloud service account. Requests run under that service account’s identity, the key is restricted to the Gemini API by default, and Google’s leaked-key enforcement disables it quickly when detection fires.
• Standard keys identify only the billing project. They name no caller, so they cannot carry granular permissions.

You MUST authenticate with an auth key. Every key created in Google AI Studio today is an auth key, but keys created earlier MAY still be standard ones. The Gemini API already rejects requests from unrestricted standard keys, and from September 2026 it rejects standard keys entirely⁹.

Why: A standard key proves only which project pays. An auth key carries a service account identity, so IAM decides what the caller may do and a leaked key can be revoked without disturbing every other consumer of the project.

Migration checklist — work through this for each environment⁹:

1. Open the AI Studio API Keys page and read the Key Type column.
2. For every key marked Standard, click Create API key; new keys are created as auth keys automatically.
3. Update application code, environment variables, and deployment configuration to the new key.
4. Test the application against the new key.
5. Delete or revoke the old standard key once traffic has moved.

Creating an auth key requires these project IAM permissions⁹: resourcemanager.projects.get, apikeys.keys.create, serviceusage.services.enable, iam.serviceAccounts.create, and iam.serviceAccountApiKeyBindings.create.

Supply the key through the environment. The client libraries read GEMINI_API_KEY or GOOGLE_API_KEY, and GOOGLE_API_KEY wins if both are set⁹:

export GEMINI_API_KEY="your-auth-key-here"

You MUST send the key in the x-goog-api-key header rather than a key query parameter⁹:

curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.8-flash:generateContent" \
    -H "x-goog-api-key: ${GEMINI_API_KEY}" \
    -H "Content-Type: application/json" \
    -X POST \
    -d '{"contents": [{"parts": [{"text": "Explain recursion in one sentence"}]}]}'

Why: Query strings leak. They land in proxy and load-balancer access logs, browser history, Referer headers, and shell history, none of which are designed to hold credentials. A header is not logged by default.

You SHOULD NOT:

• Commit API keys to version control
• Share API keys across teams
• Use the same key for dev and production
• Ship a key in client-side web or mobile code; proxy through your backend⁹

You MUST:

• Store keys in environment variables or a secret manager such as Google Cloud Secret Manager⁹
• Apply application restrictions to every key; the API blocks unrestricted keys that have been dormant for an extended period since 7 May 2026⁹
• Rotate keys regularly (every 90 days recommended)
• Use separate keys per environment

Basic REST API Example

import os
import google.generativeai as genai

genai.configure(api_key=os.environ["GOOGLE_API_KEY"])

model = genai.GenerativeModel("gemini-3.8-flash")

response = model.generate_content(
    "Write a Python function to calculate factorial recursively"
)

print(response.text)

POST https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent

Vertex AI

Overview: Vertex AI³ provides enterprise-grade access to Gemini with advanced features, SLAs, and integration with Google Cloud services.

Best For:

• Production applications
• Enterprise deployments
• High-volume processing
• Compliance requirements (HIPAA, SOC 2)
• Multi-region deployment
• Integration with GCP ecosystem

Advantages:

• 99.9% SLA availability
• VPC Service Controls
• CMEK (Customer-Managed Encryption Keys)
• Cloud Logging and Monitoring integration
• IAM-based access control
• Private endpoints
• Batch prediction support

Authentication

You MUST use Google Cloud authentication:

from google.cloud import aiplatform
from vertexai.preview.generative_models import GenerativeModel

# Initialize Vertex AI
aiplatform.init(
    project="your-project-id",
    location="us-central1"
)

model = GenerativeModel("gemini-3.8-flash")

response = model.generate_content(
    "Write a Python function to calculate factorial recursively"
)

print(response.text)

Service Account Setup

Vertex AI uses Application Default Credentials (ADC): the client libraries find credentials from the environment, so the same code runs locally and in production without a credentials file¹⁰.

You MUST choose the credential source by environment, in this order¹¹:

1. Running on Google Cloud — attach a service account to the resource (or use Workload Identity Federation for GKE) and grant it roles/aiplatform.user
2. Running outside Google Cloud with an external identity provider — configure workload identity federation
3. Local development — gcloud auth application-default login, and impersonate a service account when you need to match production’s identity
4. Downloaded JSON key — only when none of the above is possible

You MUST NOT download a service account key when an alternative applies.

Why: A downloaded key is a long-lived credential that never expires on its own. Google’s own guidance treats user-managed keys as an exception rather than the norm, because they leak into repositories, buckets, inboxes, and CI logs, and a leaked key authenticates with no further challenge¹².

# Create service account
gcloud iam service-accounts create gemini-service \
    --display-name="Gemini API Service Account"

# Grant permissions
gcloud projects add-iam-policy-binding your-project-id \
    --member="serviceAccount:gemini-service@your-project-id.iam.gserviceaccount.com" \
    --role="roles/aiplatform.user"

# Production on Google Cloud: attach the service account to the workload;
# no key material is created or stored.
gcloud run deploy gemini-app \
    --service-account=gemini-service@your-project-id.iam.gserviceaccount.com

# Local development: user credentials for ADC, impersonating the same
# service account so local and production authorization match.
gcloud auth application-default login
gcloud config set auth/impersonate_service_account \
    gemini-service@your-project-id.iam.gserviceaccount.com

If you have an exception that genuinely requires a downloaded key, you MUST write it straight to its destination, keep it out of source control and out of the downloads folder, restrict its permissions, rotate it on a schedule, and delete it once the workload can use a safer method¹²:

# EXCEPTION ONLY: no attached identity and no federation available
gcloud iam service-accounts keys create /etc/gemini/key.json \
    --iam-account=gemini-service@your-project-id.iam.gserviceaccount.com
chmod 600 /etc/gemini/key.json
export GOOGLE_APPLICATION_CREDENTIALS="/etc/gemini/key.json"

You SHOULD block the exception from becoming the habit by setting the organization policy constraint that disables service account key creation, and granting exemptions only to projects that have demonstrated they cannot use a safer method¹².

Regional Endpoints

Vertex AI supports multiple regions. You SHOULD choose regions based on:

1. Latency requirements - Select region closest to users
2. Data residency - Comply with local regulations
3. Availability - Some models may not be available in all regions

Available regions (as of December 2024)³:

• us-central1 (Iowa)
• us-east4 (Virginia)
• us-west1 (Oregon)
• europe-west1 (Belgium)
• europe-west4 (Netherlands)
• asia-northeast1 (Tokyo)
• asia-southeast1 (Singapore)

aiplatform.init(
    project="your-project-id",
    location="europe-west1"  # Choose appropriate region
)

Platform Comparison

Authentication and Setup

Google AI Studio Setup

1. Visit Google AI Studio²
2. Sign in with Google account
3. Create an API key; keys created in AI Studio are auth keys⁹
4. Set the environment variable

export GEMINI_API_KEY="your-auth-key-here"

Vertex AI Setup

1. Create or select a GCP project

2. Enable the Vertex AI API, aiplatform.googleapis.com³:

   gcloud services enable aiplatform.googleapis.com
   

3. Set up authentication (see Service Account Setup above)

4. Install the Vertex AI SDK, google-cloud-aiplatform¹³:

   pip install google-cloud-aiplatform==2.5.0
   

Footnote markers MUST stay in the surrounding prose. A marker such as [^n] inside a command changes what the command means: pip reads google-cloud-aiplatform[^n] as a malformed extras group and refuses the requirement, and a shell reads aiplatform.googleapis.com[^n] as a bracket glob that can expand to a neighboring filename.

SDK Installation

Install the Google AI Studio SDK (google-generativeai) or the Vertex AI SDK (google-cloud-aiplatform¹³) for Python:

# For Google AI Studio
pip install google-generativeai==0.8.6

# For Vertex AI
pip install google-cloud-aiplatform==2.5.0

For Node.js, install @google/generative-ai or @google-cloud/vertexai:

# For Google AI Studio
npm install @google/generative-ai@0.24.1

# For Vertex AI
npm install @google-cloud/vertexai@1.12.0

System Instructions for Coding

System instructions provide persistent context that applies to all messages in a conversation. They are CRITICAL for consistent coding assistance.

Effective System Instructions

System instructions SHOULD:

• Define the assistant’s role and expertise
• Specify output format and structure
• Establish code style preferences
• Set constraints and boundaries
• Include examples of desired behavior

System instructions MUST:

• Be clear and unambiguous
• Avoid contradictions
• Stay under 10,000 tokens for performance
• Be tested thoroughly before deployment

Basic Coding Assistant

model = genai.GenerativeModel(
    model_name="gemini-3.8-flash",
    system_instruction="""You are an expert software engineer specializing in Python, JavaScript, and Go.

Your responses MUST:
- Provide working, tested code
- Include inline comments for complex logic
- Follow language-specific style guides (PEP 8 for Python, Airbnb for JavaScript)
- Include error handling
- Consider edge cases

Your responses MUST NOT:
- Include placeholder code or TODOs
- Assume undefined variables or functions
- Ignore security considerations
- Skip input validation

When suggesting refactoring:
1. Explain the problem with current code
2. Show the refactored version
3. Explain why the refactoring improves the code
4. Note any trade-offs or considerations
"""
)

Production-Grade System Instructions

system_instruction = """You are a senior software engineer conducting code reviews for a production system.

# Role and Expertise
- 10+ years experience in distributed systems
- Expert in Python, TypeScript, and cloud architecture
- Familiar with microservices, Docker, Kubernetes

# Code Review Standards
Your reviews MUST check for:

1. **Correctness**
   - Logic errors and edge cases
   - Type safety
   - Null/undefined handling
   - Off-by-one errors

2. **Security**
   - SQL injection vulnerabilities
   - XSS vulnerabilities
   - Authentication/authorization issues
   - Secrets in code
   - Input validation

3. **Performance**
   - O(n²) or worse algorithms
   - N+1 query problems
   - Memory leaks
   - Unnecessary allocations

4. **Maintainability**
   - Code complexity (cyclomatic complexity < 10)
   - Function length (< 50 lines)
   - Single Responsibility Principle
   - Clear naming conventions

5. **Testing**
   - Test coverage > 80%
   - Edge cases covered
   - Integration tests for critical paths
   - Mocking external dependencies

# Output Format
For each issue found:

**Severity**: [CRITICAL|HIGH|MEDIUM|LOW]
**Category**: [Correctness|Security|Performance|Maintainability|Testing]
**Location**: File:Line
**Issue**: Clear description of the problem
**Recommendation**: Specific fix with code example
**Rationale**: Why this matters

# Constraints
- Focus on substantive issues, not style nitpicks
- Provide actionable feedback with code examples
- Consider the broader system context
- Balance idealism with pragmatism
"""

model = genai.GenerativeModel(
    model_name="gemini-3.1-pro-preview",
    system_instruction=system_instruction
)

Code Review Instructions

code_review_instruction = """You are an automated code review assistant for a team using:
- Language: Python 3.11+
- Framework: FastAPI
- Database: PostgreSQL with SQLAlchemy
- Testing: pytest
- Style: Black formatter, flake8 linter, mypy type checker

# Review Checklist

## FastAPI Specific
- [ ] Proper dependency injection usage
- [ ] Response model validation
- [ ] HTTP status codes match REST conventions
- [ ] Async/await used correctly
- [ ] Background tasks for long operations

## Database
- [ ] No raw SQL (use SQLAlchemy ORM)
- [ ] Proper transaction handling
- [ ] Connection pooling configured
- [ ] Migrations included for schema changes
- [ ] Indexes on foreign keys and frequently queried columns

## Testing
- [ ] Unit tests for business logic
- [ ] Integration tests for endpoints
- [ ] Test database fixtures used
- [ ] Mocking external API calls
- [ ] Edge cases covered (empty lists, None values, etc.)

## Security
- [ ] Input validation with Pydantic
- [ ] SQL injection prevention (ORM usage)
- [ ] Authentication on protected endpoints
- [ ] Rate limiting on public endpoints
- [ ] Secrets in environment variables, not code

# Output Format
Provide a structured review with:
1. Summary (2-3 sentences)
2. Critical Issues (blocking PR merge)
3. Important Issues (should fix before merge)
4. Suggestions (nice to have improvements)
5. Positive Feedback (what was done well)

Keep feedback specific, actionable, and constructive.
"""

Documentation Generation

docs_instruction = """You are a technical documentation specialist.

# Documentation Standards

## Function Documentation
Use Google-style docstrings:

```python
def example_function(param1: str, param2: int) -> bool:
    \"\"\"Brief one-line summary.

    Detailed description of what the function does, including any
    important implementation details or algorithms used.

    Args:
        param1: Description of param1
        param2: Description of param2

    Returns:
        Description of return value

    Raises:
        ValueError: When and why this is raised
        TypeError: When and why this is raised

    Example:
        >>> example_function("test", 42)
        True
    \"\"\"
```

## Class Documentation

Include:

- Purpose of the class
- Key attributes
- Important methods
- Usage examples
- Thread safety notes (if applicable)

## Module Documentation

At the top of each module:

- Module purpose
- Key classes/functions
- Dependencies
- Usage examples

# Tone and Style

- Use present tense ("Returns" not "Will return")
- Be concise but complete
- Include examples for complex behavior
- Link to related functions/classes
- Note any deprecations or future changes
"""

Testing and Quality Assurance

testing_instruction = """You are a test engineering specialist.

# Testing Philosophy
- Test behavior, not implementation
- Each test should verify one behavior
- Tests should be independent and isolated
- Use descriptive test names: test_<method>_<scenario>_<expected>

# Test Structure (Arrange-Act-Assert)
```python
def test_user_creation_with_valid_data_creates_user():
    # Arrange
    user_data = {"email": "test@example.com", "name": "Test User"}

    # Act
    user = User.create(user_data)

    # Assert
    assert user.email == "test@example.com"
    assert user.name == "Test User"
    assert user.id is not None
```

# Coverage Requirements

- Unit tests: 80%+ coverage
- Integration tests: Critical paths must be covered
- Edge cases: Empty inputs, None, boundary values
- Error cases: Invalid inputs, network failures, etc.

# Mocking Guidelines

- Mock external services (APIs, databases in unit tests)
- Use fixtures for test data
- Reset mocks between tests
- Verify mock calls when behavior matters

# Test Generation

When generating tests:

1. Cover happy path first
2. Add edge cases
3. Add error cases
4. Add integration tests for critical flows
5. Include performance tests for hot paths (> 1000 calls/sec)

# Output Format

Provide complete, runnable test files with:

- Necessary imports
- Fixtures and setup
- Complete test functions
- Teardown if needed
- Comments explaining complex assertions
"""

System Instruction Best Practices

DO

1. Be Specific About Output Format

system_instruction = """
# Output Format
Your responses MUST be valid JSON with this structure:
{
    "analysis": "Brief description",
    "issues": [
        {
            "severity": "HIGH|MEDIUM|LOW",
            "file": "path/to/file.py",
            "line": 42,
            "description": "Issue description",
            "fix": "Suggested fix"
        }
    ]
}
"""

1. Include Concrete Examples

system_instruction = """
When suggesting refactoring, use this format:

**Before**:
```python
def bad_example():
    x = get_value()
    if x:
        return x
    else:
        return None
```

**After**:

```python
def good_example():
    return get_value()
```

**Reason**: The else clause is unnecessary; Python functions return None by default.
"""

1. Set Clear Boundaries

system_instruction = """
You MUST NOT:
- Suggest removing error handling
- Recommend using deprecated APIs
- Propose changes that break backward compatibility
- Suggest optimizations without profiling data

You MUST:
- Preserve existing error handling
- Recommend current stable APIs
- Note when changes are breaking
- Suggest profiling before optimization
"""

DON’T

1. Don’t Be Vague

# BAD
system_instruction = "You are a helpful coding assistant."

# GOOD
system_instruction = """You are a Python expert specializing in Django web development.
You provide code that follows Django best practices and the project's coding standards."""

1. Don’t Contradict Yourself

# BAD
system_instruction = """
Always include comprehensive error handling.
Keep functions under 10 lines for readability.
"""
# These may conflict for complex error handling

1. Don’t Overload with Information

# BAD - Too much information
system_instruction = """
[5000 lines of detailed coding standards, architecture docs, and examples]
"""

# GOOD - Reference documentation, provide essentials
system_instruction = """
Follow the Python style guide at: docs/python-style.md

Key requirements:
- Type hints on all functions
- Docstrings on public APIs
- Error handling for external calls
- Unit tests for business logic
"""

Function Calling

Function calling¹⁴ enables Gemini to interact with external systems, APIs, and tools. This is ESSENTIAL for building AI agents and assistants that take actions.

Function Declaration

Functions MUST be declared with JSON Schema¹⁴:

get_weather_function = {
    "name": "get_weather",
    "description": "Get current weather for a location. Use this when users ask about weather conditions.",
    "parameters": {
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City name or zip code, e.g. 'San Francisco' or '94102'"
            },
            "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
                "description": "Temperature unit",
                "default": "fahrenheit"
            }
        },
        "required": ["location"]
    }
}

model = genai.GenerativeModel(
    model_name="gemini-3.8-flash",
    tools=[get_weather_function]
)

Complete Example with Implementation

import google.generativeai as genai
import json

# Define functions
def get_current_weather(location: str, unit: str = "fahrenheit") -> dict:
    """Actual implementation of weather fetching"""
    # In production, call a real weather API
    return {
        "location": location,
        "temperature": 72,
        "unit": unit,
        "conditions": "sunny"
    }

def get_stock_price(symbol: str) -> dict:
    """Get stock price for a symbol"""
    # In production, call a real stock API
    return {
        "symbol": symbol,
        "price": 150.25,
        "change": 2.5,
        "change_percent": 1.7
    }

# Declare functions for Gemini
tools = [
    {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City name, e.g. 'San Francisco, CA'"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "default": "fahrenheit"
                }
            },
            "required": ["location"]
        }
    },
    {
        "name": "get_stock_price",
        "description": "Get current stock price for a ticker symbol",
        "parameters": {
            "type": "object",
            "properties": {
                "symbol": {
                    "type": "string",
                    "description": "Stock ticker symbol, e.g. 'GOOGL'"
                }
            },
            "required": ["symbol"]
        }
    }
]

# Create model with tools
model = genai.GenerativeModel(
    model_name="gemini-3.8-flash",
    tools=tools
)

# Function call dispatcher
function_map = {
    "get_current_weather": get_current_weather,
    "get_stock_price": get_stock_price
}

# Start conversation
chat = model.start_chat()

# User message
user_message = "What's the weather in New York and the stock price of GOOGL?"
response = chat.send_message(user_message)

# Collect the populated fields. "function_call" in part is a proto-plus
# presence check: hasattr() is always True on a Part, even for text.
function_calls = []
text_chunks = []
for part in response.candidates[0].content.parts:
    if "function_call" in part:
        function_calls.append(part.function_call)
    elif "text" in part:
        text_chunks.append(part.text)

if function_calls:
    # Execute functions
    function_responses = []
    for fc in function_calls:
        function_name = fc.name
        function_args = dict(fc.args)

        # Call the actual function
        result = function_map[function_name](**function_args)

        function_responses.append({
            "name": function_name,
            "response": result
        })

    # Send function results back to model
    response = chat.send_message(
        genai.protos.Content(
            parts=[
                genai.protos.Part(
                    function_response=genai.protos.FunctionResponse(
                        name=fr["name"],
                        response={"result": fr["response"]}
                    )
                )
                for fr in function_responses
            ]
        )
    )

    print(response.text)
else:
    print("".join(text_chunks))

A response MAY mix text and tool calls in one turn, and the tool call is not always the first part. You MUST scan every part, and you MUST test which field is populated rather than whether an attribute exists.

Why: On a Part, hasattr(part, "function_call") is always true. A text-only part reports an empty FunctionCall whose name is "", so an attribute check appends a phantom tool call, skips the text branch, and then fails the dispatcher lookup on the empty name. "function_call" in part tests which field of the data oneof is actually set, so text-only, tool-only, and mixed responses all route correctly.

Function Call Flow

User Message
    ↓
Gemini Processes Input
    ↓
Decision: Need External Data?
    ├─ No → Generate text response
    └─ Yes → Generate function call(s)
         ↓
Client receives function call instructions
    ↓
Client executes actual functions
    ↓
Client sends function results to Gemini
    ↓
Gemini processes results
    ↓
Gemini generates final response
    ↓
User receives answer

Parallel Function Calling

Gemini CAN call multiple functions in parallel:

# User asks: "What's the weather in SF and NYC, and GOOGL stock price?"

# Gemini may return multiple function calls:
# 1. get_current_weather(location="San Francisco")
# 2. get_current_weather(location="New York City")
# 3. get_stock_price(symbol="GOOGL")

# You SHOULD execute these in parallel for performance:
import asyncio

async def execute_function(name: str, args: dict):
    function = function_map[name]
    # Wrap sync functions if needed
    return await asyncio.to_thread(function, **args)

async def execute_all_functions(function_calls):
    tasks = [
        execute_function(fc.name, dict(fc.args))
        for fc in function_calls
    ]
    return await asyncio.gather(*tasks)

# Execute all function calls concurrently
results = asyncio.run(execute_all_functions(function_calls))

Error Handling

Functions SHOULD return error information in the response:

def get_weather(location: str, unit: str = "fahrenheit") -> dict:
    try:
        # Call weather API
        result = call_weather_api(location, unit)
        return {
            "success": True,
            "data": result
        }
    except LocationNotFound as e:
        return {
            "success": False,
            "error": "location_not_found",
            "message": f"Could not find location: {location}"
        }
    except APIError as e:
        return {
            "success": False,
            "error": "api_error",
            "message": "Weather service temporarily unavailable"
        }

Gemini will incorporate error information into its response:

User: What's the weather in Atlantis?
Gemini: I couldn't find weather information for Atlantis as it's not a recognized location.
Could you provide a valid city name?

Function Calling Best Practices

DO

1. Provide Detailed Descriptions

# GOOD
{
    "name": "search_codebase",
    "description": """Search the codebase for matching files or code patterns.

    Use this when users ask to:
    - Find files by name or pattern
    - Locate function or class definitions
    - Search for specific code patterns
    - Find usage examples

    Examples:
    - "Find all Python files in the src directory"
    - "Where is the User class defined?"
    - "Show me examples of database queries"
    """,
    "parameters": {...}
}

1. Use Enums for Constrained Values

{
    "name": "create_file",
    "parameters": {
        "type": "object",
        "properties": {
            "file_type": {
                "type": "string",
                "enum": ["python", "javascript", "typescript", "markdown"],
                "description": "Type of file to create"
            }
        }
    }
}

1. Validate Function Arguments

A string prefix test is not a read-only boundary. Enforce the boundary in the database, let the model pick an allowlisted query by name, and bind its arguments as parameters:

-- The role the tool connects as can read the analytics schema, nothing else.
CREATE ROLE gemini_readonly LOGIN;
REVOKE ALL ON SCHEMA analytics FROM gemini_readonly;
GRANT USAGE ON SCHEMA analytics TO gemini_readonly;
GRANT SELECT ON ALL TABLES IN SCHEMA analytics TO gemini_readonly;
ALTER ROLE gemini_readonly SET default_transaction_read_only = on;
ALTER ROLE gemini_readonly SET statement_timeout = '10s';

# The model never authors SQL. It names a query and supplies parameters.
ALLOWED_QUERIES = {
    "orders_by_day": (
        "SELECT order_date, COUNT(*) AS orders "
        "FROM analytics.orders "
        "WHERE order_date BETWEEN %(start)s AND %(end)s "
        "GROUP BY order_date ORDER BY order_date"
    ),
    "top_products": (
        "SELECT product_id, SUM(quantity) AS units "
        "FROM analytics.order_items "
        "WHERE order_date >= %(start)s "
        "GROUP BY product_id ORDER BY units DESC"
    ),
}

REQUIRED_PARAMS = {
    "orders_by_day": frozenset({"start", "end"}),
    "top_products": frozenset({"start"}),
}

MAX_ROWS = 500
STATEMENT_TIMEOUT_MS = 10_000


def run_named_query(name: str, params: dict, *, principal: str) -> dict:
    """Run one allowlisted, parameterised query on a read-only connection."""
    sql = ALLOWED_QUERIES.get(name)
    if sql is None:
        return {
            "success": False,
            "error": "unknown_query",
            "allowed": sorted(ALLOWED_QUERIES),
        }

    expected = REQUIRED_PARAMS[name]
    missing = sorted(expected - params.keys())
    unexpected = sorted(params.keys() - expected)
    if missing or unexpected:
        return {
            "success": False,
            "error": "bad_parameters",
            "missing": missing,
            "unexpected": unexpected,
        }

    # Audit every call: who asked, which query, which arguments.
    audit_log.info(
        "tool_query", extra={"principal": principal, "query": name, "params": params}
    )

    with connect_read_only() as conn, conn.cursor() as cur:
        cur.execute(f"SET LOCAL statement_timeout = {STATEMENT_TIMEOUT_MS}")
        cur.execute(sql, {key: params[key] for key in expected})
        rows = cur.fetchmany(MAX_ROWS + 1)

    return {
        "success": True,
        "rows": rows[:MAX_ROWS],
        "truncated": len(rows) > MAX_ROWS,
    }

Why: sql.strip().upper().startswith("SELECT") accepts SELECT 1; DROP TABLE users, SELECT * FROM t; UPDATE t SET x = 1, and SELECT pg_sleep(600). Whether those run depends on the driver’s handling of multiple statements and on the role’s privileges, so the check enforces nothing it claims to. Privileges, a fixed query text, bound parameters, a statement timeout, and a row cap each hold on their own.

If a use case genuinely needs model-authored SQL, you MUST still connect as the read-only role and you MUST parse the statement, requiring exactly one statement with a SELECT root, using a real SQL parser rather than string inspection.

1. Return Structured Data

# GOOD - Structured response
def search_code(pattern: str) -> dict:
    return {
        "total_matches": 5,
        "matches": [
            {
                "file": "src/user.py",
                "line": 42,
                "context": "def authenticate_user(username, password):"
            },
            ...
        ]
    }

# BAD - Unstructured string
def search_code(pattern: str) -> str:
    return "Found 5 matches:\n1. src/user.py line 42\n..."

DON’T

1. Don’t Use Generic Descriptions

# BAD
{
    "name": "query",
    "description": "Runs a query",
    ...
}

# GOOD
{
    "name": "execute_database_query",
    "description": "Execute a read-only SQL SELECT query against the analytics database",
    ...
}

1. Don’t Allow Dangerous Operations Without Safeguards

# BAD - No validation
def execute_code(code: str):
    exec(code)  # Dangerous!

# GOOD - Sandboxed execution
def execute_code(code: str, timeout: int = 5):
    # Run in restricted environment
    # Set timeout
    # Validate code before execution
    # Return results safely
    ...

1. Don’t Return Raw Exceptions

# BAD
def get_file_contents(path: str):
    return open(path).read()  # Raises exception on error

# GOOD
def get_file_contents(path: str) -> dict:
    try:
        with open(path) as f:
            return {
                "success": True,
                "content": f.read()
            }
    except FileNotFoundError:
        return {
            "success": False,
            "error": "file_not_found",
            "message": f"File not found: {path}"
        }

Grounding with Google Search

Grounding¹⁵ allows Gemini to retrieve up-to-date information from Google Search, reducing hallucinations and providing current data.

Search Grounding Configuration

Availability: Vertex AI only (not available in Google AI Studio)

from vertexai.preview.generative_models import (
    GenerativeModel,
    Tool,
    grounding
)

# Create grounding tool
google_search_tool = Tool.from_google_search_retrieval(
    grounding.GoogleSearchRetrieval()
)

model = GenerativeModel(
    "gemini-3.8-flash",
    tools=[google_search_tool]
)

response = model.generate_content(
    "What are the latest Python 3.13 features released in 2024?"
)

print(response.text)

# Access grounding metadata
if response.candidates[0].grounding_metadata:
    metadata = response.candidates[0].grounding_metadata
    print(f"\nGrounding sources: {len(metadata.grounding_chunks)}")
    for chunk in metadata.grounding_chunks:
        print(f"- {chunk.web.uri}")

Dynamic Retrieval

You can control when grounding is applied:

from vertexai.preview.generative_models import grounding

# Configure dynamic retrieval threshold
google_search_tool = Tool.from_google_search_retrieval(
    grounding.GoogleSearchRetrieval(
        disable_attribution=False  # Include source attribution
    )
)

model = GenerativeModel(
    "gemini-3.1-pro-preview",
    tools=[google_search_tool]
)

# Gemini decides when to use search based on query needs
response = model.generate_content(
    """Compare the performance characteristics of Python 3.13 vs 3.12.
    Include benchmarks if available."""
)

Grounding Metadata

Grounding responses include metadata about sources:

response = model.generate_content("Latest news about Gemini 3.8 Flash")

metadata = response.candidates[0].grounding_metadata

# Grounding support score (0.0 to 1.0)
print(f"Support score: {metadata.grounding_support.support_score}")

# Search entry point (the query used)
if metadata.search_entry_point:
    print(f"Search query: {metadata.search_entry_point.rendered_content}")

# Retrieved chunks and sources
for chunk in metadata.grounding_chunks:
    if chunk.web:
        print(f"Source: {chunk.web.title}")
        print(f"URL: {chunk.web.uri}")

Use Cases for Grounding

Ideal Use Cases

1. Current Events and News

prompt = "What are the major tech announcements from CES 2025?"
# Grounding ensures up-to-date information

1. Recent API Documentation

prompt = "How do I use the new React Server Components in Next.js 15?"
# Gets latest documentation and examples

1. Version-Specific Information

prompt = "What breaking changes were introduced in TypeScript 5.3?"
# Retrieves accurate version-specific details

1. Factual Verification

prompt = "What is the current market cap of NVIDIA?"
# Grounds answer in current data

Poor Use Cases

1. Creative Writing - Grounding not needed
2. Code Generation from Requirements - Internal knowledge sufficient
3. General Programming Concepts - Model knowledge adequate
4. Refactoring Existing Code - Context-based task

You SHOULD use grounding when:

• Information changes frequently
• Accuracy of current data is critical
• User asks about recent events (last 6 months)
• Factual verification is needed

You SHOULD NOT use grounding when:

• Question is about established concepts
• Speed is more important than currency
• User’s codebase context is more relevant than web search
• Privacy concerns with query content

Large Context Window Usage

Gemini models support 1M-token context windows, enabling analysis of entire codebases, long documents, and extensive conversations.

Context Window Capabilities

Verified 8 September 2026; see Model Availability for lifecycle dates.

No current Gemini model accepts more than 1M input tokens. Inputs larger than that MUST be split, summarized, or retrieved selectively; see Long Context Strategies.

Token Estimation:

• 1 token ≈ 4 characters
• 1 token ≈ 0.75 words
• 1000 tokens ≈ 750 words
• 100K tokens ≈ 75,000 words ≈ 300 pages

Effective Context Loading

Loading Multiple Files

Sending a repository to a third-party API is a disclosure event, so the loader MUST decide what leaves the machine, not the caller’s rglob pattern.

The loader below is manifest-first: it sends the paths you list, or, with no manifest, only allowlisted file types under the repository root. It refuses symlinks and paths outside the root, skips files whose contents match secret patterns, enforces per-file and total budgets, and returns a report naming every skipped path with its reason.

import re
from dataclasses import dataclass, field
from pathlib import Path

ALLOWED_SUFFIXES = {".py", ".md", ".yaml", ".yml", ".toml"}
EXCLUDED_DIRS = {
    ".git", "node_modules", "__pycache__", "venv", ".venv",
    "dist", "build", ".next", "coverage",
}
# Files that carry credentials by convention, whatever their suffix.
EXCLUDED_NAMES = {
    ".env", ".env.local", ".npmrc", ".netrc", "id_rsa",
    "credentials.json", "service-account.json",
}
SECRET_PATTERNS = (
    re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----"),
    re.compile(r"\bAIza[0-9A-Za-z_\-]{35}\b"),
    re.compile(r"\bgh[pousr]_[0-9A-Za-z]{36}\b"),
    re.compile(
        r"(?i)\b(api[_-]?key|secret|passwd|password|token)\s*[:=]\s*"
        r"['\"][^'\"]{8,}['\"]"
    ),
)

MAX_FILE_BYTES = 200_000
MAX_TOTAL_TOKENS = 400_000
CHARS_PER_TOKEN = 4  # See Token Estimation above


@dataclass
class LoadReport:
    included: list[str] = field(default_factory=list)
    skipped: list[tuple[str, str]] = field(default_factory=list)
    estimated_tokens: int = 0


def _candidates(root: Path, manifest: list[str] | None) -> list[Path]:
    if manifest is not None:
        return [root / entry for entry in manifest]
    return sorted(
        path
        for path in root.rglob("*")
        if path.is_file() and path.suffix in ALLOWED_SUFFIXES
    )


def load_codebase(
    root_dir: str,
    manifest: list[str] | None = None,
) -> tuple[str, LoadReport]:
    """Load reviewed files into one context, reporting everything skipped."""
    root = Path(root_dir).resolve(strict=True)
    report = LoadReport()
    parts: list[str] = []

    for path in _candidates(root, manifest):
        label = str(path)

        if path.is_symlink():
            report.skipped.append((label, "symlink"))
            continue

        resolved = path.resolve()
        if root not in resolved.parents:
            report.skipped.append((label, "outside repository root"))
            continue
        if not resolved.is_file():
            report.skipped.append((label, "not a regular file"))
            continue

        relative = resolved.relative_to(root)
        if set(relative.parts) & EXCLUDED_DIRS:
            report.skipped.append((label, "excluded directory"))
            continue
        if resolved.name in EXCLUDED_NAMES:
            report.skipped.append((label, "credential filename"))
            continue
        if resolved.suffix not in ALLOWED_SUFFIXES:
            report.skipped.append((label, f"suffix not allowlisted: {resolved.suffix}"))
            continue
        if resolved.stat().st_size > MAX_FILE_BYTES:
            report.skipped.append((label, f"larger than {MAX_FILE_BYTES} bytes"))
            continue

        try:
            content = resolved.read_text(encoding="utf-8")
        except (UnicodeDecodeError, OSError) as exc:
            report.skipped.append((label, f"unreadable: {type(exc).__name__}"))
            continue

        if any(pattern.search(content) for pattern in SECRET_PATTERNS):
            report.skipped.append((label, "matched a secret pattern"))
            continue

        tokens = len(content) // CHARS_PER_TOKEN
        if report.estimated_tokens + tokens > MAX_TOTAL_TOKENS:
            report.skipped.append((label, "token budget exhausted"))
            continue

        report.estimated_tokens += tokens
        report.included.append(str(relative))
        suffix = resolved.suffix.lstrip(".")
        parts.append(f"### File: {relative}\n\n```{suffix}\n{content}\n```\n")

    return "\n".join(parts), report


context, report = load_codebase("./src")

# Review the exclusions before sending anything.
for path, reason in report.skipped:
    print(f"skipped {path}: {reason}")
print(f"{len(report.included)} files, ~{report.estimated_tokens} tokens")

prompt = f"""Analyze this codebase for security vulnerabilities.

{context}

Provide a comprehensive security audit report."""

model = genai.GenerativeModel("gemini-3.1-pro-preview")
response = model.generate_content(prompt)

Why: A recursive load by file extension follows symlinks out of the tree, picks up .env files and key material committed by accident, and grows without bound. The failure is silent and one-way: once the content is in a request, you cannot recall it.

A secret scan is a backstop, not a control: it catches known shapes only. You MUST NOT rely on it in place of keeping credentials out of the repository.

Before the first request you MUST confirm that the platform’s data terms match the classification of the code. Free-tier Gemini API traffic MAY be used to improve Google’s products; paid-tier traffic MAY NOT⁸. Logs are retained for up to 55 days by default, and any log you contribute to a shared dataset is processed under the unpaid-services terms¹⁶. Use Vertex AI for code you cannot expose to those terms; see Data Handling.

Structured Context Organization

def create_structured_context(
    files: dict[str, str],
    docs: str,
    requirements: list[str]
) -> str:
    """Create well-organized context for better understanding"""

    context = """# Project Context

## Requirements
"""
    for i, req in enumerate(requirements, 1):
        context += f"{i}. {req}\n"

    context += "\n## Documentation\n\n"
    context += docs

    context += "\n## Source Code\n\n"
    for file_path, content in files.items():
        context += f"### {file_path}\n\n```python\n{content}\n```\n\n"

    return context

# Usage
context = create_structured_context(
    files={
        "app/models.py": models_code,
        "app/views.py": views_code,
        "app/tests.py": tests_code
    },
    docs=readme_content,
    requirements=[
        "Add user authentication",
        "Implement rate limiting",
        "Add comprehensive logging"
    ]
)

Context Caching

Context caching¹⁷ reuses large context prefixes, cutting the price of repeated input tokens and improving latency.

How It Works:

1. Implicit caching is enabled by default for Gemini 2.5 and newer models, in both stateful and stateless conversation modes; savings are passed on automatically when a request hits the cache¹⁷
2. Minimum input for a cache hit: 4,096 tokens on Gemini 3.x models and 2,048 tokens on Gemini 2.5 models¹⁷
3. Cached input tokens on gemini-3.8-flash cost $0.075/1M against $0.75/1M for uncached input, a 90% discount on the cached portion⁸
4. Explicit caches add a storage charge of $0.50 per 1M tokens per hour, so a cache you never reuse costs more than no cache at all⁸
5. usage.total_cached_tokens on the response reports how many tokens hit the cache¹⁷

To raise the implicit hit rate you SHOULD place large, stable content at the start of the prompt and send similar-prefix requests close together in time¹⁷.

Why: Implicit caching needs no code, and prefix ordering is the only lever that affects it. Explicit caches only pay for themselves once reuse exceeds the hourly storage charge.

Explicit caching pins a prefix for a chosen TTL:

from google.generativeai import caching
import datetime

# Create cached content
cache = caching.CachedContent.create(
    model='models/gemini-3.8-flash',
    system_instruction="""You are a code review expert familiar with this codebase.""",
    contents=[{
        'role': 'user',
        'parts': [{
            'text': codebase_context  # Large codebase context
        }]
    }],
    ttl=datetime.timedelta(hours=1),
)

# Use cached context
model = genai.GenerativeModel.from_cached_content(cache)

# First query (uses cache)
response1 = model.generate_content(
    "Review the authentication module for security issues"
)

# Second query (reuses cache, much cheaper)
response2 = model.generate_content(
    "Check for SQL injection vulnerabilities"
)

# Third query (still using cache)
response3 = model.generate_content(
    "Analyze error handling patterns"
)

# Delete cache when done (optional - will expire after TTL)
cache.delete()

Cost Example (gemini-3.8-flash, introductory rates verified 8 September 2026⁸):

• Codebase: 500K tokens
• First request: 500K tokens × $0.75/1M = $0.375
• Cached requests: 500K tokens × $0.075/1M = $0.0375 (90% off the input price)
• 10 queries: $0.375 + (9 × $0.0375) = $0.7125
• Explicit cache storage for one hour: 500K tokens × $0.50/1M/hour = $0.25
• Without caching: 10 × $0.375 = $3.75
• Savings: 81% on tokens, 74% once the hour of cache storage is included

Long Context Strategies

1. Progressive Context Building

For very large codebases, build context progressively:

# Phase 1: High-level overview
overview_prompt = """Analyze this project structure and identify the main components:

{directory_tree}
{readme_content}
"""

overview = model.generate_content(overview_prompt)

# Phase 2: Detailed analysis of specific areas
detail_prompt = f"""Based on this overview:
{overview.text}

Now analyze these specific files in detail:
{relevant_files_context}
"""

detailed_analysis = model.generate_content(detail_prompt)

2. Hierarchical Context

def create_hierarchical_context(project_root: str) -> str:
    """Create context with hierarchical detail levels"""

    context = "# Codebase Overview\n\n"

    # Level 1: Directory structure
    context += "## Structure\n"
    context += get_directory_tree(project_root)

    # Level 2: File summaries
    context += "\n## File Summaries\n"
    for file in get_source_files(project_root):
        summary = get_file_summary(file)  # First 10 lines + function signatures
        context += f"### {file}\n{summary}\n\n"

    # Level 3: Full code for key files
    context += "\n## Key Files (Full Content)\n"
    key_files = ["main.py", "models.py", "api.py"]
    for file in key_files:
        content = read_file(file)
        context += f"### {file}\n```python\n{content}\n```\n\n"

    return context

3. Selective Context

def get_relevant_context(query: str, codebase_index: dict) -> str:
    """Retrieve only relevant files based on query"""

    # Use semantic search or keyword matching to find relevant files
    relevant_files = search_codebase(query, codebase_index)

    context = "# Relevant Code\n\n"
    for file, relevance_score in relevant_files[:20]:  # Top 20 files
        content = read_file(file)
        context += f"### {file} (relevance: {relevance_score:.2f})\n"
        context += f"```\n{content}\n```\n\n"

    return context

Context Window Limitations

What Works Well

1. Code Analysis (up to 1M lines)
2. Documentation Review (thousands of pages)
3. Log Analysis (gigabytes of logs)
4. Multi-file Refactoring
5. Comprehensive Testing

What Doesn’t Work Well

1. Extremely Dense Technical Content - Model may struggle with highly compressed information
2. Repetitive Content - Redundant information wastes context
3. Binary or Encoded Data - Not suitable for large context
4. Poorly Structured Data - Difficult to extract value

Best Practices

You MUST:

• Structure context clearly with headers and sections
• Include file paths and line numbers for code
• Remove irrelevant files (build artifacts, dependencies)
• Use markdown formatting for readability

You SHOULD:

• Prioritize important files at the beginning
• Include README and architecture docs early
• Use context caching for repeated queries
• Compress whitespace and empty lines

You SHOULD NOT:

• Include generated code (build outputs)
• Include binary files or images as text
• Include entire dependency code
• Duplicate information unnecessarily

Cost Optimization

Model Selection for Cost

Choose the right model based on task complexity:

def select_model(task_type: str, context_size: int) -> str:
    """Select a cost-effective model for a task.

    Model IDs verified 2026-09-08; re-check the deprecations page before
    pinning these in production.
    """
    if context_size > 1_000_000:
        raise ValueError(
            "No current Gemini model accepts more than 1M input tokens; "
            "split, summarize, or retrieve selectively"
        )

    if task_type in ["simple_qa", "classification", "extraction"]:
        return "gemini-3.8-flash"  # Lowest cost per token

    if task_type in ["complex_reasoning", "architecture", "refactoring"]:
        if context_size > 100_000:
            return "gemini-3.1-pro-preview"  # Preview: keep a Flash fallback
        return "gemini-3.8-flash"  # Try Flash first

    return "gemini-3.8-flash"  # Default to Flash

Context Caching for Savings

Rule of Thumb: implicit caching already applies to every Gemini 3.x request above 4,096 tokens. Create an explicit cache only when the same prefix is reused often enough to beat the hourly storage charge.

# Break-even analysis (gemini-3.8-flash, rates verified 2026-09-08)
# Uncached input:  500K tokens x $0.75/1M  = $0.375
# Cached input:    500K tokens x $0.075/1M = $0.0375
# Cache storage:   500K tokens x $0.50/1M/hour = $0.25 per hour held
# Savings per cached request: $0.375 - $0.0375 = $0.3375
# Break-even: 1 extra request per hour the cache is held

CACHE_MIN_TOKENS = 4_096  # Gemini 3.x implicit cache minimum


def should_use_caching(
    context_size: int,
    expected_queries: int,
    cache_hours: float = 1.0,
    input_cost_per_1m: float = 0.75,
    cached_cost_per_1m: float = 0.075,
    storage_cost_per_1m_hour: float = 0.50,
) -> bool:
    """Determine whether an explicit cache is cheaper than repeating input."""
    if context_size < CACHE_MIN_TOKENS:
        return False  # Below the documented cache minimum

    if expected_queries < 2:
        return False  # Nothing to reuse

    millions = context_size / 1_000_000
    without_cache = expected_queries * millions * input_cost_per_1m
    with_cache = (
        millions * input_cost_per_1m  # First request populates the cache
        + (expected_queries - 1) * millions * cached_cost_per_1m
        + millions * storage_cost_per_1m_hour * cache_hours
    )

    return with_cache < without_cache

Prompt Engineering for Efficiency

1. Be Specific to Reduce Output Tokens

# EXPENSIVE - Generates long response
prompt = "Tell me about this codebase."

# CHEAPER - Targeted response
prompt = """List the 5 most critical security issues in this codebase.
For each, provide:
1. Issue description (1 line)
2. Location (file:line)
3. Fix (1 line)

Use this format:
- Issue | Location | Fix
"""

2. Use Structured Output

# Request JSON for predictable token usage
prompt = """Analyze code quality. Return JSON only:
{
    "score": 0-100,
    "issues": [
        {"severity": "HIGH|MED|LOW", "description": "...", "file": "...", "line": 0}
    ],
    "summary": "One sentence"
}
"""

3. Avoid Unnecessary Examples

# EXPENSIVE - Many examples in every request
prompt = f"""Refactor this code. Follow these examples:
{example1}
{example2}
{example3}

Code to refactor:
{code}
"""

# CHEAPER - Use system instructions for examples (one-time cost)
system_instruction = f"""When refactoring, follow these patterns:
{example1}
{example2}
"""

model = genai.GenerativeModel(
    "gemini-3.8-flash",
    system_instruction=system_instruction
)

# Now prompts can be simple
prompt = f"Refactor this code:\n{code}"

Batch Processing

Group requests to amortize fixed costs:

async def batch_analyze_files(files: list[str], batch_size: int = 10):
    """Analyze files in batches to reduce overhead"""

    results = []

    for i in range(0, len(files), batch_size):
        batch = files[i:i + batch_size]

        # Single request for batch
        prompt = "Analyze these files for bugs:\n\n"
        for j, file_path in enumerate(batch, 1):
            content = read_file(file_path)
            prompt += f"## File {j}: {file_path}\n```\n{content}\n```\n\n"

        prompt += "\nReturn JSON array: [{\"file\": \"...\", \"issues\": [...]}]"

        response = await model.generate_content_async(prompt)
        results.extend(json.loads(response.text))

    return results

Rate Limiting and Quotas

Google AI Studio quotas (free tier)²:

• 15 requests per minute
• 1,500 requests per day
• 1 million tokens per minute

Vertex AI quotas (default, can request increases)³:

• 300 requests per minute
• 10,000 requests per hour
• 4 million tokens per minute

Implement rate limiting:

import time
from collections import deque

class RateLimiter:
    def __init__(self, requests_per_minute: int = 60):
        self.requests_per_minute = requests_per_minute
        self.requests = deque()

    def wait_if_needed(self):
        now = time.time()
        minute_ago = now - 60

        # Remove requests older than 1 minute
        while self.requests and self.requests[0] < minute_ago:
            self.requests.popleft()

        # Check if we need to wait
        if len(self.requests) >= self.requests_per_minute:
            sleep_time = 60 - (now - self.requests[0])
            if sleep_time > 0:
                time.sleep(sleep_time)

        self.requests.append(time.time())

# Usage
limiter = RateLimiter(requests_per_minute=50)

for prompt in prompts:
    limiter.wait_if_needed()
    response = model.generate_content(prompt)

Common Pitfalls

Safety Settings Issues

Problem: Requests blocked by safety filters

# Default safety settings may block legitimate requests
response = model.generate_content("Analyze this SQL injection vulnerability")
# May be blocked as "dangerous content"

Solution: Configure safety settings appropriately. The HarmCategory and HarmBlockThreshold enums come from the SDK’s types module; see the safety settings guide¹⁸.

from google.generativeai.types import HarmCategory, HarmBlockThreshold

model = genai.GenerativeModel(
    "gemini-3.8-flash",
    safety_settings={
        HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
        HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    }
)

# For security analysis, you may need BLOCK_ONLY_HIGH
# For public-facing apps, use BLOCK_MEDIUM_AND_ABOVE

Best Practice:

• Development: BLOCK_ONLY_HIGH for flexibility
• Production: BLOCK_MEDIUM_AND_ABOVE for safety
• Security tools: BLOCK_NONE with explicit disclaimers

Context Window Misuse

Problem: Exceeding context limits or inefficient context usage

# BAD - Loading unnecessary files
context = load_all_files("./")  # Includes node_modules, .git, etc.

Solution: Load through the audited loader from Effective Context Loading, which allowlists file types, rejects symlinks and paths outside the repository root, skips credential-shaped files, and caps both file size and total tokens.

# GOOD - one guarded loader, and the skip report is reviewed before sending
context, report = load_codebase("./src")

for path, reason in report.skipped:
    print(f"skipped {path}: {reason}")

if report.estimated_tokens > MAX_TOTAL_TOKENS:
    raise RuntimeError("context budget exceeded; narrow the manifest")

response = model.generate_content(f"{context}\n\nReview for bugs.")

You MUST NOT reimplement the filtering per call site. One loader means one place to audit and one place to fix.

Function Calling Errors

Problem: Infinite loops or failed function calls

# BAD - Model keeps calling function that fails
def buggy_function(arg):
    raise Exception("Always fails")

# Model calls function → Error → Model tries again → Error → ...

Solution: Implement retry limits and error feedback

MAX_FUNCTION_RETRIES = 3

def execute_function_with_retry(fc, attempt=1):
    try:
        result = function_map[fc.name](**dict(fc.args))
        return {"success": True, "result": result}
    except Exception as e:
        if attempt >= MAX_FUNCTION_RETRIES:
            return {
                "success": False,
                "error": "max_retries_exceeded",
                "message": f"Function failed after {MAX_FUNCTION_RETRIES} attempts: {str(e)}"
            }

        # Return error to model for adjustment
        return {
            "success": False,
            "error": "function_error",
            "message": str(e),
            "retry_available": True
        }

JSON Mode Mistakes

Problem: Expecting perfect JSON without configuration

# BAD - Hoping for JSON without specifying
prompt = "Return user data as JSON"
response = model.generate_content(prompt)
data = json.loads(response.text)  # May fail

Solution: Use structured output configuration

# GOOD - Explicit JSON mode (when available) or strict prompting
prompt = """Return ONLY valid JSON, no markdown, no explanation:
{
    "name": "...",
    "email": "...",
    "role": "..."
}

Extract user data from: {user_text}
"""

response = model.generate_content(prompt)

# Robust parsing
try:
    # Remove markdown code blocks if present
    text = response.text.strip()
    if text.startswith("```"):
        text = text.split("```")[1]
        if text.startswith("json"):
            text = text[4:]

    data = json.loads(text)
except json.JSONDecodeError as e:
    # Handle parse error
    print(f"JSON parse error: {e}")
    print(f"Response was: {response.text}")

Streaming Complications

Problem: Improper handling of streaming responses

# BAD - Not handling streaming properly
response = model.generate_content(prompt, stream=True)
print(response.text)  # Error! No .text attribute on streaming response

Solution: Iterate over streaming chunks¹⁹

# GOOD - Proper streaming
response = model.generate_content(prompt, stream=True)

full_response = []
for chunk in response:
    if chunk.text:
        print(chunk.text, end='', flush=True)
        full_response.append(chunk.text)

complete_text = ''.join(full_response)

For Function Calling with Streaming:

response = model.generate_content(prompt, stream=True)

function_calls = []
text_parts = []

for chunk in response:
    # Test the populated field, not attribute existence: every Part exposes
    # both `function_call` and `text` attributes.
    if chunk.candidates[0].content.parts:
        for part in chunk.candidates[0].content.parts:
            if "function_call" in part:
                function_calls.append(part.function_call)
            elif "text" in part:
                text_parts.append(part.text)

# Text and tool calls can arrive in the same response; keep both.
complete_text = ''.join(text_parts)

if function_calls:
    # Execute functions and continue conversation
    ...

Do and Don’t Examples

Prompt Engineering

DO: Be Specific and Structured

# GOOD
prompt = """Refactor this Python function following these requirements:

1. Convert to async/await
2. Add type hints
3. Add error handling for network failures
4. Add docstring with Google style

Original function:
```python
def fetch_data(url):
    response = requests.get(url)
    return response.json()
```

Return only the refactored code with brief explanation of changes.
"""

DON’T: Be Vague

# BAD
prompt = "Make this code better: {code}"

DO: Provide Examples for Complex Tasks

# GOOD
prompt = """Convert these API endpoint descriptions to OpenAPI spec.

Example:
Input: "GET /users/{id} - Returns user by ID"
Output:
```yaml
/users/{id}:
  get:
    summary: Get user by ID
    parameters:
      - name: id
        in: path
        required: true
        schema:
          type: integer
```

Now convert:
{api_descriptions}
"""

DON’T: Assume Model Knows Your Format

# BAD
prompt = f"Convert to OpenAPI: {descriptions}"
# Model may use different OpenAPI version or structure

API Usage

DO: Handle Errors Gracefully

# GOOD
import google.api_core.exceptions

try:
    response = model.generate_content(prompt)
    print(response.text)
except google.api_core.exceptions.ResourceExhausted:
    print("Quota exceeded. Please wait and retry.")
    # Implement exponential backoff
except google.api_core.exceptions.InvalidArgument as e:
    print(f"Invalid request: {e}")
    # Log and fix the request
except Exception as e:
    print(f"Unexpected error: {e}")
    # Log for investigation

DON’T: Ignore Error Handling

# BAD
response = model.generate_content(prompt)
print(response.text)
# Will crash on any error

DO: Use Streaming for Long Responses

# GOOD - Streaming for better UX
response = model.generate_content(long_prompt, stream=True)

for chunk in response:
    print(chunk.text, end='', flush=True)
    # User sees progressive output

DON’T: Block on Large Responses

# BAD - User waits 30 seconds for complete response
response = model.generate_content(long_prompt)
# ... long wait ...
print(response.text)

Function Calling

DO: Constrain Database Tools by Privilege, Not by String Check

# GOOD - allowlisted query text, bound parameters, read-only role
def execute_database_query(query_name: str, params: dict) -> dict:
    sql = ALLOWED_QUERIES.get(query_name)
    if sql is None:
        return {
            "success": False,
            "error": "unknown_query",
            "allowed": sorted(ALLOWED_QUERIES),
        }

    try:
        with connect_read_only() as conn, conn.cursor() as cur:
            cur.execute("SET LOCAL statement_timeout = 10000")
            cur.execute(sql, params)
            rows = cur.fetchmany(MAX_ROWS + 1)
    except TimeoutError:
        return {"success": False, "error": "Query timeout after 10 seconds"}

    return {
        "success": True,
        "data": rows[:MAX_ROWS],
        "truncated": len(rows) > MAX_ROWS,
    }

See Function Calling Best Practices for the full pattern, including the GRANT/REVOKE statements that make the read-only boundary real.

DON’T: Treat a SELECT Prefix as a Read-Only Boundary

# BAD - accepts "SELECT 1; DROP TABLE users"
def execute_database_query(sql: str, database: str) -> dict:
    if not sql.strip().upper().startswith('SELECT'):
        return {"success": False, "error": "Only SELECT queries allowed"}
    return {"success": True, "data": execute(sql, database)}

DON’T: Execute Unchecked Functions

# BAD - Security nightmare
def execute_code(code: str):
    return eval(code)  # Never do this!

DO: Provide Rich Error Context

# GOOD
def search_codebase(pattern: str, file_type: str = None) -> dict:
    try:
        results = perform_search(pattern, file_type)
        return {
            "success": True,
            "matches": len(results),
            "results": results[:50]  # Limit results
        }
    except re.error as e:
        return {
            "success": False,
            "error": "invalid_regex",
            "message": f"Invalid regex pattern: {str(e)}",
            "suggestion": "Try a simpler pattern or escape special characters"
        }
    except FileNotFoundError:
        return {
            "success": False,
            "error": "directory_not_found",
            "message": "Source directory not found",
            "suggestion": "Check that the project is properly initialized"
        }

DON’T: Return Generic Errors

# BAD
def search_codebase(pattern: str) -> dict:
    try:
        return perform_search(pattern)
    except:
        return {"error": "Something went wrong"}

Context Management

DO: Organize Context Hierarchically

# GOOD
context = f"""# Project: E-commerce API

## Architecture
{architecture_doc}

## Key Files

### Core Models
{models_code}

### API Endpoints
{api_code}

### Database Schema
{schema_sql}

## Configuration
{config_yaml}

## Recent Changes
{git_log}
"""

DON’T: Dump Unstructured Content

# BAD
context = all_files_concatenated
# Model struggles to understand structure

DO: Use Context Caching for Repeated Queries

# GOOD - Cache large static context
cache = caching.CachedContent.create(
    model='models/gemini-3.8-flash',
    contents=[{
        'role': 'user',
        'parts': [{'text': large_codebase_context}]
    }],
    ttl=datetime.timedelta(hours=1),
)

model = genai.GenerativeModel.from_cached_content(cache)

# Multiple queries reuse cached context
for query in queries:
    response = model.generate_content(query)

DON’T: Repeat Large Context on Every Request

# BAD - Paying full price every time
for query in queries:
    prompt = f"{large_codebase_context}\n\nQuery: {query}"
    response = model.generate_content(prompt)
    # Expensive!

Error Handling

DO: Implement Exponential Backoff

# GOOD
import random
import time

import google.api_core.exceptions

MAX_BACKOFF_SECONDS = 60.0


def generate_with_retry(prompt: str, max_retries: int = 3):
    """Retry only on transient failures, with capped exponential backoff."""
    for attempt in range(max_retries):
        try:
            return model.generate_content(prompt)
        except (
            google.api_core.exceptions.ResourceExhausted,
            google.api_core.exceptions.ServiceUnavailable,
            google.api_core.exceptions.DeadlineExceeded,
        ) as exc:
            if attempt == max_retries - 1:
                raise
            # Full jitter, capped: spreads retries instead of synchronizing
            # every client onto the same second.
            delay = min(MAX_BACKOFF_SECONDS, 2 ** attempt) * random.random()
            print(f"{type(exc).__name__}. Retrying in {delay:.1f}s...")
            time.sleep(delay)

    raise RuntimeError("unreachable: loop either returns or raises")

You MUST import every module a snippet uses, you MUST retry only retryable status codes, and you MUST cap the delay.

Why: The earlier form of this example called random.random() while importing only time, so the first rate-limit response raised NameError: name 'random' is not defined — the recovery path failed harder than the failure it was handling. Retrying non-transient errors such as InvalidArgument wastes quota, and uncapped exponential backoff can push a delay past any sensible request deadline.

DON’T: Retry Immediately Without Backoff

# BAD
for attempt in range(3):
    try:
        return model.generate_content(prompt)
    except:
        continue  # Immediate retry hammers the API

Advanced Techniques

Multi-Turn Conversations

Maintain conversation state for complex interactions:

model = genai.GenerativeModel("gemini-3.8-flash")
chat = model.start_chat(history=[])

# Turn 1
response1 = chat.send_message("Analyze this function for bugs: {code}")
print(response1.text)

# Turn 2 - Model remembers previous context
response2 = chat.send_message("Now refactor it to fix those bugs")
print(response2.text)

# Turn 3 - Model remembers both previous turns
response3 = chat.send_message("Add unit tests for the refactored version")
print(response3.text)

# Access conversation history
for message in chat.history:
    print(f"{message.role}: {message.parts[0].text[:100]}...")

Code Execution

Gemini can execute Python code²⁰ (Vertex AI only):

from vertexai.preview.generative_models import Tool

# Enable code execution
code_execution_tool = Tool.from_code_execution()

model = GenerativeModel(
    "gemini-3.1-pro-preview",
    tools=[code_execution_tool]
)

response = model.generate_content(
    """Calculate the first 10 Fibonacci numbers and plot them.
    Show both the numbers and a line graph."""
)

print(response.text)
# Model writes Python code, executes it, and returns results + visualization

Multimodal Capabilities

Process images, videos, and audio alongside text²¹:

import PIL.Image

# Analyze code screenshot
image = PIL.Image.open("screenshot.png")

response = model.generate_content([
    "What does this code do? Identify any bugs.",
    image
])

# Analyze architecture diagram
diagram = PIL.Image.open("architecture.png")

response = model.generate_content([
    "Convert this architecture diagram to a text description and mermaid diagram",
    diagram
])

# Analyze video walkthrough
video_file = genai.upload_file("code_review.mp4")

response = model.generate_content([
    "Summarize the key points from this code review video",
    video_file
])

Controlled Generation

Control output format and structure²²:

# JSON schema for structured output
json_schema = {
    "type": "object",
    "properties": {
        "vulnerabilities": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "severity": {"type": "string", "enum": ["CRITICAL", "HIGH", "MEDIUM", "LOW"]},
                    "category": {"type": "string"},
                    "location": {"type": "string"},
                    "description": {"type": "string"},
                    "fix": {"type": "string"}
                },
                "required": ["severity", "category", "location", "description", "fix"]
            }
        },
        "summary": {"type": "string"}
    },
    "required": ["vulnerabilities", "summary"]
}

prompt = f"""Analyze this code for security vulnerabilities.

Return a JSON object matching this schema:
{json.dumps(json_schema, indent=2)}

Code:
{code}
"""

Security and Privacy

Data Handling

Google’s Data Usage Policies:

Google AI Studio:

• Prompts and responses MAY be used to improve models
• NOT suitable for sensitive or private data
• No data residency guarantees
• Not covered by Google Cloud SLAs

Vertex AI:

• Your data is NOT used to train models (with standard agreement)
• Suitable for sensitive business data
• Data residency controls available
• Covered by Google Cloud SLAs and compliance certifications

You MUST:

• Use Vertex AI for production and sensitive data
• Never include PII, credentials, or secrets in prompts
• Implement data sanitization before sending to API
• Review Google’s data usage policies for your use case

You SHOULD:

• Use environment variables for all API keys
• Implement access controls on API endpoints
• Log API usage for audit trails
• Redact sensitive information from logs

API Key Management

The key you manage MUST be an authorization key bound to a service account, not a standard key: the Gemini API rejects standard keys from September 2026⁹. See Authentication for the migration checklist.

# DO: Use environment variables
import os

import requests

api_key = os.getenv("GEMINI_API_KEY") or os.getenv("GOOGLE_API_KEY")
if not api_key:
    raise ValueError("GEMINI_API_KEY environment variable not set")

# DO: Use secret management services
from google.cloud import secretmanager

client = secretmanager.SecretManagerServiceClient()
name = "projects/PROJECT_ID/secrets/gemini-api-key/versions/latest"
response = client.access_secret_version(request={"name": name})
api_key = response.payload.data.decode("UTF-8")

# DO: Send the key in a header when calling REST directly
requests.post(
    "https://generativelanguage.googleapis.com/v1beta/models/"
    "gemini-3.8-flash:generateContent",
    headers={"x-goog-api-key": api_key, "Content-Type": "application/json"},
    json={"contents": [{"parts": [{"text": "Explain recursion"}]}]},
    timeout=30,
)

# DON'T: Put the key in the URL; query strings land in proxy and access logs
url = f"https://generativelanguage.googleapis.com/v1beta/models/m:generateContent?key={api_key}"

# DON'T: Hardcode API keys
api_key = "AIzaSy..."  # Never do this!

# DON'T: Commit to version control
# .env file with API_KEY=... committed to git  # Never!

Content Filtering

Configure appropriate safety settings¹⁸ for your use case:

from google.generativeai.types import HarmCategory, HarmBlockThreshold

# For public-facing applications
public_safety_settings = {
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
}

# For security analysis tools
security_tool_settings = {
    HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_ONLY_HIGH,
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
}

model = genai.GenerativeModel(
    "gemini-3.8-flash",
    safety_settings=public_safety_settings
)

Compliance Considerations

Certifications (Vertex AI)³:

• SOC 2/3
• ISO 27001
• HIPAA (with BAA)
• PCI DSS
• GDPR compliant

Required for Compliance:

1. Use Vertex AI (not Google AI Studio)
2. Enable VPC Service Controls
3. Use customer-managed encryption keys (CMEK)
4. Enable Cloud Audit Logs
5. Implement data retention policies
6. Configure appropriate regional endpoints

Monitoring and Debugging

Response Quality Metrics

Track key quality indicators:

class ResponseMetrics:
    def __init__(self):
        self.total_requests = 0
        self.successful_requests = 0
        self.blocked_by_safety = 0
        self.function_call_errors = 0
        self.latencies = []

    def record_request(
        self,
        success: bool,
        latency_ms: float,
        blocked: bool = False,
        function_error: bool = False
    ):
        self.total_requests += 1
        if success:
            self.successful_requests += 1
        if blocked:
            self.blocked_by_safety += 1
        if function_error:
            self.function_call_errors += 1
        self.latencies.append(latency_ms)

    def get_summary(self):
        return {
            "success_rate": self.successful_requests / self.total_requests,
            "safety_block_rate": self.blocked_by_safety / self.total_requests,
            "function_error_rate": self.function_call_errors / self.total_requests,
            "avg_latency_ms": sum(self.latencies) / len(self.latencies),
            "p95_latency_ms": sorted(self.latencies)[int(len(self.latencies) * 0.95)],
            "p99_latency_ms": sorted(self.latencies)[int(len(self.latencies) * 0.99)],
        }

Performance Monitoring

import time
import logging

def monitored_generate(prompt: str, **kwargs):
    start_time = time.time()

    try:
        response = model.generate_content(prompt, **kwargs)
        latency = (time.time() - start_time) * 1000

        # Log metrics
        logging.info(
            "gemini_request",
            extra={
                "latency_ms": latency,
                "model": model.model_name,
                "prompt_length": len(prompt),
                "response_length": len(response.text),
                "success": True
            }
        )

        return response

    except Exception as e:
        latency = (time.time() - start_time) * 1000

        logging.error(
            "gemini_request_failed",
            extra={
                "latency_ms": latency,
                "error": str(e),
                "error_type": type(e).__name__,
                "success": False
            }
        )
        raise

Debugging Techniques

1. Inspect Full Response

response = model.generate_content(prompt)

# Check safety ratings
for rating in response.candidates[0].safety_ratings:
    print(f"{rating.category}: {rating.probability}")

# Check finish reason
print(f"Finish reason: {response.candidates[0].finish_reason}")
# Possible values:
# - FINISH_REASON_STOP: Natural completion
# - FINISH_REASON_MAX_TOKENS: Hit output limit
# - FINISH_REASON_SAFETY: Blocked by safety filters
# - FINISH_REASON_RECITATION: Blocked for recitation

# Check token usage (if available)
if hasattr(response, 'usage_metadata'):
    print(f"Prompt tokens: {response.usage_metadata.prompt_token_count}")
    print(f"Candidates tokens: {response.usage_metadata.candidates_token_count}")
    print(f"Total tokens: {response.usage_metadata.total_token_count}")

2. Debug Function Calling

# Enable verbose logging
import logging
logging.basicConfig(level=logging.DEBUG)

response = model.generate_content(prompt)

# Inspect function calls
for part in response.candidates[0].content.parts:
    if "function_call" in part:
        fc = part.function_call
        print(f"Function: {fc.name}")
        print(f"Arguments: {dict(fc.args)}")
        print(f"Arguments JSON: {json.dumps(dict(fc.args), indent=2)}")

3. Test Safety Filters

def test_safety_filter(prompt: str):
    try:
        response = model.generate_content(prompt)
        print(f"✓ Passed: {response.text[:100]}...")
    except Exception as e:
        print(f"✗ Blocked: {str(e)}")

    # Check detailed safety ratings
    if 'response' in locals():
        print("\nSafety Ratings:")
        for rating in response.candidates[0].safety_ratings:
            print(f"  {rating.category}: {rating.probability}")

test_safety_filter("Analyze this SQL injection vulnerability: {code}")

References

Additional Resources

Official Documentation:

• Gemini API Reference - Complete API reference
• Generative AI on Vertex AI
  • Enterprise features and capabilities

Security and Compliance:

• Vertex AI Security - Security features and best practices
• Data Usage FAQ - Privacy and data usage policies
• Google Cloud Compliance - Compliance certifications

Best Practices:

• Prompt Engineering Guide - Effective prompting strategies
• System Instructions - Using system instructions
• Safety Settings - Content filtering configuration
• Function Calling Guide - Function calling documentation

Community and Support:

• Google AI Discord - Community discussions
• Stack Overflow - Q&A tagged with google-gemini
• GitHub Discussions - Python SDK discussions

Document Version: 1.0 Last Updated: December 2024 Status: Active