GCP Management Tool - User Guide

Manage GCP resources efficiently using declarative YAML configurations and imperative CLI commands. This guide provides a comprehensive overview of the process.

User Guide: GCP Configuration with DevOps-Bot

This guide walks you through the process of configuring DevOps-Bot to work with Google Cloud Platform (GCP). Follow the steps below to ensure a successful configuration.

Pre-requisites

Ensure DevOps-Bot is Installed:

Before proceeding, make sure that DevOps-Bot (dobe) is installed on your instance. Refer to the DevOps-Bot Installation Guide for detailed installation instructions.

GCP Project:

Verify that you have a GCP project created and accessible.

Step 1: Generate GCP Credentials

Log into GCP Console:

Go to the Google Cloud Console. Navigate to the IAM & Admin section.

Create a Service Account:

Go to IAM & Admin > Service Accounts.
Click "Create Service Account."
Provide the following details:
- Name: Enter a descriptive name (e.g., DevOps-Bot).
- Description: Provide a brief description (optional).
Click "Create and Continue."

Assign Roles:

On the next screen, assign appropriate roles (e.g., Editor or Custom Role based on your requirements).
Click "Continue" and "Done."

Generate and Download JSON Key:

Go to the Service Accounts list and find the account you created.
Click the three-dot menu and select "Manage Keys."
Click "Add Key" > "Create New Key" and select JSON format.
Download the JSON file to your local machine.

Step 2: Configure GCP in DevOps-Bot

Transfer the JSON File to Your Instance:

Copy the JSON file with GCP credentials to your instance where DevOps-Bot is installed.

Initialize DevOps-Bot:

Run the following command to initialize the tool and create necessary files:

dob brood --init

Configure GCP Credentials:

Use the dob gcp config command to configure GCP with the JSON file:

dob gcp config --file-path <path_to_json_file>

Example:

dob gcp config --file-path /root/gcp_credentials.json

If the credentials are valid, you’ll see the following messages:

GCP credentials encrypted and saved locally.
GCP credentials configured successfully. You can delete the original JSON file for security.

Step 3: Test GCP Configuration

Create a YAML Configuration File:

Create a file named compute_instance.yaml with the following content:


resources:
  compute_instances:
    - name: test-compute-instance-1
      machine_type: n1-standard-1
      zone: us-central1-a
      image: projects/ubuntu-os-cloud/global/images/family/ubuntu-2004-lts
      network: default
      tags:
        - web-server
      labels:
        env: test

Run the Configuration File:

Use the dob gcp screenplay command to execute the configuration:

dob gcp screenplay compute_instance.yaml

Review and Confirm:

The tool will validate and lint the YAML file. A final review of actions will be displayed. For example:


+----+------------------+---------------------------------------------------------------+
|    | Category         | Value                                                         |
+====+==================+===============================================================+
| +  | Compute Instance | Instance 1                                                    |
+----+------------------+---------------------------------------------------------------+
| +  | Instance Name    | test-compute-instance-1                                       |
+----+------------------+---------------------------------------------------------------+
| +  | Machine Type     | n1-standard-1                                                 |
+----+------------------+---------------------------------------------------------------+
| +  | Zone             | us-central1-a                                                 |
+----+------------------+---------------------------------------------------------------+
| +  | Image            | projects/ubuntu-os-cloud/global/images/family/ubuntu-2004-lts |
+----+------------------+---------------------------------------------------------------+
| +  | Tags             | ['web-server']                                                |
+----+------------------+---------------------------------------------------------------+
| +  | Network          | default                                                       |
+----+------------------+---------------------------------------------------------------+
| +  | Labels           | {'env': 'test'}                                               |
+----+------------------+---------------------------------------------------------------+

When prompted, type Y to confirm and proceed.

Successful Execution:

The tool will create the specified resources in GCP and save the execution state:

Compute Instance 'test-compute-instance-1' created successfully.
SSH key injected into instance 'test-compute-instance-1'.
Instance 'test-compute-instance-1' created with internal IP 10.128.15.230
Execution state saved to: /etc/devops-bot/state_files/<execution_file>.yml

Notes:

Ensure that the service account has adequate permissions for the tasks you want to execute.
Use proper tags and configuration files to ensure efficient resource management.

Key Features

Declarative YAML Support: Use YAML files to describe GCP resources and execute tasks.
Direct CLI Command Support: For quick, ad-hoc resource and task management.
Environment Variables Support: Customize configurations for different environments like dev, test, or prod.
Variable Overrides: Override variables directly via the command line.
Remote Configs: Load YAML files or variables from remote URLs.
SSH Integration: Automatically inject SSH keys and add instance IPs to /etc/hosts.

Declarative YAML Approach

Workflow

Create a YAML Configuration File: Define resources such as VM instances and disks, or specify tasks to execute on remote instances.
Run the Command: Use dob gcp screenplay to validate and execute the YAML file.
Monitor Execution: The tool processes the file, validates variables, and provisions resources.

YAML Configuration Example

Creating a GCP Virtual Machine Instance


resources:
  instances:
    - name: "my-instance"
      machine_type: "n1-standard-1"
      region: "us-central1"
      zone: "us-central1-a"
      disk_size: "50GB"
      disk_type: "pd-standard"
      network: "default"
      subnetwork: "default"
      tags:
        - "web-server"
      metadata:
        startup-script: |
          #!/bin/bash
          echo "Welcome to GCP VM" > /var/www/html/index.html
      labels:
        environment: "production"

Executing Tasks on Remote Instances


resources:
  remote_tasks:
    - instance_name: "my-instance"
      command: "sudo apt update && sudo apt upgrade -y"
      user: "admin"

Command to Execute YAML Configuration


dob gcp screenplay /path/to/config.yaml --environment dev --password mypassword --show

Explanation of YAML Example

Instances:
- name: Name of the GCP instance.
- machine_type: GCP machine type (e.g., n1-standard-1).
- region and zone: Specify where the instance is created.
- disk_size and disk_type: Disk configuration for the instance.
- tags and labels: Tags for network rules and organizational metadata.
Remote Tasks:
- instance_name: Name of the instance to execute the task on.
- command: Command to execute.
- user: The SSH username.

Imperative CLI Commands

General Syntax


dob gcp screenplay [OPTIONS]

Key Options

--screenplay: Path to the YAML file defining resources and tasks.
--identifier or -id: A unique identifier for the instance.
--username or -u: SSH username for the remote instance.
--create-remote or --cr: Inject SSH key into the created instance.
--add-remote or --ar: Add the instance's IP and hostname to /etc/hosts.
--command or -c: Command to execute on the instance.
--environment or -e: Specify the environment (e.g., dev, test, prod).
--remote-config or --rc: URL of a remote YAML configuration file.
--set: Override variables directly from the command line.

Example Commands

Create and Execute Tasks on a VM


dob gcp screenplay /path/to/config.yaml \
  --username admin \
  --create-remote \
  --environment prod \
  --password "mypassword"

Execute a Command on a Remote Instance


dob gcp screenplay --identifier my-instance --command "sudo apt update && sudo apt upgrade -y" --username admin

Use Remote Configuration


dob gcp screenplay --remote-config https://example.com/config.yaml

Override Variables


dob gcp screenplay /path/to/config.yaml --set region=us-east1 zone=us-east1-b

Inject SSH Key and Update /etc/hosts


dob gcp screenplay /path/to/config.yaml --create-remote --add-remote

Key Features Explained

Remote Variables and Configurations:
- Load remote YAML configuration with --remote-config.
- Use --remote-ver for alternate variable files from a URL.
Environment Customization:
- Switch between environments using --environment.
Password and Sensitive Data Handling:
- Decrypt variables with --password and debug them using --show.
Command Overrides:
- Dynamically override variables like --set instance_type=n1-standard-1.
Instance Management:
- Add IPs and hostnames to /etc/hosts with --add-remote.

YAML vs. CLI: When to Use Which

Feature	Declarative YAML	Imperative CLI
Use Case	Complex, multi-resource setups	Quick, one-time operations
Flexibility	Automation-friendly, repeatable	Highly flexible for ad-hoc tasks
Complexity	Best for large-scale, consistent setups	Simpler, but limited for complex setups
Ease of Use	Requires YAML knowledge	Direct execution with less overhead

Best Practices

Always validate YAML files before execution.
Use --show to debug sensitive data while ensuring security.
Combine declarative YAML and imperative commands for robust resource management.

Installation Steps PDF Reference for GCP Configuration

For detailed installation steps, please refer to the PDF documentation: