Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

97 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ybdb-vanguard

Hands-on YugabyteDB exercises covering distributed SQL, data architecture, scalability, fault tolerance, multi-region distribution, disaster recovery, CDC, observability, security, and data migration. Each exercise runs in a fully pre-configured cloud development environment — no local YugabyteDB installation required.

Open in GitHub Codespaces


Table of Contents


Prerequisites

Platform Requirements
GitHub Codespaces GitHub account (free tier: 60 core-hours/month)
VS Code Dev Containers Docker Desktop · VS Code · Dev Containers extension
DevPod Docker Desktop · DevPod CLI

Getting Started

1 · Fork and clone

# Fork via GitHub UI first, then:
git clone https://github.com/<your-github-username>/ybdb-vanguard.git
cd ybdb-vanguard

2 · Pick an exercise

./launch

The launch script presents a categorised menu with role tags and resource warnings. Pick a number — the script prints a Codespaces URL, a DevPod command, and a VS Code Dev Containers instruction for the selected exercise.

3 · Alternative launch paths

Codespaces picker
  1. Navigate to your fork on GitHub
  2. Click Code → Codespaces → New codespace
  3. Select the exercise from the devcontainer configuration dropdown
GitHub CLI
gh codespace create \
  --repo <your-github-username>/ybdb-vanguard \
  --devcontainer-path .devcontainer/init-dsql/devcontainer.json
DevPod
devpod up . --devcontainer-path .devcontainer/init-dsql/devcontainer.json
VS Code Dev Containers

Open Command Palette → Dev Containers: Open Folder in Container… → select the exercise config.


Exercises

Role Badge Legend

Badge Audience
Dev Developer
Ops Operations / DBA
SRE Site Reliability Engineer
Architect Architect

Quick Reference

SQL Fundamentals

# Exercise Description Roles Notes
1 Distributed SQL Universe YSQL & YCQL basics, sharding, tablet distribution Dev Ops SRE Arc
2 Query Tuning Tips & Tricks Index strategies, EXPLAIN ANALYZE, join optimisation Dev Ops Arc
3 Query Plan Management (QPM) Detect plan regressions, pin known-good query plans Dev Ops SRE Arc ev2025.2.3+

Data Placement & Architecture

# Exercise Description Roles Notes
4 Colocation & Distributed Tables Co-locate small reference tables, keep others distributed Dev Arc
5 Tablespaces & Online Data Migration Placement tablespaces and live ALTER SET TABLESPACE Dev Ops Arc

Scalability & HA

# Exercise Description Roles Notes
6 Data Distribution and Scalability Tablet auto-splitting and scale-out under live load Ops SRE Arc ⚠ 8 CPU · 16 GB RAM
7 Fault Tolerance and High Availability Kill nodes, verify Raft replication and zero data loss Ops SRE Arc ⚠ 8 CPU · 16 GB RAM

Multi-Region & Disaster Recovery

# Exercise Description Roles Notes
8 Geo-distribution & Tablespaces Multi-region placement, geo-partitioning, follower reads Dev Ops SRE Arc
9 xCluster Replication & Disaster Recovery xCluster with auto DDL propagation and planned failover Ops SRE Arc ⚠ 8 GB RAM

Data Protection & Recovery

# Exercise Description Roles Notes
10 Point-in-Time Recovery (PITR) Online restore to any second within retention window Ops SRE
11 DB Clone/Branch — Instant Database Copies Instant CREATE DATABASE … AS OF snapshot copy Dev Ops SRE
12 Time Travel Historical read timestamps for forensic queries Dev Ops

Streaming & CDC

# Exercise Description Roles Notes
13 Change Data Capture — YugabyteDB → PostgreSQL YB Debezium connector → PostgreSQL via JDBC sink Dev Ops SRE
14 Change Data Capture — SQL Server → YugabyteDB SQL Server CDC captured into YugabyteDB via Debezium Dev Ops SRE ⚠ 8 CPU · 16 GB RAM

Observability

# Exercise Description Roles Notes
15 Observability & Performance Diagnosis pg_stat_statements, ASH, pg_locks, query diagnostics Ops SRE Arc
16 OpenSearch — Log Observability OTel tails GLog + pgaudit logs into OpenSearch Dev Ops SRE Arc
17 Elasticsearch — Logs & Metrics Observability OTel ships GLog + Prometheus metrics into Elasticsearch Dev Ops SRE Arc

Security

# Exercise Description Roles Notes
18 Encryption at Rest (EAR) + Key Rotation yb-admin key rotation on live cluster, no restart Ops SRE
19 Row Level Security & Multi-tenancy CREATE POLICY tenant isolation with session variables Dev Ops Arc
20 Data Privacy — Column Encryption & Anonymization pgcrypto column encryption, masking, anonymized views Dev Ops Arc

Search & Extensions

# Exercise Description Roles Notes
21 Full-Text Search tsvector/tsquery, ts_rank, ts_headline, ybgin index Dev Arc
22 Semantic Search with pgvector L2/cosine/IP distance operators, ybhnsw ANN index Dev Arc

Migration (YB Voyager)

# Exercise Description Roles Notes
23 Data Migration — MySQL → YugabyteDB Voyager: assess → export → import → finalize Dev Ops
24 Data Migration — MariaDB → YugabyteDB Voyager offline migration from MariaDB Dev Ops
25 Data Migration — Oracle → YugabyteDB Voyager offline migration from Oracle Free container Dev Ops ⚠ 8 CPU · 16 GB RAM · 64 GB disk
26 Live Data Migration — PostgreSQL → YugabyteDB Voyager live migration with CDC stream and cutover Dev Ops SRE

Ecosystem

# Exercise Description Roles Notes
27 Keycloak — Identity & Access Management Keycloak backed by YugabyteDB via smart JDBC driver Dev Arc
28 Kong Gateway — API Gateway Kong config stored in YugabyteDB via PostgreSQL protocol Dev Ops Arc
29 WSO2 API Manager — Enterprise API Gateway WSO2 APIM two-database setup with smart JDBC driver Dev Ops Arc ⚠ 8 CPU · 16 GB RAM

SQL Fundamentals

Distributed SQL Universe

Dev Ops Architect SRE

README → | devcontainer: init-dsql

Get started with YugabyteDB: hash vs range sharding, YSQL and YCQL basics, tablet distribution, and fault-tolerance fundamentals.


Query Tuning Tips & Tricks

Dev Ops Architect

README → | devcontainer: init-qt

Query execution patterns, pushdown operations, index strategies (hash, range, covering, partial, expression), join optimisation, advanced SQL, and programmability — all with EXPLAIN (ANALYZE, DIST).


Query Plan Management (QPM)

Dev Ops SRE Architect

README → | devcontainer: init-qpm

Detect, compare, and pin query plans with QPM (EA, v2025.2.3+). Capture every plan a query has used in yb_pg_stat_plans, spot a regression after a statistics change, and pin a known-good plan via the pg_hint_plan hint table. The database is fully QPM-ready (extensions, defaults, and seed data) the moment the container starts.


Data Placement & Architecture

Colocation & Distributed Tables

Dev Architect

README → | devcontainer: init-colocate

Co-locate small reference tables on a single shared tablet for local joins while keeping high-volume tables distributed. Covers CREATE DATABASE ... WITH COLOCATION = true, WITH (COLOCATION = false) opt-out, yb_table_properties(), and yb_is_database_colocated().


Tablespaces & Online Data Migration

Dev Ops Architect

README → | devcontainer: init-tablespace

Create placement-aware tablespaces with replica_placement JSON, pin tables and indexes to specific regions at creation time, and migrate data between tablespaces online with ALTER TABLE SET TABLESPACE — no cluster downtime. Covers ALTER INDEX SET TABLESPACE and SET default_tablespace.


Scalability & High Availability

Data Distribution and Scalability

Ops SRE Architect

README → | devcontainer: init-scale

Explore tablet-based data distribution, automatic tablet splitting, and horizontal scale-out on a 3-node cluster under yb-sample-apps load.

Note: Requires 8 CPUs · 16 GB RAM.


Fault Tolerance and High Availability

Ops SRE Architect

README → | devcontainer: init-ft

Chaos engineering on a 6-node cluster across 3 availability zones. Kill nodes, observe leader election and Raft re-replication, and verify zero data loss under continuous YSQL and YCQL load.

Note: Requires 8 CPUs · 16 GB RAM.


Multi-Region & Disaster Recovery

Geo-distribution & Tablespaces

Dev Ops SRE Architect

README → | devcontainer: init-geo

Multi-region data placement and low-latency reads on a 3-node cluster simulating US East / EU West / AP South. Covers CREATE TABLESPACE with replica_placement JSON, region-pinned tables, row-level geo-partitioning, yb_is_local_table, preferred zone configuration, and follower reads.


xCluster Replication & Disaster Recovery

Ops SRE Architect

README → | devcontainer: init-xcluster

Set up transactional xCluster replication with automatic DDL propagation between two universes (v2025.2.1+). DDL runs only on the primary — the standby receives schema changes automatically. Covers create_xcluster_checkpoint, PITR on standby, setup_xcluster_replication, role verification (yb_xcluster_ddl_replication.get_replication_role()), lag monitoring, and planned failover — all with standalone yb-admin (no YugabyteDB Anywhere required).

Note: Requires 8 GB RAM (two single-node clusters running simultaneously).


Data Protection & Recovery

Point-in-Time Recovery (PITR)

Ops SRE

README → | devcontainer: init-pitr

Create snapshot schedules, simulate accidental DELETE and DROP TABLE disasters, and restore the database online to any second within the retention window — without stopping the cluster. Bonus: SET yb_read_time time-travel queries as a forensics tool.


DB Clone — Instant Database Copies

Dev Ops SRE

README → | devcontainer: init-clone

Clone a live database with a single SQL statement: CREATE DATABASE clone TEMPLATE source [AS OF '<timestamp>']. Test migrations safely, create rollback baselines, and reproduce past states — production is never touched.


Time Travel — yb_read_time

Dev Ops

README → | devcontainer: init-tt

Read historical snapshots of your data by setting a session-level read timestamp (SET yb_read_time TO <unix_microseconds>). Audit what changed, find deleted rows, run forensic investigations — the live database is never modified.


Streaming & CDC

Change Data Capture — YugabyteDB → PostgreSQL

Dev Ops SRE

README → | devcontainer: init-cdc-ybdb-pg

Stream changes from YugabyteDB to PostgreSQL using the YugabyteDB Debezium connector (yboutput logical replication plugin) and a JDBC sink connector. Guided demo: register connectors, snapshot, live INSERT/UPDATE/DELETE propagation.


Change Data Capture — SQL Server → YugabyteDB

Dev Ops SRE

README → | devcontainer: init-cdc-sqlserver-ybdb

Stream changes from Microsoft SQL Server to YugabyteDB using the Debezium SQL Server connector (capturing SQL Server's native CDC change tables) and a Debezium JDBC sink connector. Guided demo: register connectors, snapshot, live INSERT/UPDATE/DELETE propagation.

Note: Requires 8 CPUs · 16 GB RAM.


CDC Streaming — YSQL → YCQL

Dev

Repository →

Spring Cloud Stream microservices-based CDC from YSQL to YCQL through a supplier-processor-consumer pattern (external repo).


Observability

Observability & Performance Diagnosis

Ops SRE Architect

README → | devcontainer: init-obs

End-to-end performance investigation using built-in YugabyteDB SQL views: pg_stat_statements (with docdb_rows_scanned, yb_latency_histogram, P99), Active Session History (yb_active_session_history) grouped by query / tablet / node / session, EXPLAIN (ANALYZE, DIST), pg_locks, yb_cancel_transaction(), and yb_query_diagnostics. No external agents or dashboards required.


OpenSearch — Log Observability with YugabyteDB

Dev Ops SRE Architect

README → | devcontainer: init-opensearch

Ships YugabyteDB structured logs into OpenSearch using the OpenTelemetry Collector contrib binary. The OTel Collector runs as a process inside the devcontainer (not a sibling Docker container) so it can read YB GLog files directly from the filesystem. A filelog receiver tails master and tserver .INFO log files with GLog multiline parsing; a second filelog receiver captures PostgreSQL and pgaudit log entries. Logs flow to the yb-logs index. OpenSearch Dashboards provides search, index-pattern exploration, and visualization.


Elasticsearch — Logs & Metrics Observability with YugabyteDB

Dev Ops SRE Architect

README → | devcontainer: init-elasticsearch

Ships YugabyteDB structured logs and Prometheus metrics into Elasticsearch using the OpenTelemetry Collector contrib binary. Two filelog receivers handle GLog (master/tserver) and PostgreSQL/pgaudit log files; a prometheus receiver scrapes four YB metrics endpoints (master :7000, tserver :9000, YSQL :13000, YCQL :12000) every 15 s. Logs flow to the yb-logs index; metrics flow to the yb-metrics index via the elasticsearch/metrics exporter with mapping.mode: none. Kibana provides index-pattern search and visualization for both signals.


Security

Encryption at Rest (EAR) + Key Rotation

Ops SRE

README → | devcontainer: init-ear

Enable and rotate cluster-level encryption at rest on a live YugabyteDB node — no restart required. Covers openssl rand key generation, yb-admin add_universe_key_to_all_masters, rotate_universe_key_in_memory, is_encryption_enabled, and a full quarterly key-rotation workflow.


Row Level Security & Multi-tenancy

Dev Ops Architect

README → | devcontainer: init-rls

Database-enforced tenant isolation. Covers CREATE POLICY with USING and WITH CHECK, session-variable multi-tenancy (SET app.tenant_id, current_setting()), BYPASSRLS for admin roles, SECURITY DEFINER functions, partial index optimisation for RLS predicates, and schema-per-tenant comparison.


Data Privacy — Column Encryption & Anonymization

Dev Ops Architect

README → | devcontainer: init-privacy

PII protection at the column level using the pgcrypto extension. Covers pgp_sym_encrypt / pgp_sym_decrypt (AES via GnuPG), digest (SHA-256 for searchable hashes), hmac (tamper-evident audit logs), masking and pseudonymization patterns, anonymized views, and column-level key rotation.


Search & Extensions

Full-Text Search

Dev Architect

README → | devcontainer: init-fts

SQL-native full-text search without an external search engine. Covers tsvector / tsquery, stemming and stop-word removal, boolean and phrase operators (to_tsquery, phraseto_tsquery, websearch_to_tsquery), relevance ranking (ts_rank), highlighted snippets (ts_headline), persisted tsvector columns, ybgin index for single-term fast lookups, and auto-update with tsvector_update_trigger.


Semantic Search with pgvector

Dev Architect

README → | devcontainer: init-pgvector

Vector similarity search using the bundled pgvector extension. Covers all three distance operators (<-> L2, <=> cosine, <#> inner product), converting distance to similarity, vector magnitude (l2_norm) and normalization (l2_normalize), the normalization identity that makes inner product equivalent to cosine — enabling a vector_ip_ops index as the preferred choice for normalized embeddings, hybrid SQL + vector queries, and ybhnsw approximate nearest neighbor index with ef_search tuning.


Migration (YB Voyager)

Data Migration — MySQL → YugabyteDB

Dev Ops

README → | devcontainer: init-voyager-mysql

Offline migration from MySQL to YugabyteDB using YB Voyager: assess → export schema → analyse → export data → import schema → import data → finalise.


Data Migration — MariaDB → YugabyteDB

Dev Ops

README → | devcontainer: init-voyager-mariadb

Offline migration from MariaDB to YugabyteDB using YB Voyager.


Data Migration — Oracle → YugabyteDB

Dev Ops

README → | devcontainer: init-voyager-oracle

Offline migration from Oracle Database to YugabyteDB using YB Voyager. Uses an Oracle Free container as the source.

Note: Requires 8 CPUs · 16 GB RAM · 64 GB disk — Oracle container is heavyweight.


Live Data Migration — PostgreSQL → YugabyteDB

Dev Ops SRE

README → | devcontainer: init-voyager-postgres

Live (online) migration from PostgreSQL to YugabyteDB with minimal downtime using YB Voyager — export + streaming CDC changes + cutover.


Ecosystem (PG Runtime-Compatible)

Keycloak — Identity & Access Management

Dev Architect

README → | devcontainer: init-keycloak

YugabyteDB as Keycloak's backend identity store using the YugabyteDB smart JDBC driver (com.yugabyte.Driver, jdbc:yugabytedb://). Drop in the YB JDBC JAR and set KC_DB_DRIVER + KC_DB_URL — Keycloak needs no other changes. Covers Liquibase schema migration (~90 tables), real-time YSQL queries alongside Keycloak Admin API calls, realm and user creation via REST, and connection load-balancing across YugabyteDB nodes.


Kong Gateway — API Gateway with YugabyteDB

Dev Ops Architect

README → | devcontainer: init-kong

Kong Gateway stores its entire configuration (services, routes, plugins, consumers) in YugabyteDB. This exercise connects Kong to YugabyteDB YSQL (port 5433) — Kong uses the standard PostgreSQL wire protocol and YugabyteDB speaks it back, requiring zero driver or code changes. Covers Kong bootstrap migrations, creating services and routes via the Admin API, proxying traffic, adding the rate-limiting plugin, and reading every config object back directly from YugabyteDB with YSQL.


WSO2 API Manager — Enterprise API Gateway with YugabyteDB

Dev Ops Architect

⚠ Requires 8 CPU · 16 GB RAM (WSO2 APIM JVM + YugabyteDB)

README → | devcontainer: init-wso2

WSO2 API Manager stores all platform state — API definitions, subscriptions, throttling policies, users, OAuth clients, and OIDC tokens — across two YugabyteDB databases. The standard PostgreSQL JDBC driver is replaced with the YugabyteDB smart JDBC driver (com.yugabyte.Driver, jdbc:yugabytedb://) in deployment.toml, adding built-in connection load-balancing with zero WSO2 code changes. Covers two-database architecture (wso2_amdb for API management, wso2_shareddb for Carbon/Identity), DDL bootstrap from WSO2's shipped SQL scripts, the Publisher REST API, and direct YSQL queries against both databases.


External Resources

Resource Roles Description
Java Microservices Dev Spring Boot, Quarkus, and Micronaut integration patterns with YugabyteDB
Java Testcontainers Dev Testcontainers integration with YugabyteDB for integration testing
Securing Spring Boot Microservices Dev Secure a Spring Boot application with YugabyteDB over TLS using cloud-native secret management
CDC Streaming — YSQL → YCQL Dev Spring Cloud Stream microservices-based CDC from YSQL to YCQL

Devcontainer Reference

Exercise Directory Nodes CPUs RAM Disk
SQL Fundamentals
Distributed SQL init-dsql 3 4 8 GB 32 GB
Query Tuning init-qt 3 4 8 GB 32 GB
Query Plan Management init-qpm 1 4 8 GB 32 GB
Data Placement & Architecture
Colocation init-colocate 3 4 8 GB 32 GB
Tablespaces init-tablespace 3 4 8 GB 32 GB
Scalability & HA
Scalability init-scale 3 (→6) 8 16 GB 32 GB
Fault Tolerance init-ft 6 8 16 GB 32 GB
Multi-Region & DR
Geo-distribution init-geo 3 4 8 GB 32 GB
xCluster Replication init-xcluster 1+1 4 8 GB 32 GB
Data Protection & Recovery
PITR init-pitr 1 4 8 GB 32 GB
DB Clone init-clone 1 4 8 GB 32 GB
Time Travel init-tt 1 4 8 GB 32 GB
Streaming & CDC
CDC (Debezium) init-cdc-ybdb-pg 1 4 8 GB 32 GB
CDC (SQL Server) init-cdc-sqlserver-ybdb 1 8 16 GB 32 GB
Observability
Observability init-obs 3 4 8 GB 32 GB
OpenSearch Observability init-opensearch 1 4 8 GB 32 GB
Elasticsearch Observability init-elasticsearch 1 4 8 GB 32 GB
Security
Encryption at Rest init-ear 1 4 8 GB 32 GB
Row Level Security init-rls 1 4 8 GB 32 GB
Data Privacy init-privacy 1 4 8 GB 32 GB
Search & Extensions
Full-Text Search init-fts 1 4 8 GB 32 GB
pgvector init-pgvector 1 4 8 GB 32 GB
Migration
MySQL Migration init-voyager-mysql 1 4 8 GB 32 GB
MariaDB Migration init-voyager-mariadb 1 4 8 GB 32 GB
Oracle Migration init-voyager-oracle 1 8 16 GB 64 GB
PostgreSQL Migration init-voyager-postgres 1 4 8 GB 32 GB
Ecosystem
Keycloak IAM init-keycloak 1 4 8 GB 32 GB
Kong Gateway init-kong 1 4 8 GB 32 GB
WSO2 API Manager init-wso2 1 8 16 GB 32 GB

All exercises default to Ubuntu-based devcontainer image on linux/amd64 and linux/arm64.


YugabyteDB Version

This repo is pinned to YugabyteDB 2025.2.3.2-b1.

About

YugabyteDB hands on content for dev, ops, security, sre and architect personas

Topics

Resources

Stars

3 stars

Watchers

2 watching

Forks

Contributors

Languages