GCP Professional ML EngineerGCP Generative AI LeaderGCP Professional Cloud ArchitectGCP Associate Cloud EngineerAzure Solutions Architect ExpertAzure Administrator AssociateAWS Certified AI PractitionerAWS Cloud PractitionerISTQB Advanced Test ManagerISTQB Technical Test AnalystISTQB Foundation LevelTMMi ProfessionalProfessional Scrum Master ISecret Level II Clearance
Senior Quality Engineering Leader with 10+ years architecting enterprise testing solutions
across financial services, healthcare, government, and security. I lead distributed teams,
design GenAI validation frameworks, and build multi-agent AI systems — from agentic AI
contact center QA at Fortune 100 scale to reverse-engineering test coverage from undocumented legacy code.
Led enterprise automation program accelerating release cadence from quarterly to bi-weekly — delivering 30,000+ hours in annual productivity gains across 30+ regression cycles.
🤖
500+
Contact Center Agents
Architected GenAI QA for a Fortune 100 healthcare agentic AI contact center on GCP — ran a HIPAA-compliant human-in-the-loop evaluation pipeline across 6,000 production transcripts and evaluated AI judges for continuous model quality assessment.
🔍
100K+
API Validations
Parallel Spock framework executing 100,000+ legacy-to-new API response comparisons in under 5 minutes — uncovering 30+ business rule defects and enabling a zero-defect production release.
🏆
NASTA '22
North American Software Testing Award
Best Use of Technology in Project — built the Python event-simulation framework validating data integrity across Kafka ingress and egress for a 2M+ daily-transaction SAP-to-microservices migration.
💡
3
Specialized AI Agents
Designed a multi-agent AI system that reverse-engineered test coverage from undocumented legacy code — code-analysis, verification, and automation-architect agents enabling a new team to learn what and how to test.
02 / About
About Me
As a Senior Quality Engineering Leader, I lead distributed engineering teams across
geographies — currently leading a QA team of 6 across Poland and India as part of a
strategic programme at a top-5 Canadian bank through Deloitte. I own
test strategy, tooling decisions, vendor coordination, and programme-level quality
governance. Previously, I led a 10-person automation team (full-time and contractors)
delivering 30,000+ hours in annual productivity gains, a 14-member cross-functional
team for a Fortune 100 healthcare GenAI deployment, and a 15+ member team as
Non-Functional Test Manager — creating the performance, disaster recovery, and
failover test plans for Canada's largest social development program and orchestrating
their execution across vendor, client, and architecture teams.
As a Test Architect, I design the frameworks and strategies that
teams execute against. I've architected testing solutions for microservices handling
2M+ daily transactions, enterprise NFT suites for systems processing 50K+ daily
transactions, and end-to-end GenAI validation for agentic AI contact centre platforms
— enabling teams to catch critical defects pre-production through shift-left test architecture and automated quality gates.
As a GenAI QA Strategist, I bring hands-on experience validating production
AI systems at enterprise scale. At a Fortune 100 healthcare organisation, I led
end-to-end quality assurance for an agentic AI contact centre solution built on
GCP and integrated with Genesys CCaaS — designing evaluation
architectures for LLM outputs, evaluating AI judges for continuous model quality
assessment, defining human
annotation programmes, and building feedback loops that translated evaluation findings
into prompt tuning and model optimisation recommendations.
🏆
Industry Award
North American Software Testing Award — Best Use of Technology in Project (2022).
⚡
30,000+ Hrs Saved
Led large-scale automation programme delivering 30,000+ hours in annual productivity gains across 30 regression runs.
👥
Quality Engineering Leadership
15+ as NFT Manager · 10 automation (FTE + contractors) · 10 healthcare cross-functional · 6 across Poland & India (current).
🧠
GenAI QA Lead
Led GenAI QA for Fortune 100 agentic AI on GCP + Genesys — AI judge evaluation, annotation programmes, 500+ agents.
10+Years Experience
30K+Hrs Saved / Year
13+Certifications
15+Engineers Led
03 / Career
Experience
Deloitte ConsultingOttawa, ONApr 2021 – Present
Senior QA Lead — Major Canadian Financial Institution (Top-5 Bank)Dec 2025 – Present
Own test strategy and quality delivery for a legacy modernization program migrating back-end services to cloud; lead a 6-member distributed QA team across Poland and India as technical authority on testing standards, tooling, and defect governance.
Designed and built a multi-agent AI system — three specialized agents — to accelerate test discovery on an undocumented legacy codebase: code-analysis, verification, and automation-architect agents enabling a team new to the application to learn what and how to test.
Write API automated tests using Rest Assured on the CI/CD pipeline; perform contract testing with WireMock to validate API interactions between migrated cloud services and dependent systems.
GenAI QA Lead — Fortune 100 Healthcare OrganizationJun 2025 – Dec 2025
Led 14-member cross-functional team delivering QA for a production-grade agentic AI contact center deployed to 500+ agents on GCP — applying LLM evaluation metrics (relevance, accuracy, completeness, helpfulness) across multi-channel workflows.
Operated a sampling-based human-in-the-loop evaluation pipeline over 6,000 production transcripts in Label Studio; evaluated AI judges for continuous model quality assessment and built tooling to compute inter-annotator agreement.
Validated HIPAA-compliant PII handling on real-time transcripts using Google Cloud Monitoring and DLP; channelled annotation findings to the AI engineering team to drive prompt tuning and model optimization.
Non-Functional Test Manager — Federal Government (IBM Cúram Platform)Jan 2024 – Jun 2025
Created performance, disaster recovery, and failover test plans for Canada's largest social development program — defining scope, SLA targets, and risk thresholds for a mission-critical system processing 50K+ daily transactions.
Orchestrated DR and failover test execution coordinating across Accenture, client teams, and architecture — verifying a 2-hour RPO and 24-hour RTO; directed IBM vendor performance engineering team on baseline, load, and stress test design.
Served as primary non-functional testing stakeholder interface — presented test findings, risk assessments, and go/no-go recommendations to program leadership and government executives.
Automation Manager — Federal Government (IBM Cúram Platform)Jan 2023 – Dec 2023
Led a 10-member team (5 FTEs + 5 contractors) building a JavaScript automation framework with CI/CD integration, BrowserStack cross-browser execution, and Azure DevOps test case updates — delivering 1,350+ automated test cases at 54% coverage.
Accelerated release velocity from quarterly to bi-weekly, delivering 30,000+ annual hours in productivity gains across 30+ regression cycles; championed career development through structured coaching and mentoring of QA engineers.
Quality Architect — Crown Corporation ClientAug 2021 – Dec 2022
Architected the microservices testing strategy for a SAP-to-Lightbend reactive migration handling 2M+ daily transactions; designed and built a Python event-simulation framework validating data integrity across Kafka ingress and egress topics with gRPC and PostgreSQL.
Contributed to the program's NASTA 2022 win (Best Use of Technology in Project); mentored the SDET team on event-driven testing patterns and Python framework design.
Software Development Engineer in Test — Crown Corporation ClientApr 2021 – Jul 2021
Designed parallel Spock API testing framework executing 100,000+ data point validations in under 5 minutes — uncovering 30+ business rule defects and enabling a zero-defect API production release; recognized by client leadership.
IBM SecurityFredericton, NBAug 2016 – Mar 2021
Quality Assurance Lead / Test DeveloperAug 2016 – Mar 2021
Led QA for the QRadar SIEM platform — 80% product coverage across authentication, authorization, log management, event management, and flows modules.
Extended coverage using keyword-driven automation, Spock API testing (BDD & data-driven), and Geb UI automation; tested across multiple hardware configurations mimicking enterprise client environments.
Supported production incident management for QRadar enterprise customers — reproducing, triaging, and validating hotfix releases for critical security defects; performed hardware system integration and on-premises DR testing.
Earlier Experience
Quick Service Software— QA Analyst (Co-op & Contract)May 2015 – Mar 2016
NB Department of Justice— Co-Op Junior DeveloperMay 2014 – Dec 2014
As a QA practitioner, I have always been deeply interested in understanding systems as a
whole — not just the test surface, but the architecture behind it. That habit of reading
systems end-to-end has naturally extended into an architectural perspective: how services
are composed, where failure domains live, and how design decisions upstream shape what
quality looks like downstream. The projects below reflect both sides of that lens —
how I test systems, and how I think about building them.
A proof-of-concept multi-agent quality engineering pipeline that takes a Jira story
to a verified release autonomously. Five specialised agents hand off to each other:
a Reader ingests the story and extracts test intent from the
acceptance criteria, a Generator writes the Playwright spec, a
Runner executes it against the app under test, a
Healer repairs broken locators when a run fails and re-executes
until the test runs clean, and a Reporter files the defect with
severity, repro steps, and run artifacts — linked back to the originating story.
The POC demonstrates where agentic automation genuinely removes QE toil: intent
extraction, self-healing selectors, and evidence-backed defect creation. The demo
below walks the full pipeline end-to-end. Note: the implementation is a private
POC — the source code is not exposed in this portfolio repo; only the recorded
demo is published here.
A practitioner deck on LiveKit — the open-source infrastructure behind real-time
voice AI (OpenAI's ChatGPT voice mode runs on it) — and how QA validates the agents
built on it. Covers the STT→LLM→TTS pipeline, a layer-by-layer validation model,
wiring agent traces into Arize Phoenix for LLM-as-judge evaluation, synthetic-caller
simulation as regression suites, and the release gates that block a ship: latency,
accuracy, safety, and conversation quality. Watch it in the browser with spoken
narration and live captions, or download the PowerPoint. The narrated deck download
plays the presentation with voiceover once extracted and opened.
A documented learning journey from QA Lead to AI/ML Engineer — built in public.
Covers classical ML, deep learning, NLP, and MLOps through a mix of self-written
code and vibe-coded experiments. Each session is logged, every concept noted in
plain language, and every mistake kept. Structured across 8 modules with weekly
journal entries. Currently active: Module 01 — Python for ML (8 exercises + capstone).
Comparative study of four Model Context Protocol (MCP) server architectures demonstrating
how server design — not model choice — determines output quality. Each implementation
exposes the same HR domain to the same model: Flat Tools (unstructured strings, high
hallucination risk), Resource Injection (typed JSON + pre-loaded schema resources, low
hallucination), Prompt Templates (server-side chain-of-thought templates, guaranteed
output structure), and Stateful Memory (session store enabling multi-step reasoning with
context carry-over). Includes benchmark client that scores each pattern on completeness,
format consistency, and token cost. Separate git branch per pattern; main branch contains
comparison matrix and decision guide.
Fully local, containerised Retrieval-Augmented Generation system — no API keys required.
Upload .txt or .pdf documents, ask natural-language questions, and compare RAG-grounded
answers against the same LLM answering from memory alone. FastAPI backend with
paragraph-aware chunking, Qdrant vector store (cosine similarity), and Ollama serving
both the embedding model (Nomic Embed Text) and generation model (Llama 3.2). Streamlit
UI shows retrieved chunks with similarity scores and optional raw prompt view.
Modular Python toolkit for evaluating large language models in production.
Covers faithfulness, answer relevance, context precision, and hallucination detection
with RAGAS and DeepEval backends. Designed for CI/CD integration and enterprise
RAG pipeline validation.
Production-grade backend API test framework (PyAPIElite) supporting REST, GraphQL,
SOAP, gRPC, and Contract testing. Features AI agent output validation via Arize
Phoenix Evals — LLM-as-judge evaluation for hallucination, relevance, QA correctness,
and toxicity across 9 test cases. Allure reporting, Docker, Azure Pipelines CI/CD.
A living reference of how I approach testing real-world systems — banking platforms,
AI/ML pipelines, microservices, and healthcare applications. Each case study covers
system analysis, risk identification, test strategy design, and observability.
Updated as new AI applications are built and shipped.
Test StrategyRisk AnalysisSystems ThinkingFinTechAI/MLHealthcareMicroservices
Production-grade k6 load testing framework with a full observability stack.
k6 pushes metrics to Prometheus via remote-write in real time; Grafana displays
VU ramp, p50/p95/p99 latency, error rate, and API container CPU/memory from
cAdvisor — all in a pre-provisioned dashboard. API instrumented with prom-client
for per-route duration histograms and Node.js runtime metrics.
Reference architecture diagrams for Contact Centre as a Service across GCP and Azure —
multiple configurations covering Dialogflow CX + Genesys, Vertex AI Agent Builder,
Azure Communication Services + OpenAI, Teams Direct Routing, hybrid multi-cloud,
and high-availability patterns. Built from the architectural lens that QA thinking develops.
A multi-environment performance investigation handbook spanning Linux/eBPF, Kubernetes,
Mobile (Android + iOS), and Database layers — combining operational runbooks, bpftrace
scripts, and tool decision trees with USE, RED, and Four Golden Signals methodology
baked into every investigation phase. Built as a practitioner reference for teams who
need to go from symptom to root cause without guessing at tooling.
I'm always open to discussing AI/ML quality engineering challenges, consulting opportunities,
or interesting open-source collaborations. Whether you have a specific project in mind or
just want to connect — reach out.