📖 Documentation

Gherkin scenarios

The acceptance scenarios of features/, one section per product. Run runs a whole feature or one scenario against the stack mise run up started; the run also lands in Test results.

The Driven by filter keeps the features of one tool. Run feature runs every scenario of a feature, Run one scenario. The Gherkin runner starts pytest-bdd against the running stack, the verdict shows above with the pytest output, and the run is filed in Test results like a CI job. The stack down, the scenarios skip and say how to start it.

live products write for real when their scenarios run (Redis visits, orders, accounts, emails); static ones only read.

Each feature says what drives its scenarios, a badge beside its name from its @tool- tag: pytest (pytest-bdd steps calling the stack over HTTP or the bucket), playwright (steps clicking the storefront page in a real Chromium, written from a recorded journey), schemathesis (requests generated from a contract), k6 (a load played by k6, its summary read by the steps). Every one still runs through pytest-bdd, so the console runs and files them alike.

Customer welcome live

Greets a returning customer by first name, counts the visits and shows the bank balance.

🛍️ Try it by hand in Live experimentation: the same product the scenarios below prove, opened alone on the storefront.

1 feature(s), 4 scenario(s)

Feature: Greet a returning customer pytest customer-welcome/greeting.feature

A customer typing their username is greeted by first name, told which visit this is, and shown the balance the partner bank holds for them. The greeting must never invent a balance nor greet someone the company does not serve.

A known customer is greeted with their balance

Goal the happy path through the three neighbours at once: the database gives the first name, the partner bank the balance, formatted in euros from integer cents.

When "ada" asks for a greeting
Then the greeting starts with "Hello Ada!"
And the greeting shows a balance in EUR

Every visit is counted

Goal the visit counter in Redis really grows by one per greeting, so a returning customer is recognised.

When "grace" asks for a greeting
And "grace" asks for a greeting again
Then the second visit number is one more than the first

A customer without bank account is greeted without a balance

Goal a 404 of the partner bank means no account, not an error: the greeting goes on and invents no amount.

When "zoe" asks for a greeting
Then the greeting starts with "Hello Zoe!"
And the greeting shows no balance

A closed customer is refused

Goal a customer the company no longer serves gets a 404, before any call to the bank or any visit counted.

When "dennis" asks for a greeting
Then the storefront answers 404

Checkout live

Places an order and follows it while stock is reserved, the invoice issued and the payment captured.

🛍️ Try it by hand in Live experimentation: the same product the scenarios below prove, opened alone on the storefront.

1 feature(s), 3 scenario(s)

Feature: Place an order and follow it until paid pytest checkout/place-order.feature

An order is accepted at once, then a worker reserves the stock, issues the invoice and captures the payment, one event per step. An order the stock cannot serve is cancelled before any payment. Each run of the first scenario takes one item of the stock of the running stack.

An order in stock is paid step by step

Goal the asynchronous workflow runs to the end: accepted at once with a 202, then stock, invoice and payment each done by the worker, in order.

When "ada" orders 1 "BK-0004"
Then the order is accepted at once
And the workflow ends "completed" with the order "paid"
And the step "capture_payment" is "done"

An order out of stock is cancelled before any payment

Goal a failing step stops the workflow: the order is cancelled and the customer is never charged.

When "grace" orders 1 "EL-0001"
Then the order is accepted at once
And the workflow ends "failed" with the order "cancelled"
And the step "capture_payment" is "skipped"

A closed customer cannot order

Goal the API refuses a closed customer itself, before any event reaches the queue.

When "dennis" orders 1 "BK-0004"
Then the storefront answers 404

Account access live

Creates an account, signs in, resets a password, with every email landing in the QA mailbox.

🛍️ Try it by hand in Live experimentation: the same product the scenarios below prove, opened alone on the storefront.

2 feature(s), 10 scenario(s)

Feature: Account emails reach the QA mailbox pytest account-access/account-emails.feature

Every email of the tenant lands in the QA mailbox, each test account being a plus tag of one address. A scenario reads the email of its own address, opens the link it carries and goes on, as a customer would.

Background

Given a new address of the QA mailbox

A new account receives the email to choose a password

Goal creating an account really sends an email, readable by a test like a customer reads an inbox.

When the customer creates an account with it
Then the mailbox receives "choose your password" for that address
And the email holds a password link

A customer sets the password from the email, then signs in

Goal the whole onboarding, from the email link to the first sign-in.

When the customer creates an account with it
And the customer sets "Foundation-2026" from the link of "choose your password"
Then the customer signs in with "Foundation-2026"

The password link works only once

Goal a leaked or replayed link cannot change the password again.

When the customer creates an account with it
And the customer sets "Foundation-2026" from the link of "choose your password"
Then setting "Another-2026" from the same link is refused

A forgotten password gets a reset link

Goal a customer who forgot the password can get back in through the mailbox.

When the customer creates an account with it
And the customer sets "Foundation-2026" from the link of "choose your password"
And the customer asks to reset the password
Then the mailbox receives "Reset your" for that address

Ten wrong passwords block the account and send an unblock email

Goal brute force is stopped at ten tries, and the real owner is told by email.

When the customer creates an account with it
And the customer sets "Foundation-2026" from the link of "choose your password"
And the customer signs in 10 times with "Wrong-password-1"
Then the mailbox receives "blocked" for that address
And the customer cannot sign in with "Foundation-2026"

Feature: Sign in to a customer account pytest account-access/sign-in.feature

The simulated Auth0 tenant signs a customer in with their email and password. It never says whether an account exists, tells a blocked customer to check their email, asks a new customer to set a password first, and asks a customer with a second factor for the code of their authenticator app.

A customer signs in with their password

Goal the plain sign-in works for a customer with an email and a password.

When "qa-mailbox+ada@mycompany.test" signs in with "Foundation-2026"
Then the sign-in is accepted

An unknown email gets the same answer as a wrong password

Goal the tenant never tells an attacker which emails have an account.

When "qa-mailbox+nobody@mycompany.test" signs in with "Foundation-2026"
Then the tenant answers 401 "Wrong email or password"

A blocked customer is told to check their email

Goal a blocked account says why and where to go next, with its own status code.

When "qa-mailbox+grace@mycompany.test" signs in with "Ultrabook-2026"
Then the tenant answers 423 "Account blocked, check your email"

A customer without a password yet is sent back to their email

Goal an account created without a password cannot be entered until its link is used.

When "qa-mailbox+zoe@mycompany.test" signs in with "Anything-2026"
Then the tenant answers 403 "No password yet, open the link of your email"

A customer with a second factor signs in with the code of their app

Goal the password alone is not enough with a second factor: the tenant asks for the code, then accepts it.

When "qa-mailbox+lin@mycompany.test" signs in with "Authenticator-2026"
Then the tenant answers 403 "mfa_required"
When the customer sends the code of their authenticator app
Then the sign-in is accepted

Bucket checks static

The same generic checks on every team bucket, each bucket bringing its own contract (names, format, freshness, manifest).

1 feature(s), 2 scenario(s)

Feature: Every team bucket keeps its file contract pytest bucket-checks/file-contracts.feature

Teams exchange files through buckets: finance exports its invoices every day, the catalog publishes its feeds. The same generic checks run on every bucket, each bucket bringing its own expectations as a row of examples: where the files are, how they are named, which format they hold, how fresh they must be.

The files of a bucket follow its contract

Goal the files another team drops are where, named, shaped and as fresh as agreed, before our jobs read them.

Given the files of bucket "<bucket>" under "<prefix>"
Then there is at least one file
And every file name matches "<pattern>"
And no file is empty
And every file reads as <format>
And the newest file is less than <hours> hours old

Examples, one run per row

bucket	prefix	pattern	format	hours
finance-exports	daily/	^daily/[0-9]{4}-[0-9]{2}-[0-9]{2}[.]csv$	csv	720
catalog-feeds	feeds/	^feeds/[a-z]+[.]json$	json	720

The manifest of a bucket lists exactly its files

Goal the manifest and the files agree both ways: nothing listed is missing, nothing present is unlisted.

Given the files of bucket "<bucket>" under "<prefix>"
When the manifest "manifest.json" of the bucket is read
Then every file it lists exists
And every file under the prefix is listed

Examples, one run per row

bucket	prefix
finance-exports	daily/
catalog-feeds	feeds/

Quality Console static

The back office of the quality platform, its tools and the company data, read-only.

1 feature(s), 3 scenario(s)

Feature: The Quality Console shows the platform, read-only pytest quality-console/back-office.feature

The back office lists what the company runs and lets anyone browse its data without the right to change it. A name nobody listed never reaches a query.

The catalog lists every system of the company

Goal the catalog the tests' metadata relies on is the one the console shows.

When I open the console page "/catalog"
Then the page shows "Storefront"
And the page shows "Quality platform"

The company database can be browsed

Goal anyone can read the company data without a database client.

When I open the console page "/databases/mycompany"
Then the page shows "30 tables"

A table nobody listed is refused

Goal a name from the URL never reaches a SQL statement unless MySQL listed it.

When I open the console page "/databases/mycompany/tables/salaries"
Then the console answers 404