Development preview

Evaluation suites for professional work.

We are developing task suites and resettable environments for evaluating AI agents on ecommerce, search, editorial, and acquisition workflows.

Each task combines a brief, source records, scoped tools, and grading criteria. Agents investigate the evidence, prepare changes, and submit work for evaluation.

Cedar & LoomDemo environment

Investigate a retention incident

Reconcile store and campaign records, identify configuration defects, and prepare a repair plan.

Store ordersPurchase eventsCustomer permissionsCampaign reportsSupport ticketsFlow settings
Findings with source references
Proposed configuration changes
Validation and release handoff
Synthetic merchant snapshot

Practitioner-led task design.

Commercial assignments will be designed by domain practitioners, with explicit success criteria and acceptable alternatives.

The environment supplies the records, tools, and starting state. Automated checks assess calculations, configuration, and policy constraints. Independent expert review assesses evidence and judgment.

Task authorship
Practitioners define the assignment and material failure cases. AI assists with specification and implementation.
Source records
Tasks include linked records, operating policies, and defined action boundaries.
Evaluation
Score observable outcomes and preserved constraints. Record expert assessments separately.

Cedar & Loom

A synthetic homewares merchant with ten related assignments covering tracking, audiences, lifecycle messaging, offers, and release review.

Download public inputs
Demo status The current tasks and records are AI-designed fixtures for testing the tools and grading workflow. Expert-authored commercial tasks, independent reviews, and live model evaluations are not yet available.
Public workspace

Loading the public task catalogue…

This browser preview lets you inspect briefs and records. The downloadable pack contains public inputs and local Python tools for preparing submissions. Grading is run separately; private graders and reference answers are not included.

Domains and current status.

Ecommerce has a working synthetic demo. The remaining collections are planned; commercial task design will be led by practitioners in each domain.

Research and pilot enquiries.

For evaluation requirements, scoped pilots, or practitioner contributions, contact Dom Wells.

Contact