DataLLM LabStep 5 Preview Coding Evaluation: A Small Task With Clear Gates
A Step 5 Preview coding evaluation should start with a task small enough to inspect and a tool setup that cannot quietly expand the work. A long context window may help an agent read more files. It does not prove that its patch is correct, that a preview offer applies to your account or that broad terminal access is appropriate.
AI-generated editorial illustration of the proposed evaluation. No model patch or test result is shown.
This guide checks public documentation as of October 9, 2026 and proposes an evaluation contract. No model requests, coding-tool setup, account verification or model-quality tests were performed.
Verify the Step 5 Preview coding evaluation route
StepFun's model documentation identifies the direct model as step-5-preview. It lists text, image and video input, text output, a one-million-token context window and a 64k maximum output. Tool calling is supported, but the integrating application supplies the tools and access to local or external resources. Official model documentation.
OpenRouter lists stepfun/step-5-preview, with an October 8 listing date and prices of $1 per million input tokens, $2.70 per million output tokens and $0.05 per million cache-read tokens. Those are this gateway's published rates, not a free coding-tool allowance or proof of successful access. OpenRouter listing.
Keep the provider and identifier together. A direct StepFun configuration, an OpenRouter model selection and an OpenCode Zen promotion are different routes. Do not reuse credentials or assume that pricing, quotas and data handling transfer between them.
Check introductory access without turning it into a promise
OpenCode's Zen documentation lists step-5-preview-free, says it is free for a limited time and describes a provider zero-retention, no-training policy for that route. OpenCode configurations use the opencode/<model-id> format, making its documented selector opencode/step-5-preview-free. The page does not establish an exact expiry or personal quota. Zen model and pricing documentation.
Cline's free-model documentation describes rotating, quota-limited promotions in its IDE extension and CLI, not its API. It does not identify Step 5 Preview or confirm a model-specific end date. Check the current account's model selector rather than treating a reported free week as verified entitlement. Cline also says free-model usage may help improve models; that differs from the policy stated for the OpenCode route. Cline free-model rules.
For Cline with an already authorized OpenRouter key, the documented configuration is Settings, API Provider OpenRouter, the key field and the Model selector. Look for the exact listing identifier. Before evaluating, check that your installed version lists it and that your authorized account can use the route. Cline OpenRouter configuration.
Record the offer and its timestamp before an evaluation. Stop when the free route disappears; do not silently switch to a paid model or enable a paid fallback.
Restrict permissions before opening a repository
Use a disposable project containing only the fixture module and tests. Remove production credentials, private datasets and unrelated directories. Prepare dependencies beforehand so the agent has no reason to install packages during the run.
Cline separates auto-approval for reads, edits, commands, browser actions and MCP tools. Its command safety classification is not a fixed allowlist. For this proposed evaluation, leave edits and commands subject to review, keep outside-workspace access closed and leave browser, MCP and YOLO automation off. Cline approval controls.
OpenCode supports allow, ask and deny, and its documented defaults allow most actions. Set explicit review requirements for editing and shell commands; deny outside-directory access and unnecessary network tools. Keep automatic approval disabled. Inspect the effective project and agent configuration rather than assuming a prose instruction enforces a restriction. OpenCode permissions.
These controls need an operating-system or container boundary when stronger isolation is required. A permission setting is not proof that an environment is sandboxed.
Freeze a small-task contract with expected invariants
The following is an original proposed task, not a model-generated patch or benchmark result:
Implement
summarize_orders(orders)inorders.pyusing Python's standard library. Accept a list of dictionaries with a nonblank stringid, acurrencyof USD, EUR or CNY, and a nonnegative integeramount_minor. Reject booleans as amounts. Reject non-list input, malformed rows, missing required fields and duplicate IDs withValueError. Compare IDs as exact strings, but reject whitespace-only IDs. Return one summary per currency, sorted by currency, withcurrency,countandtotal_minor. Do not mutate the input. Edit onlyorders.py; the tests belong to the evaluator.
A concrete fabricated input and its contract-defined expected output are:
{
"input": [
{"id": "a", "currency": "USD", "amount_minor": 1250},
{"id": "b", "currency": "EUR", "amount_minor": 200},
{"id": "c", "currency": "USD", "amount_minor": 750}
],
"expected": [
{"currency": "EUR", "count": 1, "total_minor": 200},
{"currency": "USD", "count": 2, "total_minor": 2000}
]
}
Before prompting the model, the reviewer should create and freeze correctness tests from these requirements:
| Fixture or check | Required result |
|---|---|
| Empty list | Empty list, without error |
| Example above | Exact sorted summaries and integer totals |
| Zero-value order | Included in count and total |
| Repeated ID, even across currencies | ValueError, not silent deduplication |
| Negative, float or boolean amount | ValueError for each case |
| Missing field, unsupported currency, blank/nonstring ID | ValueError for each case |
| Non-list input or nondictionary row | ValueError |
| Input snapshot before and after either outcome | No mutation, including on error |
| Repeated invocation on identical input | Identical output |
Integer minor units make the arithmetic exact without claiming a real payment integration. Add a large-integer case so an implementation cannot quietly convert totals to floating point. Keep a held-out ordering case to detect code that only fits the displayed example.
Run a bounded evaluation and retain the patch
After access, data use and any spending are authorized, allow one implementation attempt in the isolated project. A proposed limit is ten minutes and twenty tool calls, with a separately approved spending ceiling on paid routes. These are evaluator stop rules, not documented product quotas or automatically enforced settings.
Review every requested command and diff. Do not approve package installation, downloads, network calls, commits or deployment. Run the frozen tests independently and retain raw output; an agent saying its tests passed is not sufficient evidence. Treat changes to the evaluator or unrelated files as a scope failure even if the displayed tests pass.
Record patch correctness, scope compliance and operational errors separately. An HTTP error or exhausted allowance is an access outcome, not a low model score. A failed assertion is a correctness finding for that fixture, not proof that every coding task will fail.
Use a result record that cannot disguise missing evidence
This unfilled template is an original evaluation artifact. Replace not_recorded only with observed information; leave the status NOT_RUN until an authorized run actually occurs.
status: NOT_RUN
checked_at: not_recorded
tool_and_version: not_recorded
provider_and_model_id: not_recorded
offer_or_rate_snapshot: not_recorded
permission_snapshot: not_recorded
prompt_and_fixture_version: not_recorded
attempt_and_tool_call_count: not_recorded
patch_and_raw_test_output: not_recorded
invariants_passed_failed_unrun: not_recorded
scope_violations: not_recorded
usage_and_actual_or_derived_cost: not_recorded
reviewer_decision: not_recorded
Keep secrets out of the record. Distinguish provider-reported usage, a calculated list-price estimate and actual billed spending. A temporary zero-price run cannot establish the economics after an offer ends. Existing Step 3.7 content concerns another model; its scores and costs are not Step 5 evidence.
Conclusion
A useful Step 5 Preview coding evaluation ends with an inspectable patch, frozen-test output and a record of the route and permissions used. Verify introductory access separately, start with the small contract and expand only after reviewing actual results. Passing this task would still not establish reliability on an entire codebase.
FAQ
Is Step 5 Preview free on every route?
No. OpenRouter publishes paid rates, while OpenCode documents a limited-time free selector. Cline's general promotion rules do not establish a Step 5 entitlement or expiry for your account.
Does one-million-token context make a patch trustworthy?
No. Context capacity is an input limit, not a correctness test. Inspect the diff, run the frozen assertions and retain failures and scope violations.
Are the fixtures or result template actual Step 5 outputs?
No. They are proposed evaluator artifacts. No inference was run, and the result template intentionally records no scores, latency, setup success or spending.