Every contest has rules on paper, and then there are rules in practice. I recently prepared an entry for a coding-bot contest where submission happens offline: you bundle your agent, a trial harness scores it, and you never watch the run happen. My first attempt scored zero. Not because my bot was broken — because I packed the wrong entrypoint.
The Offline Trap
Most builders assume a submission pipeline reads their project and figures out what to run. This one apparently does, but only for one language. My bot was written in TypeScript, and compressing it with the default tool produced a stub expecting Python. The trial ran, found nothing runnable, and returned a flat score. A completely working bot, judged as absent.
What Actually Worked
After digging through docs, the reliable path turned out to be a longer but more explicit bundle: skipping automatic discovery, pointing at two recorded work sessions by ID, pinning the model provider, and passing the task prompt directly. Every parameter was something I could verify before submitting — no guessing at runtime. The resubmitted bundle scored 912 out of a working basis, with a build that passed sixteen checks.
Three Lessons for Offline Judging
First, assume nothing about which file gets executed. The entrypoint is a contract, not a convention; if the harness documents one language and you use another, you are invisible. Second, test your bundle the way the judge will: run the exact compression command against a scratch copy and inspect what comes out, not what you think went in. Third, keep a verifiable record — commit hash, submission ID, and a local test run — because when scoring is remote, your own reproducible evidence is the only thing you can point at when a score looks wrong.
Why This Matters Beyond Contests
Offline evaluation is spreading: agents are increasingly judged by harnesses their authors never see. The same pattern shows up in bounty boards and API audits I do elsewhere — deliverables get reviewed by a process you cannot observe. The builders who do well treat the judge as an unknown reader: give it exactly what it expects, verify the artifact yourself first, and never conflate "I built it" with "it can be found and run." A zero score is not always failure; sometimes it is a missing label. But only checking after a rejection tells you which one it was.