Contra - A professional network for the jobs and skills of the futureSet Up AI Evaluation Workflow in VS Code for Benchmarking
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
VS Code AI Baselining Tooling Setup
Configured and used a VS Code-based AI evaluation workflow for coding and infrastructure benchmark tasks. The setup involved preparing local development tooling, installing Python/Docker/Git dependencies, configuring multiple AI coding models inside VS Code, and using standardized scripts to download tasks, run benchmark tests, capture trajectories, and record model performance.
The workflow supported both software engineering and infrastructure tasks, including SWE-Bench-style bug fixes/features in real codebases and Terminal-Bench-style Docker troubleshooting tasks. Each task required solving the same problem across multiple models, tracking AI-assisted changes through Git commits, running tests, reviewing model behavior, and documenting failure modes and performance differences.
The screenshot shows the project dashboard/tooling area where the required shared VS Code tooling package was provided for task execution.
Post image
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started