Bug Description
BashTrial.collect() currently returns a plain TrialResult derived from exit code and raw output. Bash jobs that emit the reward protocol artifacts (<trial_name>/result.json, reward.txt, reward.json) are not parsed by the Python SDK, so trial.score stays 0.0 even when the job produced a valid reward value.
Steps to Reproduce
- Run a BashJob whose script writes a trial-level reward protocol file under the job artifact root, for example
/data/logs/user-defined/task-1__abc/result.json.
- Put the score under
verifier_result.rewards.reward, as required by the reward protocol.
- Let
BashTrial.collect() collect the result.
Expected Behavior
The SDK should discover trial-level reward result files, parse the reward protocol payload, and expose the primary reward through trial.score. If no result file is available, it should still be able to use the standard stdout score: line as a compatibility fallback.
Actual Behavior
BashTrial.collect() only maps the process exit code to a plain TrialResult. It does not read Bash reward result files, and the base TrialResult.score property always returns 0.0.
Error Logs
No exception is required to reproduce this. A successful Bash job can print a valid score and still return trial.score == 0.0 from the SDK.
Environment Information
- OS: macOS / Linux
- Python Version: 3.10+
- ROCK Version: master / rl-rock 1.10.x development branch
- Installation Method: source installation with uv
- Docker Version: N/A
- Deployment Type: local SDK collection against BashJob sandbox artifacts
ROCK Configuration
- Runtime Environment Type: uv
- Sandbox Image: any BashJob image that writes reward protocol artifacts
- Resource Allocation: N/A
Component Affected
Bug Description
BashTrial.collect()currently returns a plainTrialResultderived from exit code and raw output. Bash jobs that emit the reward protocol artifacts (<trial_name>/result.json,reward.txt,reward.json) are not parsed by the Python SDK, sotrial.scorestays0.0even when the job produced a valid reward value.Steps to Reproduce
/data/logs/user-defined/task-1__abc/result.json.verifier_result.rewards.reward, as required by the reward protocol.BashTrial.collect()collect the result.Expected Behavior
The SDK should discover trial-level reward result files, parse the reward protocol payload, and expose the primary reward through
trial.score. If no result file is available, it should still be able to use the standard stdoutscore:line as a compatibility fallback.Actual Behavior
BashTrial.collect()only maps the process exit code to a plainTrialResult. It does not read Bash reward result files, and the baseTrialResult.scoreproperty always returns0.0.Error Logs
No exception is required to reproduce this. A successful Bash job can print a valid score and still return
trial.score == 0.0from the SDK.Environment Information
ROCK Configuration
Component Affected