Development: Enable Atlas Orchestrator on all Exercise Types#13178
Open
jaylann wants to merge 11 commits into
Open
Development: Enable Atlas Orchestrator on all Exercise Types#13178jaylann wants to merge 11 commits into
Development: Enable Atlas Orchestrator on all Exercise Types#13178jaylann wants to merge 11 commits into
Conversation
github-actions
Bot
added
tests
server
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1d7bcbb704
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
Contributor
WalkthroughAtlas orchestration now targets generic exercises instead of programming-only exercises. The backend listener, orchestration service, extraction pipeline, tool endpoint, REST route, frontend trigger, exercise detail views, translations, and tests were updated to support text, modeling, file upload, and quiz exercises. ChangesBackend orchestration, extraction, and tool updates
Frontend trigger, API, and view wiring
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
src/main/java/de/tum/cit/aet/artemis/atlas/service/OrchestratorToolsService.java (1)
265-269: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winConsider capping returned content length for
getExerciseContent.Unlike the batch/system-prompt path (
CompetencyOrchestrationService.sanitizeForPromptwithPROBLEM_STATEMENT_MAX), this on-demand tool has no length cap on the extracted content. Now that it applies to every exercise type — including quiz, whose assembled learning text can be large — an unusually large exercise could return an oversized payload per call, inflating token cost.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main/java/de/tum/cit/aet/artemis/atlas/service/OrchestratorToolsService.java` around lines 265 - 269, `OrchestratorToolsService.getExerciseContent` currently returns the full `ExtractedContentDTO` content without any length cap, which can produce oversized payloads. Add a truncation/limit step after `contentExtractionService.extractContent(exercise, false)` and before returning, using a shared max-length constant or helper similar to `CompetencyOrchestrationService.sanitizeForPrompt`/`PROBLEM_STATEMENT_MAX`. Make sure the cap applies for all exercise types, including quiz, and preserve the existing `getExerciseContent` behavior for extraction while limiting only the returned text.src/main/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionService.java (1)
364-502: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider extracting quiz rendering into a dedicated helper/class.
renderQuizQuestions/renderMultipleChoice/renderShortAnswer/renderDragAndDrop/renderDragAndDropSolutionadd ~140 lines of quiz-specific rendering logic onto a service whose primary responsibility is orchestrating extraction + flavor stripping. Moving this to a dedicatedQuizContentRenderer(or similar) would keepContentExtractionServicefocused on the extraction dispatch/flavor-strip lifecycle and make the quiz-rendering rules independently testable/extensible as more question types are added.As per path instructions,
src/main/java/**/*.javashould followsingle_responsibility.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionService.java` around lines 364 - 502, The quiz rendering logic is doing too much inside ContentExtractionService and should be moved out to a dedicated helper/class. Extract renderQuizQuestions, renderMultipleChoice, renderShortAnswer, renderDragAndDrop, renderDragAndDropSolution, and appendField into a QuizContentRenderer (or similarly named) component, then have ContentExtractionService delegate to it so the service stays focused on extraction orchestration. Keep the existing behavior intact while making the quiz-specific rules easier to test and extend by locating the new logic via those method names.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@src/main/java/de/tum/cit/aet/artemis/atlas/service/CompetencyOrchestrationService.java`:
- Around line 444-461: Track per-exercise extraction failures in
CompetencyOrchestrationService.orchestrateBatch instead of only logging and
skipping them; the current logic loses those exercise ids when
runWithQueuedFlush/runBatch drains the queue and the batch still returns SUCCESS
or PARTIAL. Record the ids that fail in contentExtractionService.extractContent,
return or expose them alongside the CompetencyOrchestrationResultDTO, and make
the batch orchestration path requeue those failed ids even when the overall
batch succeeds. Keep the change localized around orchestrateBatch and the queued
flush/batch requeue flow so the skipped exercises are retried on the next run.
In
`@src/main/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionService.java`:
- Around line 58-65: The extension guide Javadoc in ContentExtractionService is
stale because extractContent(LearningObject, boolean) now uses a
pattern-matching switch rather than an instanceof chain. Update the
add-new-learning-object instructions to refer to the switch-based dispatch in
extractContent, and keep the guidance for creating a private extractFrom* method
and setting exerciseType or another type discriminator in baseMetadata/related
metadata helpers.
---
Nitpick comments:
In
`@src/main/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionService.java`:
- Around line 364-502: The quiz rendering logic is doing too much inside
ContentExtractionService and should be moved out to a dedicated helper/class.
Extract renderQuizQuestions, renderMultipleChoice, renderShortAnswer,
renderDragAndDrop, renderDragAndDropSolution, and appendField into a
QuizContentRenderer (or similarly named) component, then have
ContentExtractionService delegate to it so the service stays focused on
extraction orchestration. Keep the existing behavior intact while making the
quiz-specific rules easier to test and extend by locating the new logic via
those method names.
In
`@src/main/java/de/tum/cit/aet/artemis/atlas/service/OrchestratorToolsService.java`:
- Around line 265-269: `OrchestratorToolsService.getExerciseContent` currently
returns the full `ExtractedContentDTO` content without any length cap, which can
produce oversized payloads. Add a truncation/limit step after
`contentExtractionService.extractContent(exercise, false)` and before returning,
using a shared max-length constant or helper similar to
`CompetencyOrchestrationService.sanitizeForPrompt`/`PROBLEM_STATEMENT_MAX`. Make
sure the cap applies for all exercise types, including quiz, and preserve the
existing `getExerciseContent` behavior for extraction while limiting only the
returned text.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: b1bd2579-c241-4386-8d3b-911b25408dcb
📒 Files selected for processing (35)
src/main/java/de/tum/cit/aet/artemis/atlas/service/AutonomousCompetencyExerciseEventListener.javasrc/main/java/de/tum/cit/aet/artemis/atlas/service/CompetencyOrchestrationService.javasrc/main/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionService.javasrc/main/java/de/tum/cit/aet/artemis/atlas/service/OrchestratorToolsService.javasrc/main/java/de/tum/cit/aet/artemis/atlas/web/CompetencyOrchestrationResource.javasrc/main/webapp/app/atlas/manage/orchestration-trigger/atlas-orchestration-trigger.component.htmlsrc/main/webapp/app/atlas/manage/orchestration-trigger/atlas-orchestration-trigger.component.spec.tssrc/main/webapp/app/atlas/manage/orchestration-trigger/atlas-orchestration-trigger.component.tssrc/main/webapp/app/atlas/shared/services/competency-orchestration-api.service.spec.tssrc/main/webapp/app/atlas/shared/services/competency-orchestration-api.service.tssrc/main/webapp/app/exercise/exercise-detail-common-actions/non-programming-exercise-detail-common-actions.component.htmlsrc/main/webapp/app/fileupload/manage/exercise-details/file-upload-exercise-detail.component.htmlsrc/main/webapp/app/fileupload/manage/exercise-details/file-upload-exercise-detail.component.spec.tssrc/main/webapp/app/fileupload/manage/exercise-details/file-upload-exercise-detail.component.tssrc/main/webapp/app/modeling/manage/detail/modeling-exercise-detail.component.htmlsrc/main/webapp/app/modeling/manage/detail/modeling-exercise-detail.component.tssrc/main/webapp/app/programming/manage/detail/programming-exercise-detail.component.htmlsrc/main/webapp/app/programming/manage/detail/programming-exercise-detail.component.tssrc/main/webapp/app/quiz/manage/detail/quiz-exercise-detail.component.htmlsrc/main/webapp/app/quiz/manage/detail/quiz-exercise-detail.component.spec.tssrc/main/webapp/app/quiz/manage/detail/quiz-exercise-detail.component.tssrc/main/webapp/app/quiz/manage/manage-buttons/quiz-exercise-manage-buttons.component.htmlsrc/main/webapp/app/text/manage/detail/text-exercise-detail.component.htmlsrc/main/webapp/app/text/manage/detail/text-exercise-detail.component.tssrc/main/webapp/i18n/de/atlasOrchestrator.jsonsrc/main/webapp/i18n/de/programmingExercise.jsonsrc/main/webapp/i18n/en/atlasOrchestrator.jsonsrc/main/webapp/i18n/en/programmingExercise.jsonsrc/test/java/de/tum/cit/aet/artemis/atlas/service/AutonomousCompetencyExerciseEventListenerTest.javasrc/test/java/de/tum/cit/aet/artemis/atlas/service/CompetencyOrchestrationServiceTest.javasrc/test/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionServiceFlavorStripTest.javasrc/test/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionServiceIntegrationTest.javasrc/test/java/de/tum/cit/aet/artemis/atlas/service/ContentExtractionServiceTest.javasrc/test/java/de/tum/cit/aet/artemis/atlas/service/OrchestratorToolsServiceTest.javasrc/test/java/de/tum/cit/aet/artemis/atlas/web/CompetencyOrchestrationResourceIntegrationTest.java
💤 Files with no reviewable changes (3)
- src/main/java/de/tum/cit/aet/artemis/atlas/service/AutonomousCompetencyExerciseEventListener.java
- src/main/webapp/i18n/en/programmingExercise.json
- src/main/webapp/i18n/de/programmingExercise.json
github-project-automation
Bot
moved this from Work In Progress
to Ready For Review
in Artemis Development
End-to-End Test Results
Test Strategy: Two-phase execution
Overall: ✅ E2E tests passed 🔗 Workflow Run · 📊 Test Report Phase 1 · 📊 Test Report Phase 2 |
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
Use jspecify @nullable in ContentExtractionService (matches sibling services, fixes testNullnessAnnotations and the deprecation warning) and drop JUnit assertDoesNotThrow in the integration test (fixes testNoJunitJupiterAssertions).
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
Claudia-Anthropica
approved these changes
Jul 9, 2026
Claudia-Anthropica
left a comment
Claudia-Anthropica
left a comment
Contributor
There was a problem hiding this comment.
@jaylann The implementation looks solid: the generic extraction and orchestration paths keep the course/exam guards intact, handle per-exercise extraction failures without losing the rest of the batch, and the UI wiring is covered across the exercise detail pages. CI is green. The e2e run could not exercise the feature because the wrapper stack rendered only the app shell at login and /management/health reported DOWN, so I proceeded on static review and CI.
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
jaylann
temporarily deployed
to
playwright-e2e-tests
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Atlas competency auto-orchestration agent previously mapped only programming exercises to competencies; every other type was silently dropped at three choke points. This PR generalizes the whole pipeline — the record gate, the orchestration service, and content extraction — to all course exercise types (programming, text, quiz, modeling, file-upload). Instructors now get the "Suggest competencies" trigger on every exercise detail page, and content edits to any type feed the automatic pipeline. Lecture units remain out of scope (they need net-new change-event infrastructure).
Checklist
General
Server
Client
authoritiesto all new routes and checked the course groups for displaying navigation elements (links, buttons).Motivation and Context
Adaptive learning is only useful if competencies cover the whole course, but auto-orchestration was gated to programming exercises by an
instanceof ProgrammingExerciseearly-return, aProgrammingExercise-typed service, and an extractor that threw for anything else. Text/quiz/modeling/file-upload content never reached the competency mapper. This closes that gap so the agent works for the full set of course exercise types.Description
AutonomousCompetencyExerciseEventListener): removed the programming-only early return; all non-exam course exercise versions now record into the change accumulator.CompetencyOrchestrationService): retyped fromProgrammingExercise/ProgrammingExerciseRepositoryto genericExercise/ExerciseRepositorythroughout (resolve / run / precheck / orchestrate). Batch extraction now isolates per-exercise failures — a single bad entry (e.g. a quiz deleted mid-run whose refetch throws) is logged and skipped rather than failing the whole batch and consuming the course's daily-run slot; a fully-empty batch returnsINTERNAL_ERROR.ContentExtractionService): theinstanceofchain is replaced by aswitchover all five types. Text / modeling / file-upload use the (flavor-stripped) problem statement plus their labeled example solution — modeling uses the solution explanation, not the Apollon JSON model, which is noise for competency reasoning. Quizzes carry no problem statement, so content is assembled from the questions, answer options, and correct solutions across multiple-choice / short-answer / drag-and-drop; quiz extraction re-fetches lazy questions (QuizExerciseRepository.findByIdWithQuestionsElseThrow) and skips the flavor-strip pass (terse structured Q&A, not narrative prose). All extracted content is server-side only and is sanitized (fence-neutralized, truncated) by the orchestrator before it reaches the prompt.OrchestratorToolsService.getExerciseContent): returns real extracted content for every type instead of a title-only stub for non-programming; runs with the flavor-strip disabled to avoid an extra uncapped LLM round-trip on this read path.POST atlas/orchestrator/programming-exercises/{id}/run→POST atlas/orchestrator/exercises/{id}/run(runForExercise), still guarded by@EnforceAtLeastInstructorInExercise+@FeatureToggle(AtlasAgent).AtlasOrchestrationTriggerComponentis retyped to a genericexerciseinput, gains abuttonClassinput, and self-owns theatlasModuleActivemodule gate so host pages stay free of Atlas knowledge. It is projected via<ng-content>into the text/modeling/file-upload common-actions bar (non-programming-exercise-detail-common-actions) and the quiz manage-buttons group (quiz-exercise-manage-buttons); the programming detail page is updated to the new inputs. The API service method is renamed torunForExercise, and the i18nbuttonkey moves fromprogrammingExercisetoatlasOrchestrator(en + de).Steps for Testing
Prerequisites:
AtlasAgentfeature toggle onTestserver States
You can manage test servers using Helios. Check environment statuses in the environment list. To deploy to a test server, go to the CI/CD page, find your PR or branch, and trigger the deployment.
Review Progress
Code Review
Manual Tests
Test Coverage
Note: Some tests in the Test job did not pass (
failure). Coverage below may be partial.Client
Server
Last updated: 2026-07-12 14:32:26 UTC
Screenshots
Summary by CodeRabbit
New Features
Bug Fixes