Skip to content

Commit b252f6c

Browse files
docs: update README examples for v1.2.0 createService test case (#69)
Signed-off-by: Maximilian Techritz <maximilian.techritz@sap.com>
1 parent bf0feac commit b252f6c

1 file changed

Lines changed: 129 additions & 37 deletions

File tree

README.md

Lines changed: 129 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -9,23 +9,49 @@ Platform-service-test-runner allows to define and run in-cluster tests on the op
99
## Architecture
1010

1111
### E2ETestSpecification
12-
The `E2ETestSpecification` custom resource defines a set of test cases and its configuration. It creates an `E2ETestRun` resource to execute the tests defined in the specification.
12+
The `E2ETestSpecification` custom resource defines a set of test cases and its configuration. It creates an `E2ETestRun` resource to execute the tests defined in the specification.
1313
```yaml
1414
apiVersion: test-runner.openmcp.cloud/v1alpha1
1515
kind: E2ETestSpecification
1616
metadata:
17-
name: test-spec
17+
name: test-full-suite
1818
spec:
19-
# optional cron schedule for periodic test execution.
19+
# optional cron schedule for periodic test execution.
2020
# Defaults to @daily if not specified.
21-
schedule: "0 0 * * *"
21+
schedule: "0 0 * * *"
2222
testCases:
2323
- name: createProject
2424
config: {}
2525
- name: createWorkspace
2626
config: {}
2727
- name: createControlPlane
2828
config: {}
29+
- name: createService
30+
config:
31+
assertions:
32+
- path: .status.phase
33+
value: Ready
34+
manifest: |
35+
apiVersion: flux.services.open-control-plane.io/v1alpha1
36+
kind: Flux
37+
spec:
38+
version: "2.8.3"
39+
- name: createService
40+
config:
41+
assertions:
42+
- path: .status.phase
43+
value: Ready
44+
manifest: |
45+
apiVersion: crossplane.services.open-control-plane.io/v1alpha1
46+
kind: Crossplane
47+
spec:
48+
version: "1.20.5"
49+
providers:
50+
- name: provider-kubernetes
51+
version: v1.3.0
52+
functions:
53+
- name: function-patch-and-transform
54+
version: v0.8.0
2955
```
3056
3157
#### Test case configuration
@@ -37,87 +63,143 @@ Currently, the following configuration keys exist:
3763
| --- | --- | --- |
3864
| `chargingTarget` | `createProject` | Specifies the charging target for the project. Value should be a UUID. |
3965
| `chargingTargetType` | `createProject` | Specifies the type of the charging target for the project. |
66+
| `manifest` | `createService` | Required. Inline YAML manifest for the ServiceProvider API resource to apply (e.g. a `Flux` or `Crossplane` resource). The `name` and `namespace` are set automatically from the `createControlPlane` exports. |
67+
| `assertions` | `createService` | Optional list of JSONPath assertions to wait for on the created resource. Each entry has a `path` (JSONPath expression, e.g. `.status.phase`) and an expected `value`. The test case polls until all assertions pass. |
4068
| `identity` | All test cases | Automatically added by the controller, contains the username retrieved via `authenticationv1.SelfSubjectReview{}`. |
4169
| `pollInterval` | All test cases | Polling interval for waiting for conditions. Duration string (e.g., "30s"). Defaults to 10 seconds if not specified. |
4270
| `pollTimeout` | All test cases | Polling timeout for waiting for conditions. Duration string (e.g., "5m"). Defaults to 5 minutes if not specified. |
4371

4472

4573

4674
### E2ETestRun
47-
The `E2ETestRun` custom resource represents a test execution.
75+
The `E2ETestRun` custom resource represents a test execution.
4876
It is created by the `e2etestspecification_controller.go` controller when an `E2ETestSpecification` is created and a new test run needs to be started (based on the schedule).
4977
It contains the status of the test execution, including which test cases have been executed, their results, and any exports they produced for subsequent test cases.
50-
This resource is not deleted after the test execution, but remains in the cluster for inspection and debugging purposes.
78+
This resource is not deleted after the test execution, but remains in the cluster for inspection and debugging purposes.
5179
Example:
5280
```yaml
5381
apiVersion: test-runner.openmcp.cloud/v1alpha1
5482
kind: E2ETestRun
5583
metadata:
5684
labels:
57-
test-runner.openmcp.cloud/specification: test-create-p-w-m
58-
name: run-1772628360009
85+
test-runner.openmcp.cloud/specification: test-full-suite # referencing E2ETestSpecification
86+
name: run-1786536360005
5987
spec:
6088
runner:
61-
version: 0.0.1-dev
89+
version: v1.2.0
6290
testcases:
63-
- config:
64-
chargingTarget: 12345678-1234-1234-1234-123456789012
65-
chargingTargetType: btp
66-
name: createProject
91+
- name: createProject
6792
- name: createWorkspace
93+
- name: createControlPlane
6894
- config:
69-
pollInterval: 20s
70-
pollTimeout: 15m
71-
name: createControlPlane
72-
95+
assertions:
96+
- path: .status.phase
97+
value: Ready
98+
manifest: |
99+
apiVersion: flux.services.open-control-plane.io/v1alpha1
100+
kind: Flux
101+
spec:
102+
version: "2.8.3"
103+
name: createService
104+
- config:
105+
assertions:
106+
- path: .status.phase
107+
value: Ready
108+
manifest: |
109+
apiVersion: crossplane.services.open-control-plane.io/v1alpha1
110+
kind: Crossplane
111+
spec:
112+
version: "1.20.5"
113+
providers:
114+
- name: provider-kubernetes
115+
version: v1.3.0
116+
functions:
117+
- name: function-patch-and-transform
118+
version: v0.8.0
119+
name: createService
120+
73121
status:
74122
testCases:
75123
- conditions:
76-
- lastTransitionTime: "2026-03-04T13:01:04Z"
124+
- lastTransitionTime: "2026-08-12T12:13:50Z"
77125
message: Test case cleanup completed successfully
78126
reason: CleanupSuccess
79127
status: "True"
80128
type: CleanupCompleted
81-
- lastTransitionTime: "2026-03-04T12:59:54Z"
129+
- lastTransitionTime: "2026-08-12T12:08:50Z"
82130
message: Test case run completed successfully
83131
reason: TestPassed
84132
status: "True"
85133
type: RunCompleted
86134
exports:
87-
project.name: run-1772628360009-p
88-
project.status.namespace: project-run-1772628360009-p
135+
project.name: run-1786536360005-p
136+
project.status.namespace: project-run-1786536360005-p
89137
name: createProject
90138
- conditions:
91-
- lastTransitionTime: "2026-03-04T13:00:54Z"
139+
- lastTransitionTime: "2026-08-12T12:13:30Z"
92140
message: Test case cleanup completed successfully
93141
reason: CleanupSuccess
94142
status: "True"
95143
type: CleanupCompleted
96-
- lastTransitionTime: "2026-03-04T13:00:04Z"
144+
- lastTransitionTime: "2026-08-12T12:09:00Z"
97145
message: Test case run completed successfully
98146
reason: TestPassed
99147
status: "True"
100148
type: RunCompleted
101149
exports:
102-
workspace.name: run-1772628360009-ws
103-
workspace.namespace: project-run-1772628360009-p
104-
workspace.status.namespace: project-run-1772628360009-p--ws-run-1772628360009-ws
150+
workspace.name: run-1786536360005-ws
151+
workspace.namespace: project-run-1786536360005-p
152+
workspace.status.namespace: project-run-1786536360005-p--ws-run-1786536360005-ws
105153
name: createWorkspace
106154
- conditions:
107-
- lastTransitionTime: "2026-03-04T13:00:44Z"
155+
- lastTransitionTime: "2026-08-12T12:13:10Z"
108156
message: Test case cleanup completed successfully
109157
reason: CleanupSuccess
110158
status: "True"
111159
type: CleanupCompleted
112-
- lastTransitionTime: "2026-03-04T13:00:14Z"
160+
- lastTransitionTime: "2026-08-12T12:09:10Z"
113161
message: Test case run completed successfully
114162
reason: TestPassed
115163
status: "True"
116164
type: RunCompleted
117165
exports:
118-
mcp.name: run-1772628360009-cp
119-
mcp.namespace: project-run-1772628360009-p--ws-run-1772628360009-ws
166+
controlplane.name: run-1786536360005-cp
167+
controlplane.namespace: project-run-1786536360005-p--ws-run-1786536360005-ws
120168
name: createControlPlane
169+
- conditions:
170+
- lastTransitionTime: "2026-08-12T12:12:50Z"
171+
message: Test case cleanup completed successfully
172+
reason: CleanupSuccess
173+
status: "True"
174+
type: CleanupCompleted
175+
- lastTransitionTime: "2026-08-12T12:10:30Z"
176+
message: Test case run completed successfully
177+
reason: TestPassed
178+
status: "True"
179+
type: RunCompleted
180+
exports:
181+
service.apiVersion: flux.services.open-control-plane.io/v1alpha1
182+
service.kind: Flux
183+
service.name: run-1786536360005-cp
184+
service.namespace: project-run-1786536360005-p--ws-run-1786536360005-ws
185+
name: createService/Flux
186+
- conditions:
187+
- lastTransitionTime: "2026-08-12T12:12:30Z"
188+
message: Test case cleanup completed successfully
189+
reason: CleanupSuccess
190+
status: "True"
191+
type: CleanupCompleted
192+
- lastTransitionTime: "2026-08-12T12:11:20Z"
193+
message: Test case run completed successfully
194+
reason: TestPassed
195+
status: "True"
196+
type: RunCompleted
197+
exports:
198+
service.apiVersion: crossplane.services.open-control-plane.io/v1alpha1
199+
service.kind: Crossplane
200+
service.name: run-1786536360005-cp
201+
service.namespace: project-run-1786536360005-p--ws-run-1786536360005-ws
202+
name: createService/Crossplane
121203
```
122204

123205
## Implementing New Test Cases
@@ -128,12 +210,14 @@ All test cases must implement the `TestCase` interface defined in `internal/runn
128210

129211
```go
130212
type TestCase interface {
213+
StatusName(config Config) string
131214
Run(ctx context.Context, testRun *v1alpha1.E2ETestRun, config Config) (Exports, DebugInfo, error)
132215
Cleanup(ctx context.Context, testRun *v1alpha1.E2ETestRun, config Config) error
133216
}
134217
```
135218

136-
- **Run**: Executes the test case. Returns exports (key-value pairs for subsequent tests), error details, and error.
219+
- **StatusName**: Returns the name used to identify this test case's status entry. For test cases that can appear multiple times (e.g. `createService`), incorporate a discriminator from config (e.g. the manifest kind) to produce a unique name like `createService/Flux`.
220+
- **Run**: Executes the test case. Returns exports (key-value pairs for subsequent tests), debug info, and error.
137221
- **Cleanup**: Performs cleanup after the test (e.g., delete created resources).
138222

139223
### Example Implementation
@@ -144,12 +228,19 @@ type MyTestCase struct {
144228
Client client.Client
145229
}
146230
147-
func (t *MyTestCase) Run(ctx context.Context, testRun *v1alpha1.E2ETestRun, config Config (Exports, DebugInfo, error) {
148-
// Use testRun to access status and exports from previous test cases.
149-
status := util.GetStatus(testRun, "previousTestCaseName")
150-
exports := status.Exports
231+
func (t *MyTestCase) StatusName(config Config) string {
232+
return "myTestCase"
233+
}
234+
235+
func (t *MyTestCase) Run(ctx context.Context, testRun *v1alpha1.E2ETestRun, config Config) (Exports, DebugInfo, error) {
236+
// Use testRun to access status and exports from previous test cases.
237+
status, found := runner.GetStatus("previousTestCaseName", testRun.Status.TestCases)
238+
if !found {
239+
return nil, nil, fmt.Errorf("dependent test case has no status")
240+
}
241+
_ = status.Exports
151242
// Use config to access test case specific configuration.
152-
// By default, the controller is adding a key `identity` to the config, which is its own username retrieved via authenticationv1.SelfSubjectReview{}
243+
// By default, the controller adds a key `identity` to the config with its own username.
153244
// Implement the test logic here, e.g. create resources, wait for conditions, etc.
154245
// Create own Exports to be used by subsequent test cases.
155246
return Exports{}, DebugInfo{}, nil
@@ -164,13 +255,14 @@ func (t *MyTestCase) Cleanup(ctx context.Context, testRun *v1alpha1.E2ETestRun,
164255
## Registering Test Cases
165256

166257
Test cases must be registered with the `TestRegistry` to be discoverable by the controller. Use `NewTestRegistry()` to create a registry instance.
167-
Currently, test cases are registered in `cmd/plaform-service-test-runner/app/run.go`:
258+
Currently, test cases are registered in `cmd/platform-service-test-runner/app/run.go`:
168259
```go
169260
testRegistry := runner.NewTestRegistry()
170261
// register test cases here
171262
testRegistry.RegisterTestCase("createProject", &runner.CreateProjectTest{OnboardingClient: onboardingCluster.Client()})
172263
testRegistry.RegisterTestCase("createWorkspace", &runner.CreateWorkspaceTest{OnboardingClient: onboardingCluster.Client()})
173264
testRegistry.RegisterTestCase("createControlPlane", &runner.CreateControlPlaneTest{OnboardingClient: onboardingCluster.Client()})
265+
testRegistry.RegisterTestCase("createService", &runner.CreateServiceTest{OnboardingClient: onboardingCluster.Client()})
174266
```
175267

176268
## Requirements and Setup

0 commit comments

Comments
 (0)