Run Hurl tests multiple times to catch flakiness#813
Merged
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
🤖 Augment PR SummarySummary: Runs the Hurl E2E suite multiple times to increase the chance of catching flaky HTTP tests. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
jviotti
force-pushed
the
e2e-hurl-multiple
branch
from
9238251 to
2f9df89
Compare
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
jviotti
force-pushed
the
e2e-hurl-multiple
branch
from
2f9df89 to
9666326
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Benchmark Index (community)
Details
| Benchmark suite | Current: 9666326 | Previous: 1ab17eb | Ratio |
|---|---|---|---|
Add one schema (0 existing) |
20 ms |
18 ms |
1.11 |
Add one schema (100 existing) |
23 ms |
23 ms |
1 |
Add one schema (1000 existing) |
73 ms |
81 ms |
0.90 |
Add one schema (10000 existing) |
634 ms |
685 ms |
0.93 |
Update one schema (1 existing) |
16 ms |
16 ms |
1 |
Update one schema (101 existing) |
23 ms |
23 ms |
1 |
Update one schema (1001 existing) |
74 ms |
81 ms |
0.91 |
Update one schema (10001 existing) |
638 ms |
694 ms |
0.92 |
Cached rebuild (1 existing) |
8 ms |
9 ms |
0.89 |
Cached rebuild (101 existing) |
11 ms |
15 ms |
0.73 |
Cached rebuild (1001 existing) |
30 ms |
34 ms |
0.88 |
Cached rebuild (10001 existing) |
247 ms |
279 ms |
0.89 |
Index 100 schemas |
136 ms |
132 ms |
1.03 |
Index 1000 schemas |
1053 ms |
1041 ms |
1.01 |
Index 10000 schemas |
13450 ms |
13229 ms |
1.02 |
This comment was automatically generated by workflow using github-action-benchmark.
Contributor
There was a problem hiding this comment.
Benchmark Index (enterprise)
Details
| Benchmark suite | Current: 9666326 | Previous: 1ab17eb | Ratio |
|---|---|---|---|
Add one schema (0 existing) |
16 ms |
19 ms |
0.84 |
Add one schema (100 existing) |
20 ms |
24 ms |
0.83 |
Add one schema (1000 existing) |
58 ms |
74 ms |
0.78 |
Add one schema (10000 existing) |
482 ms |
906 ms |
0.53 |
Update one schema (1 existing) |
16 ms |
18 ms |
0.89 |
Update one schema (101 existing) |
20 ms |
24 ms |
0.83 |
Update one schema (1001 existing) |
59 ms |
73 ms |
0.81 |
Update one schema (10001 existing) |
493 ms |
836 ms |
0.59 |
Cached rebuild (1 existing) |
9 ms |
9 ms |
1 |
Cached rebuild (101 existing) |
9 ms |
12 ms |
0.75 |
Cached rebuild (1001 existing) |
22 ms |
31 ms |
0.71 |
Cached rebuild (10001 existing) |
164 ms |
258 ms |
0.64 |
Index 100 schemas |
88 ms |
114 ms |
0.77 |
Index 1000 schemas |
1150 ms |
910 ms |
1.26 |
Index 10000 schemas |
13831 ms |
13258 ms |
1.04 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Juan Cruz Viotti jv@jviotti.com