Implement a test runner module #574
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Bindings (linux/nodejs)
Details
| Benchmark suite | Current: b781460 | Previous: cd5ebdb | Ratio |
|---|---|---|---|
E2E_Parse_BlazeJSON_ui5_manifest |
108 ops/sec (±0.36%) |
111 ops/sec (±0.28%) |
1.03 |
E2E_Parse_JSON_ui5_manifest |
794 ops/sec (±0.40%) |
803 ops/sec (±0.36%) |
1.01 |
E2E_Evaluator_adaptivecard |
14095 ops/sec (±0.20%) |
14170 ops/sec (±0.27%) |
1.01 |
E2E_Evaluator_ansible_meta |
1818 ops/sec (±0.25%) |
1850 ops/sec (±0.13%) |
1.02 |
E2E_Evaluator_aws_cdk |
2646 ops/sec (±0.11%) |
2721 ops/sec (±0.12%) |
1.03 |
E2E_Evaluator_babelrc |
1480 ops/sec (±0.16%) |
1522 ops/sec (±0.48%) |
1.03 |
E2E_Evaluator_clang_format |
4949 ops/sec (±1.05%) |
5013 ops/sec (±1.01%) |
1.01 |
E2E_Evaluator_cmake_presets |
162 ops/sec (±0.84%) |
159 ops/sec (±1.07%) |
0.98 |
E2E_Evaluator_code_climate |
535 ops/sec (±0.15%) |
557 ops/sec (±0.18%) |
1.04 |
E2E_Evaluator_cql2 |
2758 ops/sec (±0.40%) |
2781 ops/sec (±0.39%) |
1.01 |
E2E_Evaluator_cspell |
764 ops/sec (±0.18%) |
781 ops/sec (±0.13%) |
1.02 |
E2E_Evaluator_cypress |
1202 ops/sec (±0.86%) |
1238 ops/sec (±0.98%) |
1.03 |
E2E_Evaluator_deno |
936 ops/sec (±0.18%) |
958 ops/sec (±0.16%) |
1.02 |
E2E_Evaluator_dependabot |
812 ops/sec (±0.19%) |
837 ops/sec (±0.13%) |
1.03 |
E2E_Evaluator_draft_04 |
107 ops/sec (±1.11%) |
118 ops/sec (±1.23%) |
1.10 |
E2E_Evaluator_fabric_mod |
584 ops/sec (±0.14%) |
586 ops/sec (±0.21%) |
1.00 |
E2E_Evaluator_geojson |
37.48 ops/sec (±0.31%) |
38.61 ops/sec (±0.18%) |
1.03 |
E2E_Evaluator_gitpod_configuration |
1024 ops/sec (±0.16%) |
1059 ops/sec (±0.15%) |
1.03 |
E2E_Evaluator_helm_chart_lock |
339 ops/sec (±0.20%) |
355 ops/sec (±0.10%) |
1.05 |
E2E_Evaluator_importmap |
1499 ops/sec (±0.18%) |
1574 ops/sec (±0.13%) |
1.05 |
E2E_Evaluator_jasmine |
1229 ops/sec (±0.15%) |
1276 ops/sec (±0.12%) |
1.04 |
E2E_Evaluator_jsconfig |
827 ops/sec (±0.87%) |
863 ops/sec (±0.49%) |
1.04 |
E2E_Evaluator_jshintrc |
583 ops/sec (±0.13%) |
583 ops/sec (±0.13%) |
1 |
E2E_Evaluator_krakend |
3404 ops/sec (±0.18%) |
3427 ops/sec (±0.20%) |
1.01 |
E2E_Evaluator_lazygit |
2652 ops/sec (±0.17%) |
2692 ops/sec (±0.18%) |
1.02 |
E2E_Evaluator_lerna |
1209 ops/sec (±0.20%) |
1248 ops/sec (±0.16%) |
1.03 |
E2E_Evaluator_nest_cli |
1078 ops/sec (±0.14%) |
1113 ops/sec (±0.11%) |
1.03 |
E2E_Evaluator_omc |
21782 ops/sec (±0.13%) |
22309 ops/sec (±0.10%) |
1.02 |
E2E_Evaluator_omnisharp |
787 ops/sec (±0.12%) |
798 ops/sec (±0.46%) |
1.01 |
E2E_Evaluator_openapi |
72.3 ops/sec (±1.02%) |
80.81 ops/sec (±0.97%) |
1.12 |
E2E_Evaluator_pre_commit_hooks |
781 ops/sec (±0.11%) |
794 ops/sec (±0.39%) |
1.02 |
E2E_Evaluator_pulumi |
284 ops/sec (±0.19%) |
293 ops/sec (±0.14%) |
1.03 |
E2E_Evaluator_semantic_release |
1412 ops/sec (±0.12%) |
1539 ops/sec (±0.32%) |
1.09 |
E2E_Evaluator_stale |
1216 ops/sec (±0.17%) |
1250 ops/sec (±0.10%) |
1.03 |
E2E_Evaluator_stylecop |
951 ops/sec (±0.19%) |
982 ops/sec (±0.64%) |
1.03 |
E2E_Evaluator_tmuxinator |
2577 ops/sec (±0.13%) |
2641 ops/sec (±0.17%) |
1.02 |
E2E_Evaluator_ui5 |
699 ops/sec (±0.15%) |
713 ops/sec (±0.23%) |
1.02 |
E2E_Evaluator_ui5_manifest |
245 ops/sec (±0.37%) |
245 ops/sec (±0.47%) |
1 |
E2E_Evaluator_unreal_engine_uproject |
788 ops/sec (±0.18%) |
808 ops/sec (±0.17%) |
1.03 |
E2E_Evaluator_vercel |
1199 ops/sec (±0.14%) |
1230 ops/sec (±0.15%) |
1.03 |
E2E_Evaluator_yamllint |
1597 ops/sec (±0.16%) |
1672 ops/sec (±0.12%) |
1.05 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Benchmark (linux/llvm)
Details
| Benchmark suite | Current: b781460 | Previous: cd5ebdb | Ratio |
|---|---|---|---|
E2E_Compiler_adaptivecard |
992238487.9999981 ns/iter |
976481239.9999982 ns/iter |
1.02 |
E2E_Compiler_ansible_meta |
28525421.41666703 ns/iter |
28097554.480000325 ns/iter |
1.02 |
E2E_Compiler_aws_cdk |
530527.3106060757 ns/iter |
531222.3011406085 ns/iter |
1.00 |
E2E_Compiler_babelrc |
3523237.472361827 ns/iter |
3531399.111111431 ns/iter |
1.00 |
E2E_Compiler_clang_format |
18919156.56756897 ns/iter |
18927850.702702064 ns/iter |
1.00 |
E2E_Compiler_cmake_presets |
212741694.66666612 ns/iter |
211636481.66664946 ns/iter |
1.01 |
E2E_Compiler_code_climate |
2966809.123404346 ns/iter |
2980722.5940171285 ns/iter |
1.00 |
E2E_Compiler_cql2 |
166789227.99999896 ns/iter |
167288049.50000152 ns/iter |
1.00 |
E2E_Compiler_cspell |
57349413.615384035 ns/iter |
55652126.46153073 ns/iter |
1.03 |
E2E_Compiler_cypress |
6831229.326732596 ns/iter |
6711340.523809537 ns/iter |
1.02 |
E2E_Compiler_deno |
8667938.72839542 ns/iter |
8635548.641975366 ns/iter |
1.00 |
E2E_Compiler_dependabot |
4614042.370860715 ns/iter |
4627986.973333312 ns/iter |
1.00 |
E2E_Compiler_draft_04 |
17445812.900000133 ns/iter |
17626309.950000517 ns/iter |
0.99 |
E2E_Compiler_fabric_mod |
6182560.663716637 ns/iter |
6165314.584069946 ns/iter |
1.00 |
E2E_Compiler_geojson |
29339876.499998733 ns/iter |
28792218.958334576 ns/iter |
1.02 |
E2E_Compiler_gitpod_configuration |
6330406.180180173 ns/iter |
6370055.945454954 ns/iter |
0.99 |
E2E_Compiler_helm_chart_lock |
887211.4728191731 ns/iter |
891136.7108280241 ns/iter |
1.00 |
E2E_Compiler_importmap |
442677.08684708946 ns/iter |
443398.48857864155 ns/iter |
1.00 |
E2E_Compiler_jasmine |
1770349.2896725317 ns/iter |
1768196.7499999765 ns/iter |
1.00 |
E2E_Compiler_jshintrc |
4651997.113333361 ns/iter |
4643784.473333502 ns/iter |
1.00 |
E2E_Compiler_jsconfig |
24727400.214286733 ns/iter |
23636748.100000203 ns/iter |
1.05 |
E2E_Compiler_krakend |
261721086.6666634 ns/iter |
247437740.9999988 ns/iter |
1.06 |
E2E_Compiler_lazygit |
41463231.94117606 ns/iter |
38889541.444442935 ns/iter |
1.07 |
E2E_Compiler_lerna |
1907154.8016305158 ns/iter |
1917587.1813188423 ns/iter |
0.99 |
E2E_Compiler_nest_cli |
10393822.41790989 ns/iter |
10383147.764706697 ns/iter |
1.00 |
E2E_Compiler_omc |
1578505107.0000122 ns/iter |
1629284554.999913 ns/iter |
0.97 |
E2E_Compiler_omnisharp |
7827385.23333377 ns/iter |
7834484.300000592 ns/iter |
1.00 |
E2E_Compiler_openapi |
1139260210.9999642 ns/iter |
1149688866.0000196 ns/iter |
0.99 |
E2E_Compiler_pre_commit_hooks |
5140311.6592593035 ns/iter |
5134856.007407506 ns/iter |
1.00 |
E2E_Compiler_pulumi |
4855637.1805557795 ns/iter |
4885946.8951046765 ns/iter |
0.99 |
E2E_Compiler_semantic_release |
1826371.1540471057 ns/iter |
1841241.522309693 ns/iter |
0.99 |
E2E_Compiler_stale |
2164470.6358023244 ns/iter |
2184345.6105921813 ns/iter |
0.99 |
E2E_Compiler_stylecop |
6610025.566037899 ns/iter |
6677158.142856786 ns/iter |
0.99 |
E2E_Compiler_tmuxinator |
2329911.4433333063 ns/iter |
2351858.916107548 ns/iter |
0.99 |
E2E_Compiler_ui5 |
154479905.5999988 ns/iter |
152596730.79999632 ns/iter |
1.01 |
E2E_Compiler_ui5_manifest |
2054978669.9999685 ns/iter |
2094620062.9999795 ns/iter |
0.98 |
E2E_Compiler_unreal_engine_uproject |
5285362.091603234 ns/iter |
5285192.44615364 ns/iter |
1.00 |
E2E_Compiler_vercel |
16895247.976190742 ns/iter |
16752523.119047662 ns/iter |
1.01 |
E2E_Compiler_yamllint |
448464.27405991626 ns/iter |
451144.9271437609 ns/iter |
0.99 |
E2E_Evaluator_adaptivecard |
53619.25700393435 ns/iter |
56458.31268626855 ns/iter |
0.95 |
E2E_Evaluator_ansible_meta |
320638.0220284629 ns/iter |
324065.1570363823 ns/iter |
0.99 |
E2E_Evaluator_aws_cdk |
71108.53228266034 ns/iter |
71066.64432160929 ns/iter |
1.00 |
E2E_Evaluator_babelrc |
161680.19776899338 ns/iter |
161072.39555349512 ns/iter |
1.00 |
E2E_Evaluator_clang_format |
137466.4458065735 ns/iter |
139856.81810976248 ns/iter |
0.98 |
E2E_Evaluator_cmake_presets |
5533801.104838913 ns/iter |
5536961.368000447 ns/iter |
1.00 |
E2E_Evaluator_code_climate |
281144.8467387039 ns/iter |
276706.9644409468 ns/iter |
1.02 |
E2E_Evaluator_cql2 |
259593.66030677495 ns/iter |
266696.8557729571 ns/iter |
0.97 |
E2E_Evaluator_cspell |
733018.6912681896 ns/iter |
742070.9543039295 ns/iter |
0.99 |
E2E_Evaluator_cypress |
229605.4826789808 ns/iter |
230433.96309840633 ns/iter |
1.00 |
E2E_Evaluator_deno |
393983.11650487705 ns/iter |
400766.842683665 ns/iter |
0.98 |
E2E_Evaluator_dependabot |
567714.3794212138 ns/iter |
568841.5527597034 ns/iter |
1.00 |
E2E_Evaluator_draft_04 |
9465081.460526312 ns/iter |
8247751.831325068 ns/iter |
1.15 |
E2E_Evaluator_fabric_mod |
951739.0728745448 ns/iter |
973940.4979368066 ns/iter |
0.98 |
E2E_Evaluator_geojson |
30848813.08695764 ns/iter |
30601607.739126652 ns/iter |
1.01 |
E2E_Evaluator_gitpod_configuration |
329202.6140845013 ns/iter |
343669.8299711936 ns/iter |
0.96 |
E2E_Evaluator_helm_chart_lock |
463822.41789051483 ns/iter |
465688.9300931634 ns/iter |
1.00 |
E2E_Evaluator_importmap |
73399.05751744464 ns/iter |
74071.16996006374 ns/iter |
0.99 |
E2E_Evaluator_jasmine |
168199.8237959026 ns/iter |
168113.1217917642 ns/iter |
1.00 |
E2E_Evaluator_jshintrc |
1323805.0284091297 ns/iter |
1332317.903408907 ns/iter |
0.99 |
E2E_Evaluator_jsconfig |
518895.84473879903 ns/iter |
523501.4377810795 ns/iter |
0.99 |
E2E_Evaluator_krakend |
265380.95694496017 ns/iter |
277879.77724358375 ns/iter |
0.96 |
E2E_Evaluator_lazygit |
189320.92605730463 ns/iter |
192920.19811579533 ns/iter |
0.98 |
E2E_Evaluator_lerna |
202741.3643523946 ns/iter |
205793.88797652902 ns/iter |
0.99 |
E2E_Evaluator_nest_cli |
261784.58919926937 ns/iter |
260575.79738318335 ns/iter |
1.00 |
E2E_Evaluator_omc |
30083.646519907874 ns/iter |
31186.281065219417 ns/iter |
0.96 |
E2E_Evaluator_omnisharp |
720907.5702564568 ns/iter |
730263.2223381959 ns/iter |
0.99 |
E2E_Evaluator_openapi |
15586516.348838191 ns/iter |
14727365.12765947 ns/iter |
1.06 |
E2E_Evaluator_pre_commit_hooks |
619629.0572687165 ns/iter |
619330.3421751412 ns/iter |
1.00 |
E2E_Evaluator_pulumi |
895634.9097840163 ns/iter |
909989.7961538645 ns/iter |
0.98 |
E2E_Evaluator_semantic_release |
140700.49008899357 ns/iter |
141360.21891454552 ns/iter |
1.00 |
E2E_Evaluator_stale |
209513.10360360012 ns/iter |
210274.86756756352 ns/iter |
1.00 |
E2E_Evaluator_stylecop |
377664.8023567121 ns/iter |
384769.37090507103 ns/iter |
0.98 |
E2E_Evaluator_tmuxinator |
139565.59268051575 ns/iter |
141270.13902148948 ns/iter |
0.99 |
E2E_Evaluator_ui5 |
714490.6501547473 ns/iter |
733587.6795407106 ns/iter |
0.97 |
E2E_Evaluator_ui5_manifest |
3478283.2878050148 ns/iter |
3514990.9752470935 ns/iter |
0.99 |
E2E_Evaluator_unreal_engine_uproject |
549022.7611589474 ns/iter |
551783.2798418729 ns/iter |
0.99 |
E2E_Evaluator_vercel |
400312.9536878229 ns/iter |
400686.14179530286 ns/iter |
1.00 |
E2E_Evaluator_yamllint |
17493.40179426438 ns/iter |
17336.288851521615 ns/iter |
1.01 |
Micro_Draft4_Meta_1_No_Callback |
295.1779987361789 ns/iter |
285.38542074444996 ns/iter |
1.03 |
Micro_Draft4_Required_Properties |
722.6694406493868 ns/iter |
719.0469879220277 ns/iter |
1.01 |
Micro_Draft4_Many_Optional_Properties_Minimal_Match |
34.74091391373872 ns/iter |
35.230674787429315 ns/iter |
0.99 |
Micro_Draft4_Few_Optional_Properties_Minimal_Match |
16.185697412061852 ns/iter |
16.187962942784026 ns/iter |
1.00 |
Micro_Draft4_Items_Schema |
659.5243755102169 ns/iter |
656.5166224739759 ns/iter |
1.00 |
Micro_Draft4_Nested_Object |
7.472358255796923 ns/iter |
7.471700717759935 ns/iter |
1.00 |
Micro_Draft4_Properties_Triad_Optional |
384.6231981132985 ns/iter |
383.8481738762746 ns/iter |
1.00 |
Micro_Draft4_Properties_Triad_Closed |
320.11538257196787 ns/iter |
321.78776803363587 ns/iter |
0.99 |
Micro_Draft4_Properties_Triad_Required |
397.15501590631084 ns/iter |
400.6840219383106 ns/iter |
0.99 |
Micro_Draft4_Properties_Closed |
89.08276218440528 ns/iter |
88.36166744459415 ns/iter |
1.01 |
Micro_Draft4_Non_Recursive_Ref |
36.069533541533055 ns/iter |
36.08646298688111 ns/iter |
1.00 |
Micro_Draft4_Pattern_Properties_True |
263.88013627648513 ns/iter |
263.3691508804347 ns/iter |
1.00 |
Micro_Draft4_Ref_To_Single_Property |
16.18499002084948 ns/iter |
16.177484755575968 ns/iter |
1.00 |
Micro_Draft4_Additional_Properties_Type |
63.777105212675586 ns/iter |
63.775705606512304 ns/iter |
1.00 |
Micro_Draft4_Nested_Oneof |
89.91182569725835 ns/iter |
89.90881873905268 ns/iter |
1.00 |
Micro_Draft4_Short_Enum |
19.023646005379288 ns/iter |
19.044499598745055 ns/iter |
1.00 |
Micro_Draft4_Long_Enum |
23.678838359253405 ns/iter |
23.65662249048965 ns/iter |
1.00 |
Micro_Draft4_Long_Enum_Short_Strings |
18.669833048606424 ns/iter |
18.671922756322672 ns/iter |
1.00 |
Micro_Draft4_Type_Object |
13.384349721464032 ns/iter |
13.471664598303091 ns/iter |
0.99 |
Micro_Draft4_Ref_Single_100 |
3395874.8846152695 ns/iter |
3414384.6796116596 ns/iter |
0.99 |
Micro_Draft4_Ref_Many_Nested |
4571677317.999956 ns/iter |
4165575939.000064 ns/iter |
1.10 |
Micro_Draft6_Property_Names |
134.9652935648986 ns/iter |
129.68323740379913 ns/iter |
1.04 |
Micro_Draft7_If_Then_Else |
42.02090393595256 ns/iter |
42.026354619936804 ns/iter |
1.00 |
Micro_2019_09_Unevaluated_Properties |
55.38302007715209 ns/iter |
55.39318532594191 ns/iter |
1.00 |
Micro_2020_12_Dynamic_Ref |
522.7837901116345 ns/iter |
519.3791453967677 ns/iter |
1.01 |
Micro_2020_12_Dynamic_Ref_Single |
13.381150373143694 ns/iter |
13.388418544870857 ns/iter |
1.00 |
Micro_2020_12_Simple_Output_Mask |
139656.80881195673 ns/iter |
137770.1862899991 ns/iter |
1.01 |
Micro_2020_12_Simple_Output_Annotations |
167421.0302093283 ns/iter |
166441.63863744301 ns/iter |
1.01 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Benchmark (macos/llvm)
Details
| Benchmark suite | Current: b781460 | Previous: cd5ebdb | Ratio |
|---|---|---|---|
E2E_Compiler_adaptivecard |
614313625.0000225 ns/iter |
625416042.0000972 ns/iter |
0.98 |
E2E_Compiler_ansible_meta |
22073731.19354968 ns/iter |
22212260.741936564 ns/iter |
0.99 |
E2E_Compiler_aws_cdk |
328356.5959595841 ns/iter |
324474.60937501054 ns/iter |
1.01 |
E2E_Compiler_babelrc |
2397585.6254294575 ns/iter |
2425903.350515525 ns/iter |
0.99 |
E2E_Compiler_clang_format |
14190365.196082342 ns/iter |
15109247.596154336 ns/iter |
0.94 |
E2E_Compiler_cmake_presets |
152753916.59997696 ns/iter |
142923574.99998614 ns/iter |
1.07 |
E2E_Compiler_code_climate |
2182565.2515722057 ns/iter |
2063212.5132743947 ns/iter |
1.06 |
E2E_Compiler_cql2 |
112187270.8333361 ns/iter |
115373604.16665858 ns/iter |
0.97 |
E2E_Compiler_cspell |
35389564.59999554 ns/iter |
37683688.57894482 ns/iter |
0.94 |
E2E_Compiler_cypress |
4666199.544216812 ns/iter |
5590862.920000745 ns/iter |
0.83 |
E2E_Compiler_deno |
6698857.431372602 ns/iter |
6892021.038834394 ns/iter |
0.97 |
E2E_Compiler_dependabot |
3501728.348038903 ns/iter |
3493069.074865925 ns/iter |
1.00 |
E2E_Compiler_draft_04 |
12140255.38709965 ns/iter |
11739262.98360666 ns/iter |
1.03 |
E2E_Compiler_fabric_mod |
4559021.10526327 ns/iter |
4488212.556962141 ns/iter |
1.02 |
E2E_Compiler_geojson |
24024885.741937395 ns/iter |
23625633.620691463 ns/iter |
1.02 |
E2E_Compiler_gitpod_configuration |
4859152.365672493 ns/iter |
4847918.089042227 ns/iter |
1.00 |
E2E_Compiler_helm_chart_lock |
570563.6741293642 ns/iter |
612942.5153473803 ns/iter |
0.93 |
E2E_Compiler_importmap |
265581.9888589699 ns/iter |
259555.33256967698 ns/iter |
1.02 |
E2E_Compiler_jasmine |
1243237.7572994262 ns/iter |
1218149.8556524077 ns/iter |
1.02 |
E2E_Compiler_jshintrc |
3211913.6363627897 ns/iter |
2983494.7478991486 ns/iter |
1.08 |
E2E_Compiler_jsconfig |
18945536.4374993 ns/iter |
16947322.14634048 ns/iter |
1.12 |
E2E_Compiler_krakend |
225850760.2500117 ns/iter |
187826197.999982 ns/iter |
1.20 |
E2E_Compiler_lazygit |
38820175.57142652 ns/iter |
31263338.78260638 ns/iter |
1.24 |
E2E_Compiler_lerna |
1402710.8433736314 ns/iter |
1324656.76037752 ns/iter |
1.06 |
E2E_Compiler_nest_cli |
7191125.44086011 ns/iter |
6760978.138613204 ns/iter |
1.06 |
E2E_Compiler_omc |
1216721000.000007 ns/iter |
1187716707.9999254 ns/iter |
1.02 |
E2E_Compiler_omnisharp |
5859147.354544803 ns/iter |
5914019.199998738 ns/iter |
0.99 |
E2E_Compiler_openapi |
799784292.0001403 ns/iter |
776718750.0001 ns/iter |
1.03 |
E2E_Compiler_pre_commit_hooks |
3558151.0666664448 ns/iter |
3807935.0376346083 ns/iter |
0.93 |
E2E_Compiler_pulumi |
3226422.837962496 ns/iter |
3564157.4285713644 ns/iter |
0.91 |
E2E_Compiler_semantic_release |
1254381.7593583937 ns/iter |
1375997.6415095257 ns/iter |
0.91 |
E2E_Compiler_stale |
1572751.2098214293 ns/iter |
1624640.2298852392 ns/iter |
0.97 |
E2E_Compiler_stylecop |
5408544.992753602 ns/iter |
5300346.694656659 ns/iter |
1.02 |
E2E_Compiler_tmuxinator |
1631351.9794520908 ns/iter |
1669442.7679814948 ns/iter |
0.98 |
E2E_Compiler_ui5 |
122420486.00001 ns/iter |
123184708.40000372 ns/iter |
0.99 |
E2E_Compiler_ui5_manifest |
1611164375.0000439 ns/iter |
1514917833.0000267 ns/iter |
1.06 |
E2E_Compiler_unreal_engine_uproject |
4118812.1345036924 ns/iter |
3524921.270717797 ns/iter |
1.17 |
E2E_Compiler_vercel |
15640978.723405033 ns/iter |
12548912.946426427 ns/iter |
1.25 |
E2E_Compiler_yamllint |
308977.34788238816 ns/iter |
274727.4526844453 ns/iter |
1.12 |
E2E_Evaluator_adaptivecard |
35150.04967406993 ns/iter |
35564.89795918034 ns/iter |
0.99 |
E2E_Evaluator_ansible_meta |
182994.31966573675 ns/iter |
182717.521282041 ns/iter |
1.00 |
E2E_Evaluator_aws_cdk |
39537.99764282558 ns/iter |
39401.264276762275 ns/iter |
1.00 |
E2E_Evaluator_babelrc |
93868.61886337941 ns/iter |
93900.70763897341 ns/iter |
1.00 |
E2E_Evaluator_clang_format |
100109.74143613978 ns/iter |
101309.58972175016 ns/iter |
0.99 |
E2E_Evaluator_cmake_presets |
3656651.155555816 ns/iter |
3746465.1693125516 ns/iter |
0.98 |
E2E_Evaluator_code_climate |
158148.97773822598 ns/iter |
166439.87095237986 ns/iter |
0.95 |
E2E_Evaluator_cql2 |
176142.12351187263 ns/iter |
170201.0995860744 ns/iter |
1.03 |
E2E_Evaluator_cspell |
450888.75636135275 ns/iter |
433742.8995006959 ns/iter |
1.04 |
E2E_Evaluator_cypress |
155761.7754878114 ns/iter |
145791.33639210995 ns/iter |
1.07 |
E2E_Evaluator_deno |
260594.00253529396 ns/iter |
233190.76900674263 ns/iter |
1.12 |
E2E_Evaluator_dependabot |
362889.0771637424 ns/iter |
336881.8392944498 ns/iter |
1.08 |
E2E_Evaluator_draft_04 |
5579301.429687433 ns/iter |
5448874.031007877 ns/iter |
1.02 |
E2E_Evaluator_fabric_mod |
561598.4816970584 ns/iter |
553173.9357652466 ns/iter |
1.02 |
E2E_Evaluator_geojson |
15314460.13043676 ns/iter |
15569190.755554194 ns/iter |
0.98 |
E2E_Evaluator_gitpod_configuration |
185661.47664488884 ns/iter |
178102.31453035807 ns/iter |
1.04 |
E2E_Evaluator_helm_chart_lock |
246236.13547483302 ns/iter |
258446.97119943317 ns/iter |
0.95 |
E2E_Evaluator_importmap |
40308.720588231576 ns/iter |
50034.668319100594 ns/iter |
0.81 |
E2E_Evaluator_jasmine |
101442.86013106946 ns/iter |
138282.498427198 ns/iter |
0.73 |
E2E_Evaluator_jshintrc |
947555.6106666166 ns/iter |
1008985.4166665191 ns/iter |
0.94 |
E2E_Evaluator_jsconfig |
356333.8129496639 ns/iter |
358872.8168663696 ns/iter |
0.99 |
E2E_Evaluator_krakend |
161707.77281900815 ns/iter |
156032.00475882637 ns/iter |
1.04 |
E2E_Evaluator_lazygit |
109342.75563794145 ns/iter |
104415.974303737 ns/iter |
1.05 |
E2E_Evaluator_lerna |
127069.13266967553 ns/iter |
126803.96884513821 ns/iter |
1.00 |
E2E_Evaluator_nest_cli |
150066.89759691508 ns/iter |
143281.92211314643 ns/iter |
1.05 |
E2E_Evaluator_omc |
19334.915115254036 ns/iter |
19157.965388745866 ns/iter |
1.01 |
E2E_Evaluator_omnisharp |
443040.27061865287 ns/iter |
443953.77584130265 ns/iter |
1.00 |
E2E_Evaluator_openapi |
9052389.064935325 ns/iter |
8628591.555555958 ns/iter |
1.05 |
E2E_Evaluator_pre_commit_hooks |
366975.22948157834 ns/iter |
365359.80225413246 ns/iter |
1.00 |
E2E_Evaluator_pulumi |
567268.518518585 ns/iter |
566773.9616613584 ns/iter |
1.00 |
E2E_Evaluator_semantic_release |
92014.93024739178 ns/iter |
86692.84219141841 ns/iter |
1.06 |
E2E_Evaluator_stale |
128547.35663080562 ns/iter |
119000.53524933937 ns/iter |
1.08 |
E2E_Evaluator_stylecop |
236355.3509050479 ns/iter |
203725.49369313833 ns/iter |
1.16 |
E2E_Evaluator_tmuxinator |
82974.66928117907 ns/iter |
77946.72273782798 ns/iter |
1.06 |
E2E_Evaluator_ui5 |
384445.1718494451 ns/iter |
384343.1193780908 ns/iter |
1.00 |
E2E_Evaluator_ui5_manifest |
1884326.6574585917 ns/iter |
2115721.1538458304 ns/iter |
0.89 |
E2E_Evaluator_unreal_engine_uproject |
355245.09909912007 ns/iter |
366366.055907158 ns/iter |
0.97 |
E2E_Evaluator_vercel |
242953.36302246354 ns/iter |
240004.98877557513 ns/iter |
1.01 |
E2E_Evaluator_yamllint |
11927.054541338017 ns/iter |
11331.25778582865 ns/iter |
1.05 |
Micro_Draft4_Meta_1_No_Callback |
192.76244791622506 ns/iter |
190.66885405596872 ns/iter |
1.01 |
Micro_Draft4_Required_Properties |
459.5775472665097 ns/iter |
452.17978983086545 ns/iter |
1.02 |
Micro_Draft4_Many_Optional_Properties_Minimal_Match |
20.03645651252015 ns/iter |
19.43403185842849 ns/iter |
1.03 |
Micro_Draft4_Few_Optional_Properties_Minimal_Match |
7.402960797925244 ns/iter |
7.041698724216116 ns/iter |
1.05 |
Micro_Draft4_Items_Schema |
454.4166770043688 ns/iter |
430.0208290823304 ns/iter |
1.06 |
Micro_Draft4_Nested_Object |
3.1950498520779886 ns/iter |
2.840750016478541 ns/iter |
1.12 |
Micro_Draft4_Properties_Triad_Optional |
256.59010890114195 ns/iter |
295.4537908818736 ns/iter |
0.87 |
Micro_Draft4_Properties_Triad_Closed |
221.3753246147529 ns/iter |
244.8474853547934 ns/iter |
0.90 |
Micro_Draft4_Properties_Triad_Required |
283.09646226044595 ns/iter |
302.2964240153556 ns/iter |
0.94 |
Micro_Draft4_Properties_Closed |
49.81175805881264 ns/iter |
47.226536301125876 ns/iter |
1.05 |
Micro_Draft4_Non_Recursive_Ref |
21.420743233521975 ns/iter |
21.322903642241574 ns/iter |
1.00 |
Micro_Draft4_Pattern_Properties_True |
154.1231231303094 ns/iter |
156.90408023225538 ns/iter |
0.98 |
Micro_Draft4_Ref_To_Single_Property |
7.24013934458756 ns/iter |
7.396500669978916 ns/iter |
0.98 |
Micro_Draft4_Additional_Properties_Type |
42.11977027067912 ns/iter |
43.3088491641169 ns/iter |
0.97 |
Micro_Draft4_Nested_Oneof |
58.022148087201664 ns/iter |
55.806945811317775 ns/iter |
1.04 |
Micro_Draft4_Short_Enum |
8.298636249022907 ns/iter |
8.636491758004997 ns/iter |
0.96 |
Micro_Draft4_Long_Enum |
22.984921711549497 ns/iter |
22.818264706551364 ns/iter |
1.01 |
Micro_Draft4_Long_Enum_Short_Strings |
10.365854585683339 ns/iter |
10.50864742842955 ns/iter |
0.99 |
Micro_Draft4_Type_Object |
6.0547448594933755 ns/iter |
5.981651831525054 ns/iter |
1.01 |
Micro_Draft4_Ref_Single_100 |
2288043.69480521 ns/iter |
2229402.9498435496 ns/iter |
1.03 |
Micro_Draft4_Ref_Many_Nested |
2860674749.9998164 ns/iter |
2678493708.0001326 ns/iter |
1.07 |
Micro_Draft6_Property_Names |
100.5785916572177 ns/iter |
95.28525987403009 ns/iter |
1.06 |
Micro_Draft7_If_Then_Else |
31.90386750519394 ns/iter |
26.709709110079963 ns/iter |
1.19 |
Micro_2019_09_Unevaluated_Properties |
36.319437933758124 ns/iter |
35.12707035640309 ns/iter |
1.03 |
Micro_2020_12_Dynamic_Ref |
408.32256619257424 ns/iter |
408.05631026998657 ns/iter |
1.00 |
Micro_2020_12_Dynamic_Ref_Single |
5.919178291512485 ns/iter |
5.912587476985444 ns/iter |
1.00 |
Micro_2020_12_Simple_Output_Mask |
130964.46022623223 ns/iter |
128729.56732025118 ns/iter |
1.02 |
Micro_2020_12_Simple_Output_Annotations |
146984.8169744563 ns/iter |
146500.4927206099 ns/iter |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Benchmark (linux/gcc)
Details
| Benchmark suite | Current: b781460 | Previous: cd5ebdb | Ratio |
|---|---|---|---|
Micro_2020_12_Dynamic_Ref |
615.7217597338081 ns/iter |
656.5926348532055 ns/iter |
0.94 |
Micro_2020_12_Dynamic_Ref_Single |
11.630511906783699 ns/iter |
9.633644380681618 ns/iter |
1.21 |
Micro_2020_12_Simple_Output_Mask |
138200.5587320109 ns/iter |
125036.5594855311 ns/iter |
1.11 |
Micro_2020_12_Simple_Output_Annotations |
175236.530815126 ns/iter |
166401.71703034863 ns/iter |
1.05 |
Micro_2019_09_Unevaluated_Properties |
50.3682681498177 ns/iter |
42.91112526187555 ns/iter |
1.17 |
Micro_Draft7_If_Then_Else |
46.74020811022788 ns/iter |
39.559339815909055 ns/iter |
1.18 |
Micro_Draft6_Property_Names |
175.27275135426902 ns/iter |
157.19178998958253 ns/iter |
1.12 |
Micro_Draft4_Meta_1_No_Callback |
251.14021423149063 ns/iter |
252.40980529979342 ns/iter |
0.99 |
Micro_Draft4_Required_Properties |
538.5375749991946 ns/iter |
566.8450988744831 ns/iter |
0.95 |
Micro_Draft4_Many_Optional_Properties_Minimal_Match |
31.504255393325394 ns/iter |
34.0712383290029 ns/iter |
0.92 |
Micro_Draft4_Few_Optional_Properties_Minimal_Match |
13.52543007877098 ns/iter |
12.147187230707392 ns/iter |
1.11 |
Micro_Draft4_Items_Schema |
549.0061845755368 ns/iter |
470.3494595790798 ns/iter |
1.17 |
Micro_Draft4_Nested_Object |
6.964365919549836 ns/iter |
5.87137952786087 ns/iter |
1.19 |
Micro_Draft4_Properties_Triad_Optional |
471.06971101964785 ns/iter |
428.2128406005662 ns/iter |
1.10 |
Micro_Draft4_Properties_Triad_Closed |
378.7318685068029 ns/iter |
356.13798548004263 ns/iter |
1.06 |
Micro_Draft4_Properties_Triad_Required |
462.684408538821 ns/iter |
437.934512896222 ns/iter |
1.06 |
Micro_Draft4_Properties_Closed |
79.92925569881929 ns/iter |
82.41561409604061 ns/iter |
0.97 |
Micro_Draft4_Non_Recursive_Ref |
16.532611929594026 ns/iter |
13.684173704112723 ns/iter |
1.21 |
Micro_Draft4_Pattern_Properties_True |
250.8179434279723 ns/iter |
215.00831738819008 ns/iter |
1.17 |
Micro_Draft4_Ref_To_Single_Property |
13.534331807903657 ns/iter |
12.196477507319818 ns/iter |
1.11 |
Micro_Draft4_Additional_Properties_Type |
21.20873462882943 ns/iter |
18.29859423346292 ns/iter |
1.16 |
Micro_Draft4_Nested_Oneof |
114.05302880532962 ns/iter |
103.96960501860882 ns/iter |
1.10 |
Micro_Draft4_Short_Enum |
18.698683330999735 ns/iter |
14.908442377000364 ns/iter |
1.25 |
Micro_Draft4_Long_Enum |
20.960132471377896 ns/iter |
17.60922041768511 ns/iter |
1.19 |
Micro_Draft4_Long_Enum_Short_Strings |
18.703191940156476 ns/iter |
15.153904870769358 ns/iter |
1.23 |
Micro_Draft4_Type_Object |
11.593898530226427 ns/iter |
9.605858752341334 ns/iter |
1.21 |
Micro_Draft4_Ref_Single_100 |
3546781.6300001685 ns/iter |
3193890.155251431 ns/iter |
1.11 |
Micro_Draft4_Ref_Many_Nested |
5138912718.000029 ns/iter |
5060047315.00002 ns/iter |
1.02 |
E2E_Compiler_adaptivecard |
1000037509.0000944 ns/iter |
926747917.000057 ns/iter |
1.08 |
E2E_Compiler_ansible_meta |
31503379.272728372 ns/iter |
28308221.37499922 ns/iter |
1.11 |
E2E_Compiler_aws_cdk |
591641.0859309848 ns/iter |
565137.7152103005 ns/iter |
1.05 |
E2E_Compiler_babelrc |
3920455.2471911293 ns/iter |
3602610.458762763 ns/iter |
1.09 |
E2E_Compiler_clang_format |
21064022.121215727 ns/iter |
19389189.722223442 ns/iter |
1.09 |
E2E_Compiler_cmake_presets |
217304730.6666831 ns/iter |
206708914.99999774 ns/iter |
1.05 |
E2E_Compiler_code_climate |
3361515.421052839 ns/iter |
3104022.4424779722 ns/iter |
1.08 |
E2E_Compiler_cql2 |
178457341.5000068 ns/iter |
167694997.7500044 ns/iter |
1.06 |
E2E_Compiler_cspell |
58389325.33331823 ns/iter |
54063776.84615703 ns/iter |
1.08 |
E2E_Compiler_cypress |
7068760.606060353 ns/iter |
6643961.771428189 ns/iter |
1.06 |
E2E_Compiler_deno |
9601073.410958136 ns/iter |
8670934.851850541 ns/iter |
1.11 |
E2E_Compiler_dependabot |
5224720.365672239 ns/iter |
4772831.027210697 ns/iter |
1.09 |
E2E_Compiler_draft_04 |
19283463.833333924 ns/iter |
17861784.871794764 ns/iter |
1.08 |
E2E_Compiler_fabric_mod |
6839877.330098584 ns/iter |
6339742.572727532 ns/iter |
1.08 |
E2E_Compiler_geojson |
33134561.714286108 ns/iter |
30135105.478260137 ns/iter |
1.10 |
E2E_Compiler_gitpod_configuration |
7089797.161616117 ns/iter |
6553817.813083467 ns/iter |
1.08 |
E2E_Compiler_helm_chart_lock |
970168.5444443658 ns/iter |
918739.4133858816 ns/iter |
1.06 |
E2E_Compiler_importmap |
486134.9916954071 ns/iter |
467314.3102068167 ns/iter |
1.04 |
E2E_Compiler_jasmine |
1963470.1887324054 ns/iter |
1847077.9021163734 ns/iter |
1.06 |
E2E_Compiler_jshintrc |
5187234.777777459 ns/iter |
4720951.256756107 ns/iter |
1.10 |
E2E_Compiler_jsconfig |
26511208.23076361 ns/iter |
23370998.56666782 ns/iter |
1.13 |
E2E_Compiler_krakend |
267416385.33333874 ns/iter |
246591353.33331504 ns/iter |
1.08 |
E2E_Compiler_lazygit |
44879315.31249956 ns/iter |
39560992.222220644 ns/iter |
1.13 |
E2E_Compiler_lerna |
2166080.9287923174 ns/iter |
2024226.9190751782 ns/iter |
1.07 |
E2E_Compiler_nest_cli |
11479755.540982356 ns/iter |
10578479.54545347 ns/iter |
1.09 |
E2E_Compiler_omc |
1779278759.9998975 ns/iter |
1502366918.9999964 ns/iter |
1.18 |
E2E_Compiler_omnisharp |
8763502.70000046 ns/iter |
8027372.908046167 ns/iter |
1.09 |
E2E_Compiler_openapi |
1255847141.9999933 ns/iter |
1161276121.0000145 ns/iter |
1.08 |
E2E_Compiler_pre_commit_hooks |
5718732.170733095 ns/iter |
5201521.814814846 ns/iter |
1.10 |
E2E_Compiler_pulumi |
5491156.843749679 ns/iter |
5083896.413044135 ns/iter |
1.08 |
E2E_Compiler_semantic_release |
2066613.8495570654 ns/iter |
1945032.43333328 ns/iter |
1.06 |
E2E_Compiler_stale |
2388511.287671469 ns/iter |
2199065.5754715386 ns/iter |
1.09 |
E2E_Compiler_stylecop |
7360067.17708193 ns/iter |
6655958.390475851 ns/iter |
1.11 |
E2E_Compiler_tmuxinator |
2643797.063909115 ns/iter |
2461122.6807018397 ns/iter |
1.07 |
E2E_Compiler_ui5 |
167407025.2500428 ns/iter |
151803018.39998263 ns/iter |
1.10 |
E2E_Compiler_ui5_manifest |
2269208583.999898 ns/iter |
1940783323.9999945 ns/iter |
1.17 |
E2E_Compiler_unreal_engine_uproject |
5857615.008475307 ns/iter |
5433608.2890626965 ns/iter |
1.08 |
E2E_Compiler_vercel |
18326684.42105052 ns/iter |
17021446.121952422 ns/iter |
1.08 |
E2E_Compiler_yamllint |
495362.8528996302 ns/iter |
470915.13758388045 ns/iter |
1.05 |
E2E_Evaluator_adaptivecard |
57282.36255943577 ns/iter |
49990.59062296029 ns/iter |
1.15 |
E2E_Evaluator_ansible_meta |
301340.14316329267 ns/iter |
263657.7722846391 ns/iter |
1.14 |
E2E_Evaluator_aws_cdk |
57758.14215209138 ns/iter |
48211.5197724639 ns/iter |
1.20 |
E2E_Evaluator_babelrc |
137747.11510650246 ns/iter |
121913.30194917254 ns/iter |
1.13 |
E2E_Evaluator_clang_format |
106700.01934720257 ns/iter |
100015.28186663061 ns/iter |
1.07 |
E2E_Evaluator_cmake_presets |
5473398.007751705 ns/iter |
5515336.252032528 ns/iter |
0.99 |
E2E_Evaluator_code_climate |
226161.88013030106 ns/iter |
233537.4609739648 ns/iter |
0.97 |
E2E_Evaluator_cql2 |
269173.05618823325 ns/iter |
230406.30695042806 ns/iter |
1.17 |
E2E_Evaluator_cspell |
583155.5311205009 ns/iter |
588862.5964315247 ns/iter |
0.99 |
E2E_Evaluator_cypress |
166045.56136966683 ns/iter |
149664.50776867 ns/iter |
1.11 |
E2E_Evaluator_deno |
347471.9621404078 ns/iter |
373385.47900050005 ns/iter |
0.93 |
E2E_Evaluator_dependabot |
483545.9680554285 ns/iter |
478640.574149623 ns/iter |
1.01 |
E2E_Evaluator_draft_04 |
8022158.24675144 ns/iter |
8170628.602409903 ns/iter |
0.98 |
E2E_Evaluator_fabric_mod |
924659.9496020826 ns/iter |
903238.7643977776 ns/iter |
1.02 |
E2E_Evaluator_geojson |
26068057.407411724 ns/iter |
25852771.222221065 ns/iter |
1.01 |
E2E_Evaluator_gitpod_configuration |
285546.0881147844 ns/iter |
279708.9250604368 ns/iter |
1.02 |
E2E_Evaluator_helm_chart_lock |
366632.602895542 ns/iter |
414270.5059951513 ns/iter |
0.89 |
E2E_Evaluator_importmap |
60961.615703535615 ns/iter |
42352.91472774583 ns/iter |
1.44 |
E2E_Evaluator_jasmine |
142602.8046000421 ns/iter |
133497.79291969744 ns/iter |
1.07 |
E2E_Evaluator_jshintrc |
949866.8099731511 ns/iter |
1019865.0174927592 ns/iter |
0.93 |
E2E_Evaluator_jsconfig |
454496.68278530164 ns/iter |
414359.2926253931 ns/iter |
1.10 |
E2E_Evaluator_krakend |
229457.57004832185 ns/iter |
194208.46221976247 ns/iter |
1.18 |
E2E_Evaluator_lazygit |
163502.20013916216 ns/iter |
136836.99195291384 ns/iter |
1.19 |
E2E_Evaluator_lerna |
144460.33677119124 ns/iter |
165959.42333880666 ns/iter |
0.87 |
E2E_Evaluator_nest_cli |
229157.88721316768 ns/iter |
228302.39754761607 ns/iter |
1.00 |
E2E_Evaluator_omc |
29157.83296148088 ns/iter |
26349.5111304634 ns/iter |
1.11 |
E2E_Evaluator_omnisharp |
539155.960949447 ns/iter |
572199.6342668954 ns/iter |
0.94 |
E2E_Evaluator_openapi |
13176924.574470328 ns/iter |
12165705.053569857 ns/iter |
1.08 |
E2E_Evaluator_pre_commit_hooks |
538401.8424289048 ns/iter |
565937.1281224756 ns/iter |
0.95 |
E2E_Evaluator_pulumi |
830901.8112307524 ns/iter |
893121.7583119429 ns/iter |
0.93 |
E2E_Evaluator_semantic_release |
139232.1260338724 ns/iter |
118043.65840406546 ns/iter |
1.18 |
E2E_Evaluator_stale |
173241.21270310594 ns/iter |
168336.50156439736 ns/iter |
1.03 |
E2E_Evaluator_stylecop |
341658.3503248335 ns/iter |
329569.269248949 ns/iter |
1.04 |
E2E_Evaluator_tmuxinator |
127719.27583078491 ns/iter |
113196.93014230812 ns/iter |
1.13 |
E2E_Evaluator_ui5 |
661335.8966824394 ns/iter |
600694.3422876145 ns/iter |
1.10 |
E2E_Evaluator_ui5_manifest |
3033982.775330731 ns/iter |
3228724.5990779237 ns/iter |
0.94 |
E2E_Evaluator_unreal_engine_uproject |
504188.6322674423 ns/iter |
501795.99123446154 ns/iter |
1.00 |
E2E_Evaluator_vercel |
345058.8870335022 ns/iter |
375803.42895298026 ns/iter |
0.92 |
E2E_Evaluator_yamllint |
14234.881866370628 ns/iter |
12652.41586324242 ns/iter |
1.13 |
This comment was automatically generated by workflow using github-action-benchmark.
jviotti
force-pushed
the
test-module
branch
2 times, most recently
from
c4b7fb9 to
d1f93ed
Compare
jviotti
changed the title
Signed-off-by: Juan Cruz Viotti <[email protected]>
🤖 Augment PR SummarySummary: This PR introduces a new Blaze "test" module that can parse and execute JSON Schema test suites. Changes:
Notes: Test suites compile both fast and exhaustive templates, and the runner currently validates using the fast template during execution. 🤖 Was this summary useful? React with 👍 or 👎 |
![augmentcode[bot]](https://avatars.githubusercontent.com/in/1027498?s=60&v=4){kind=small}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Juan Cruz Viotti [email protected]