Skip to content

Commit cc96f6e

Browse files
BHUVANSH855BHUVANSH855
authored
Merge branch 'main' into update-generics-pep695
2 parents 0e9a0fb + c0cc0ab commit cc96f6e

173 files changed

Lines changed: 2450 additions & 172 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎conformance/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ Note that some type checkers may not run on some platforms. If a type checker fa
8181

8282
Different type checkers report errors in different ways (with different wording in error messages and different line numbers or character ranges for errors). This variation makes it difficult to fully automate test validation given that tests will want to check for both false positive and false negative type errors. Some level of manual inspection will therefore be needed to determine whether a type checker is fully conformant with all tests in any given test file. This "scoring" process is required only when the output of a test changes — e.g. when a new version of that type checker is released and the tests are rerun. We assume that the output of a type checker will be the same from one run to the next unless/until a new version is released that fixes or introduces a bug. In this case, the output will need to be manually inspected and the conformance results re-scored for those tests whose output has changed.
8383

84-
[Conformance results](https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html) are reported and summarized for each supported type checker. Currently, results are reported for mypy, pyrefly, pyright, zuban and ty. It is the goal and desire to add additional type checkers over time.
84+
[Conformance results](https://htmlpreview.github.io/?https://github.com/python/typing/blob/main/conformance/results/results.html) are reported and summarized for each supported type checker. Currently, results are reported for mypy, pyrefly, pyright, zuban, ty, and pycroscope. It is the goal and desire to add additional type checkers over time.
8585

8686
## Adding a New Test Case
8787

‎conformance/pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ version = "0.1.0"
44
requires-python = "==3.12.*"
55
dependencies = [
66
"mypy",
7+
"pycroscope",
78
"pyrefly",
89
"pyright",
910
"tomli",

‎conformance/results/pycroscope/aliases_explicit.toml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
conformance_automated = "Pass"
2+
errors_diff = """
3+
"""
4+
output = """
5+
./aliases_explicit.py:67:8: Expected 0 type arguments for type alias, got 1 [invalid_specialization]
6+
./aliases_explicit.py:68:8: Expected 0 type arguments for type alias, got 1 [invalid_specialization]
7+
./aliases_explicit.py:69:8: Expected 1 type arguments for type alias, got 2 [invalid_specialization]
8+
./aliases_explicit.py:70:8: Expected 1 type arguments for type alias, got 2 [invalid_specialization]
9+
./aliases_explicit.py:71:8: ParamSpec specialization must use list form, Concatenate[..., P], P, or ... [invalid_annotation]
10+
./aliases_explicit.py:79:20: Invalid type annotation [invalid_annotation]
11+
./aliases_explicit.py:80:20: Unrecognized annotation <list containing [type 'int', type 'str']> [invalid_annotation]
12+
./aliases_explicit.py:81:20: Unrecognized annotation tuple[tuple[type 'int', type 'str']] [invalid_annotation]
13+
./aliases_explicit.py:82:20: Invalid type annotation [invalid_annotation]
14+
./aliases_explicit.py:83:20: Unrecognized annotation <dict containing {Literal['a']: Literal['b']}> [invalid_annotation]
15+
./aliases_explicit.py:84:20: Invalid type annotation [invalid_annotation]
16+
./aliases_explicit.py:85:20: Cannot resolve subscripted annotation: <list containing [type 'int']> [invalid_annotation]
17+
./aliases_explicit.py:86:20: Invalid type annotation [invalid_annotation]
18+
./aliases_explicit.py:87:20: Invalid type annotation 3 [invalid_annotation]
19+
./aliases_explicit.py:88:21: Invalid type annotation True [invalid_annotation]
20+
./aliases_explicit.py:89:21: Invalid type annotation 1 [invalid_annotation]
21+
./aliases_explicit.py:90:21: Invalid type annotation [invalid_annotation]
22+
./aliases_explicit.py:91:21: Invalid type annotation [invalid_annotation]
23+
./aliases_explicit.py:100:4: Expected 0 type arguments for type alias, got 1 [invalid_specialization]
24+
./aliases_explicit.py:101:5: TypeForm[.ListOrSetAlias] (synthetic from Literal[list | set]) is not callable [not_callable]
25+
./aliases_explicit.py:102:4: Expected 0 type arguments for type alias, got 1 [invalid_specialization]
26+
"""

‎conformance/results/pycroscope/aliases_implicit.toml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
conformant = "Partial"
2+
notes = """
3+
Fails to handle various weird annotations.
4+
Various bugs with resolving generic aliases.
5+
"""
6+
conformance_automated = "Fail"
7+
errors_diff = """
8+
Line 76: Expected 1 errors
9+
Line 106: Expected 1 errors
10+
Line 111: Expected 1 errors
11+
Line 112: Expected 1 errors
12+
Line 113: Expected 1 errors
13+
Line 117: Expected 1 errors
14+
Line 118: Expected 1 errors
15+
Line 119: Expected 1 errors
16+
Line 68: Unexpected errors ['./aliases_implicit.py:68:16: .GoodTypeAlias9[<list containing [str, str]>, None] = (int, /, ****P) -> ~R is not equivalent to (int, str, str, /) -> None']
17+
Line 70: Unexpected errors ['./aliases_implicit.py:70:16: Any[from_another] is not equivalent to int | str | list[list[int]] | None']
18+
Line 72: Unexpected errors ['./aliases_implicit.py:72:16: .GoodTypeAlias13 = (**__P: **P) -> None is not equivalent to (...) -> None']
19+
Line 95: Unexpected errors ["./aliases_implicit.py:95:16: Cannot resolve subscripted annotation: <list containing [type 'int']> [invalid_annotation]"]
20+
Line 100: Unexpected errors ["./aliases_implicit.py:100:17: Annotated[type 'list', (<+list is_truthy None> == NullConstraint())] is always True because it does not provide __bool__ [type_always_true]"]
21+
"""
22+
output = """
23+
./aliases_implicit.py:68:16: .GoodTypeAlias9[<list containing [str, str]>, None] = (int, /, ****P) -> ~R is not equivalent to (int, str, str, /) -> None
24+
./aliases_implicit.py:70:16: Any[from_another] is not equivalent to int | str | list[list[int]] | None
25+
./aliases_implicit.py:72:16: .GoodTypeAlias13 = (**__P: **P) -> None is not equivalent to (...) -> None
26+
./aliases_implicit.py:77:8: Unrecognized annotation types.GenericAlias [invalid_annotation]
27+
./aliases_implicit.py:78:8: Expected 1 type arguments for type alias, got 2 [invalid_specialization]
28+
./aliases_implicit.py:79:8: Expected 1 type arguments for type alias, got 2 [invalid_specialization]
29+
./aliases_implicit.py:80:8: ParamSpec specialization must use list form, Concatenate[..., P], P, or ... [invalid_annotation]
30+
./aliases_implicit.py:81:8: Type argument str is not compatible with ~TFloat@./aliases_implicit.py.GoodTypeAlias12 [invalid_specialization]
31+
./aliases_implicit.py:95:16: Cannot resolve subscripted annotation: <list containing [type 'int']> [invalid_annotation]
32+
./aliases_implicit.py:100:17: Annotated[type 'list', (<+list is_truthy None> == NullConstraint())] is always True because it does not provide __bool__ [type_always_true]
33+
./aliases_implicit.py:107:8: Invalid type annotation [<class 'int'>, <class 'str'>] [invalid_annotation]
34+
./aliases_implicit.py:108:8: Invalid type annotation ((<class 'int'>, <class 'str'>),) [invalid_annotation]
35+
./aliases_implicit.py:109:8: Unrecognized annotation <list containing [type 'int']> [invalid_annotation]
36+
./aliases_implicit.py:110:8: Invalid type annotation {'a': 'b'} [invalid_annotation]
37+
./aliases_implicit.py:114:8: Invalid type annotation 3 [invalid_annotation]
38+
./aliases_implicit.py:115:9: Invalid type annotation True [invalid_annotation]
39+
./aliases_implicit.py:116:9: Invalid type annotation 1 [invalid_annotation]
40+
./aliases_implicit.py:133:5: Literal[list | set] is not callable [not_callable]
41+
./aliases_implicit.py:135:4: Unrecognized annotation object [invalid_annotation]
42+
"""

‎conformance/results/pycroscope/aliases_newtype.toml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
conformance_automated = "Pass"
2+
errors_diff = """
3+
"""
4+
output = """
5+
./aliases_newtype.py:11:7: Incompatible argument type for x: expected int but got Literal['user'] [incompatible_argument]
6+
./aliases_newtype.py:12:0: Incompatible assignment: expected NewType('UserId', int), got Literal[42] [incompatible_assignment]
7+
./aliases_newtype.py:18:0: Incompatible assignment: expected type, got Literal[pycroscope.implementation.UserId] [incompatible_assignment]
8+
./aliases_newtype.py:23:15: Second argument to "isinstance" must be a type, union, or tuple of types, not "pycroscope.implementation.UserId" [incompatible_argument]
9+
./aliases_newtype.py:26:20: NewType cannot be used as a base class [invalid_base]
10+
./aliases_newtype.py:35:19: NewType named 'BadName' must be assigned to a variable with the same name [must_have_same_name]
11+
./aliases_newtype.py:41:5: Object Literal[pycroscope.implementation.GoodNewType1] does not support subscripting [unsupported_operation]
12+
./aliases_newtype.py:47:37: NewType base type cannot be generic [incompatible_call]
13+
./aliases_newtype.py:50:37: NewType base type cannot be generic [incompatible_call]
14+
./aliases_newtype.py:52:37: NewType base type cannot be a protocol [incompatible_call]
15+
./aliases_newtype.py:54:37: NewType base type cannot be a literal type [incompatible_call]
16+
./aliases_newtype.py:61:37: NewType base type cannot be a TypedDict [incompatible_call]
17+
./aliases_newtype.py:63:14: In call to typing.NewType: Takes 2 positional arguments but 3 were given [incompatible_call]
18+
./aliases_newtype.py:65:37: NewType base type cannot be Any [incompatible_call]
19+
"""

‎conformance/results/pycroscope/aliases_recursive.toml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
conformance_automated = "Pass"
2+
errors_diff = """
3+
"""
4+
output = """
5+
./aliases_recursive.py:19:0: Incompatible assignment: expected int | str | float | list[int | str | float | list[<unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | dict[str, <unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | None] | dict[str, int | str | float | list[<unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | dict[str, <unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | None] | None, got Literal[{'a': 1, 'b': 3j}] [incompatible_assignment]
6+
./aliases_recursive.py:20:0: Incompatible assignment: expected int | str | float | list[int | str | float | list[<unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | dict[str, <unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | None] | dict[str, int | str | float | list[<unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | dict[str, <unknown>.Json = int | str | float | list[int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | dict[str, int | str | float | list[<unknown>.Json] | dict[str, <unknown>.Json] | None] | None] | None] | None, got Literal[[2, 3j]] [incompatible_assignment]
7+
./aliases_recursive.py:38:0: Incompatible assignment: expected str | int | tuple[str | int | tuple[<unknown>.RecursiveTuple = str | int | tuple[str | int | tuple[<unknown>.RecursiveTuple, ...], ...], ...], ...], got Literal[(1, ('1', 1), (1, (1, [2])))] [incompatible_assignment]
8+
./aliases_recursive.py:39:0: Incompatible assignment: expected str | int | tuple[str | int | tuple[<unknown>.RecursiveTuple = str | int | tuple[str | int | tuple[<unknown>.RecursiveTuple, ...], ...], ...], ...], got Literal[(1, [1])] [incompatible_assignment]
9+
./aliases_recursive.py:50:0: Incompatible assignment: expected str | int | collections.abc.Mapping[str, str | int | collections.abc.Mapping[str, None.RecursiveMapping = str | int | collections.abc.Mapping[str, str | int | collections.abc.Mapping[str, None.RecursiveMapping]]]], got Literal[{'1': [1]}] [incompatible_assignment]
10+
./aliases_recursive.py:51:0: Incompatible assignment: expected str | int | collections.abc.Mapping[str, str | int | collections.abc.Mapping[str, None.RecursiveMapping = str | int | collections.abc.Mapping[str, str | int | collections.abc.Mapping[str, None.RecursiveMapping]]]], got Literal[{'1': '1', '2': 1, '3': [1, 2]}] [incompatible_assignment]
11+
./aliases_recursive.py:52:0: Incompatible assignment: expected str | int | collections.abc.Mapping[str, str | int | collections.abc.Mapping[str, None.RecursiveMapping = str | int | collections.abc.Mapping[str, str | int | collections.abc.Mapping[str, None.RecursiveMapping]]]], got Literal[{'1': '1', '2': 1, '3': {'0': '0', '1': 1, '2': [1, 2, 3]}}] [incompatible_assignment]
12+
./aliases_recursive.py:63:0: Incompatible assignment: expected aliases_recursive.GenericTypeAlias1[str] = list[aliases_recursive.GenericTypeAlias1[~T1] = list[aliases_recursive.GenericTypeAlias1[~T1] | ~T1] | ~T1], got Literal[['hi', [2.4]]] [incompatible_assignment]
13+
./aliases_recursive.py:69:0: Incompatible assignment: expected aliases_recursive.GenericTypeAlias2[str, int] = list[aliases_recursive.GenericTypeAlias2[~T1, ~T2] = list[aliases_recursive.GenericTypeAlias2[~T1, ~T2] | ~T1 | ~T2] | ~T1 | ~T2], got Literal[[[3, ['hi', 3, [3.4]]], 'hi']] [incompatible_assignment]
14+
./aliases_recursive.py:72:0: Type alias RecursiveUnion has a circular definition [invalid_type_alias]
15+
./aliases_recursive.py:75:0: Type alias MutualReference1 has a circular definition [invalid_type_alias]
16+
./aliases_recursive.py:75:62: Type alias MutualReference2 has a circular definition [invalid_type_alias]
17+
"""

‎conformance/results/pycroscope/aliases_type_statement.toml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
conformant = "Partial"
2+
notes = """
3+
Fails to reject various weird annotations.
4+
"""
5+
conformance_automated = "Fail"
6+
errors_diff = """
7+
Line 37: Expected 1 errors
8+
Line 38: Expected 1 errors
9+
Line 39: Expected 1 errors
10+
Line 40: Expected 1 errors
11+
Line 41: Expected 1 errors
12+
Line 42: Expected 1 errors
13+
Line 43: Expected 1 errors
14+
Line 44: Expected 1 errors
15+
Line 45: Expected 1 errors
16+
Line 46: Expected 1 errors
17+
Line 47: Expected 1 errors
18+
Line 49: Expected 1 errors
19+
"""
20+
output = """
21+
./aliases_type_statement.py:17:0: typing.TypeAliasType has no attribute 'bit_count' [undefined_attribute]
22+
./aliases_type_statement.py:19:0: TypeForm[.GoodAlias1] (synthetic from typing.TypeAliasType) is not callable [not_callable]
23+
./aliases_type_statement.py:23:6: typing.TypeAliasType has no attribute 'other_attrib' [undefined_attribute]
24+
./aliases_type_statement.py:26:17: Type alias cannot be used as a base class [invalid_base]
25+
./aliases_type_statement.py:31:21: Second argument to "isinstance" cannot be a type alias [incompatible_argument]
26+
./aliases_type_statement.py:48:22: Annotated[type 'list', (<+list is_truthy None> == NullConstraint())] is always True because it does not provide __bool__ [type_always_true]
27+
./aliases_type_statement.py:53:22: Type alias cannot combine old-style TypeVar declarations with type statement parameters [invalid_type_alias]
28+
./aliases_type_statement.py:58:16: Type alias must declare type parameters in the type statement [invalid_type_alias]
29+
./aliases_type_statement.py:68:6: Type argument str is not compatible with ~S@./aliases_type_statement.py.RecursiveTypeAlias2 [invalid_specialization]
30+
./aliases_type_statement.py:70:6: Type argument int is not compatible with ~T@./aliases_type_statement.py.RecursiveTypeAlias2 [invalid_specialization]
31+
./aliases_type_statement.py:73:0: Type alias RecursiveTypeAlias3 has a circular definition [invalid_type_alias]
32+
./aliases_type_statement.py:75:0: Type alias RecursiveTypeAlias4 has a circular definition [invalid_type_alias]
33+
./aliases_type_statement.py:79:0: Type alias RecursiveTypeAlias6 has a circular definition [invalid_type_alias]
34+
./aliases_type_statement.py:80:0: Type alias RecursiveTypeAlias7 has a circular definition [invalid_type_alias]
35+
"""

‎conformance/results/pycroscope/aliases_typealiastype.toml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
conformant = "Partial"
2+
notes = """
3+
Rejects valid ParamSpec specialization.
4+
"""
5+
conformance_automated = "Fail"
6+
errors_diff = """
7+
Line 39: Unexpected errors ["./aliases_typealiastype.py:39:4: Unrecognized annotation <list containing [type 'int', type 'str']> [invalid_annotation]"]
8+
"""
9+
output = """
10+
./aliases_typealiastype.py:32:6: typing.TypeAliasType has no attribute 'other_attrib' [undefined_attribute]
11+
./aliases_typealiastype.py:39:4: Unrecognized annotation <list containing [type 'int', type 'str']> [invalid_annotation]
12+
./aliases_typealiastype.py:40:4: Type argument int is not compatible with ~TStr@./aliases_typealiastype.py.GoodAlias5 [invalid_specialization]
13+
./aliases_typealiastype.py:43:44: Type alias cannot combine old-style TypeVar declarations with type statement parameters [invalid_type_alias]
14+
./aliases_typealiastype.py:44:44: Type alias must declare type parameters in the type statement [invalid_type_alias]
15+
./aliases_typealiastype.py:45:56: type_params argument to TypeAliasType must be a literal tuple [invalid_type_alias]
16+
./aliases_typealiastype.py:46:0: Type alias BadAlias4 has a circular definition [invalid_type_alias]
17+
./aliases_typealiastype.py:47:0: Type alias BadAlias5 has a circular definition [invalid_type_alias]
18+
./aliases_typealiastype.py:48:0: Type alias BadAlias6 has a circular definition [invalid_type_alias]
19+
./aliases_typealiastype.py:49:0: Type alias BadAlias7 has a circular definition [invalid_type_alias]
20+
./aliases_typealiastype.py:52:39: Invalid type annotation [invalid_annotation]
21+
./aliases_typealiastype.py:53:39: Unrecognized annotation <list containing [type 'int', type 'str']> [invalid_annotation]
22+
./aliases_typealiastype.py:54:41: Unrecognized annotation tuple[tuple[type 'int', type 'str']] [invalid_annotation]
23+
./aliases_typealiastype.py:55:41: Invalid type annotation [invalid_annotation]
24+
./aliases_typealiastype.py:56:41: Unrecognized annotation <dict containing {Literal['a']: Literal['b']}> [invalid_annotation]
25+
./aliases_typealiastype.py:57:41: Invalid type annotation [invalid_annotation]
26+
./aliases_typealiastype.py:58:41: Cannot resolve subscripted annotation: <list containing [type 'int']> [invalid_annotation]
27+
./aliases_typealiastype.py:59:41: Invalid type annotation [invalid_annotation]
28+
./aliases_typealiastype.py:60:41: Invalid type annotation 3 [invalid_annotation]
29+
./aliases_typealiastype.py:61:41: Invalid type annotation True [invalid_annotation]
30+
./aliases_typealiastype.py:62:41: Invalid type annotation 1 [invalid_annotation]
31+
./aliases_typealiastype.py:63:41: Invalid type annotation [invalid_annotation]
32+
./aliases_typealiastype.py:64:41: Invalid type annotation [invalid_annotation]
33+
./aliases_typealiastype.py:66:0: Type alias BadAlias21 has a circular definition [invalid_type_alias]
34+
"""

‎conformance/results/pycroscope/aliases_variance.toml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
conformance_automated = "Pass"
2+
errors_diff = """
3+
"""
4+
output = """
5+
./aliases_variance.py:24:15: T_co has incompatible variance in base class [invalid_base]
6+
./aliases_variance.py:28:15: T_co has incompatible variance in base class [invalid_base]
7+
./aliases_variance.py:32:15: T_co has incompatible variance in base class [invalid_base]
8+
./aliases_variance.py:44:15: T_contra has incompatible variance in base class [invalid_base]
9+
"""

‎conformance/results/pycroscope/annotations_coroutines.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
conformance_automated = "Pass"
2+
errors_diff = """
3+
"""
4+
output = """
5+
"""

0 commit comments

Comments
 (0)