`dataclass_transform` support posed a number of challenges to the current
AST/PSI stubs separation in our architecture. For the standard "dataclasses"
module and the "attrs" package API, we could rely on well-known names
and defaults to recognize and preserve the information about decorator
arguments and field specifier arguments in PSI stubs. With `dataclass_transform`
however, effectively any decorator call or extending any base class can indicate
using a "magical" API that should generate dataclass-like entities.
At the moment of building PSI stubs we can't be sure, because we can't leave
the boundaries of the current file to resolve these names and determine if these
decorators and base classes are decorated themselves with `dataclass_transform`.
To support that, we instead rely on well-known keyword argument names documented
in the spec, e.g. "kw_only" or "frozen". In other words, whenever we encounter
any decorator call or a superclass list with such keyword arguments, we generate
a `PyDataclassStub` stub for the corresponding class.
The same thing is happening with class attribute initializers, i.e. whenever
we see a function call with argument such as "default" or "kw_only" in their
RHS, we generate a `PyDataclassFieldStub` for the corresponding target expression.
Both of these stub interfaces now can contain null values for the corresponding
properties if they were not specified directly in the class definition.
Finally, for the `dataclass_transform` decorator itself, a new custom decorator stub
was introduced -- `PyDataclassTransformDecoratorStub`, it preserves its keyword
arguments, such as "keyword_only_default" or "frozen_default" controlling
the default properties of generated dataclasses.
Later, when we need concluded information about specific dataclass properties,
e.g. in `PyDataclassTypeProvider` to generate a constructor signature, or in
`PyDataclassInspection`, we try to "resolve" this incomplete information from stubs
into finalized `PyDataclassParameters` and `PyDataclassFieldParameters` that
contain non-null versions of the same fields. The main entry points for that
are `resolveDataclassParameters` and `resolveDataclassFieldParameters`.
These methods additionally handle the situations where decorators, superclass
lists and field specifiers lack any keyword arguments, and thus, there were no
automatically created custom stubs for them.
All the existing usages of `PyDataclassStub` and `PyDataclassFieldStub`
were updated to operate on `PyDataclassParameters` and `PyDataclassFieldParameters`
instead.
Counterparts of the tests on various inspection checks for the standard dataclasses
definitions were added for dataclasses created with `dataclass_transform`, even
though the spec is unclear on some aspects the expected type checker semantics, e.g.
if combining "eq=False" and "order=True" or specifying both "default" and
"default_factory" for a field should be reported.
I tried to follow common sense when enabling existing checks for such arbitrary
user-defined dataclass APIs.
GitOrigin-RevId: 4180a1e32b5e4025fc4e3ed49bb8d67af0d60e66
- PEP-696 adds a new syntax for declaring the default types of Type Parameters in new-new style generic classes, functions and type alias statements. Support these grammar changes.
- Store info about default types in stubs for Type Parameters
- Increment the stub version counter in PyFileElementType
GitOrigin-RevId: b6b22e3eaa86ce06132885781e5775a89bf4b840
The introduction of TypeVarTuples and the concept of unpacked tuple types made us
revise all the places where we match sequences of types in type inference.
For instance, when matching type parameters and type arguments for generic
specialization in:
* type hints, i.e. xs: MyGeneric[int, str] = MyGeneric()
* constructor invocations, i.e. xs = MyGeneric[int, str]()
* class declarations, i.e. class MyGeneric(Base[T1, T2, str]): ...
* type alias declarations, i.e. MyAlias: TypeAlias = MyGeneric[T, int]
as well as during type matching of all generic types, both normal non-variadic and
existing "built-in" generic variadics in the type system, namely tuples and
Callables.
Previously, this logic was spread across numerous places in PyTypeChecker and
PyTypingTypeProvider, all with their own subtle differences. The first attempt
of PEP 646 support put all the code for uniform matching of type parameters directly
in PyTypeChecker, significantly complicating its already arcane internals.
I've introduced a unified API for that called PyTypeParameterMapping.
It still retains some of the former quirks in form of its Option flags, controlling
in particular how we handle having some of the expected types unmatched
(imagine expecting MyGeneric[T1, T2, *Ts] and receiving MyGeneric[int]),
but I'm planning to gradually eliminate this conditional logic.
The same class is now also responsible for matching parameter types of callables
that already allowed to fix some of the known problems, such as ignoring their
arity (PY-16994), but I'm going to extract a separate API entity for that, since
matching of callable signatures is a much more complicated task involving
compatibility of different types of parameters (positional-only, keyword-only,
defaults, varargs, etc.).
Another positive side effect of these changes is that substitution of type
parameters during type inference became more consistent, and we no longer lose
useful type information by replacing all unbound type parameters with Any. It's
particularly visible in type checker errors where we stopped dropping unbound type
parameters from messages about mismatched parameter-argument types.
Among other improvements in this changeset are proper scoping for
TypeVarTuples, consistent with other type parameters, and recognizing TypeVarTuples
and unpacked tuples in types of *args parameters in function bodies, e.g.
`*args: *Ts` translates to "args" parameter having the type `tuple[*Ts]`.
Confusing PyNoMatchedType used only for reporting of missing arguments for *args
parameters annotated with unpacked tuples in the type checker inspection, e.g.
def f(*args: *tuple[int, str]): ...
f(42) # a type checker error about a missing argument for str
was also removed from the type system in favor of a simpler approach with handling
such errors directly in the inspection. We might need such a general type in
the future, but it has to be well thought-through.
GitOrigin-RevId: 63db6202254205863657f014632d141d340fe147
Both typing.TypeAlias (available only in 3.10) and typing_extensions.TypeAlias
names are supported.
RHS values of assignments annotated with TypeAlias are always retained in
stubs and injected into (if it's a string literal), regardless of whether
they look similar to a well-formed type hint. It seems natural to assume
that if a user employs such as specific marker as "TypeAlias" at all, they
clearly indicate that the value is supposed to be a type.
The inspections "Type hints definitions and usages" and "Final classes, methods
and variables" properly analyse RHS of assignments annotated with TypeAlias.
Type hinting inspection also reports illegal usages of TypeAlias, as it was
done for other special forms in the typing module.
The type of such variables themselves is Any, however they're still displayed
as having the type "TypeAlias" in Quick Documentation to avoid confusion.
GitOrigin-RevId: fab02f6e1060c0994e1d21201768e7b28ba7d9e0
- Added PyDecorator.getExpression() to reflect that in Python 3.9
an arbitrary expression can be used after "@". Reused it where possible
in the implementation.
- Removed hasPlainReferenceCallee() in favor of existing getQualifiedName().
Updated javadoc for the latter to clarify how it handles non-trivial callees.
- Added PSI stub tests to make sure that right values are still persisted
with the new flexible decorator grammar.
GitOrigin-RevId: 8109b834c8d257c72f6f6e3c0ce5005060a2b971
PY-38422 Unify warning message for 'clear' and 'popitem' TypedDict methods
PY-38505 Infer proper type for TypedDict subscription expressions
PY-38439 Fix TypedDict consistency check
PY-38415 Fix TypedDict keys order in parameter info
PY-38413 Add * before all arguments in TypedDict parameter hint
PY-38873 Fix detection of TypedDict subscription expression type when value is a dict, Dict or TypedDict
GitOrigin-RevId: 25ff441042f2e2d7791e28632a5016fb367685df
by reusing the logic from "Convert to variable annotation" intention
This change also partially remedies the problem described in PY-28879
when the whole file was parsed whenever we found typing.Tuple in a
variable type comment, because we no longer transform type hints like
"int, str" into "Tuple[int, str]" as an intermediate step and process
them right away instead. We still need to handle general type comments
with unpacking in stub-safe manner, though.
Update PyDataclassesTypeProvider to ignore such fields or correctly specify default value for them.
Create custom target stubs for such fields to store parameters.
Infer `tuple` class type when `collections.namedtuple` could not be analyzed.
Infer `namedtuple` class type when `typing.NamedTuple` could not be analyzed.
Create callable type for `typing.Callable`.
Update ignoring `__getitem__` for ancestors, docstrings and annotations.
Infer superclass collection type correctly.
Namely, retain them only for function return type annotations and
named parameters, since for these two PSI nodes we use
getStubOrPsiChild() in getAnnotation() as a way to quickly check
whether there is annotation at all. Otherwise, stubs for annotations
of local variables might be stored directly in a function stub and
thus falsely recognized as a type hint for its return value.
Since, otherwise, these custom stubs conflict with the standard
"initializer" field of PyTargetExpressionStub and, thus, break
resolve in stubbed files, e.g. when exported symbols are aliased in
"__init__.py" of a package, etc. The current workaround is not to
keep RHS text of such assignments in the custom stubs, relying
on existing functionality of PyTypingAliasStubType instead, but inspect
both when extracting type aliases from the stub tree.
