The original false positive was caused by the protocol NestedSequence in matplotlib
defined as:
class NestedSequence(Protocol[_T_co]):
def __getitem__(self, key: int, /) -> _T_co | NestedSequence[_T_co]: ...
def __len__(self, /) -> int: ...
which was used to annotate parameters of matplotlib.pyplot.plot and was considered
incompatible with the builtin list.
Skipping positional- and keyword-only parameter separators of the expected callable
is a workaround until we have a comprehensive mechanism for matching signatures.
GitOrigin-RevId: 93d8bb4c6c4405d0e24b5f98152a461691f6197e
- Drop support for python 3.5 & 3.6 in compatibility inspection
- Fix and remove some outdated tests
- Remove xmls for long-unsupported python 2.6 & 3.5
- Regenerate versions.xml
- Remove mentions of OS-specific modules
GitOrigin-RevId: 3265dd1de8a4f7a41119e10c95bb705ca5845efe
Inspection covers such cases:
* Extending typing.Generic in new-style generic classes
* Extending parameterized typing.Protocol in new-style generic classes
* Using generic upper bounds and constraints with type parameters for ParamSpec and TypeVarTuple
* Mixing traditional and new-style type variables
* Using traditional type variables in new-style type aliases
GitOrigin-RevId: 8812959f64d2d87e1b72f713405edb86936220b9
It reports such cases:
* Duplicated type parameter names in type parameter lists
* Wrong number of type var constraints (one or zero) defined with new-style PEP 695 syntax
* Use of constraints for ParamSpec and TypeVarTuple type parameter kinds
GitOrigin-RevId: e0e8e7eb4dcef0c1b56ea49a3527666e3c713d86
Inherit PyTypeAliasStatement from PyQualifiedNameOwner to re-use type aliases stack in PyTypingTypeProvider
Various tests for the changes above
Co-authored-by: Mikhail Golubev <mikhail.golubev@jetbrains.com>
GitOrigin-RevId: 242427c6f84c05ec48c94085f20675b8e30f8625
I prefer the notations used for defining a type and rendering it in the IDE not to diverge,
unless there is a strong incentive to do that. Doing that for Callables is already forcing
us to use two different type renderers for Quick Documentation and generating type hints.
GitOrigin-RevId: f6340182bad60d6a7e94156aadc0f17440ee59d5
The introduction of TypeVarTuples and the concept of unpacked tuple types made us
revise all the places where we match sequences of types in type inference.
For instance, when matching type parameters and type arguments for generic
specialization in:
* type hints, i.e. xs: MyGeneric[int, str] = MyGeneric()
* constructor invocations, i.e. xs = MyGeneric[int, str]()
* class declarations, i.e. class MyGeneric(Base[T1, T2, str]): ...
* type alias declarations, i.e. MyAlias: TypeAlias = MyGeneric[T, int]
as well as during type matching of all generic types, both normal non-variadic and
existing "built-in" generic variadics in the type system, namely tuples and
Callables.
Previously, this logic was spread across numerous places in PyTypeChecker and
PyTypingTypeProvider, all with their own subtle differences. The first attempt
of PEP 646 support put all the code for uniform matching of type parameters directly
in PyTypeChecker, significantly complicating its already arcane internals.
I've introduced a unified API for that called PyTypeParameterMapping.
It still retains some of the former quirks in form of its Option flags, controlling
in particular how we handle having some of the expected types unmatched
(imagine expecting MyGeneric[T1, T2, *Ts] and receiving MyGeneric[int]),
but I'm planning to gradually eliminate this conditional logic.
The same class is now also responsible for matching parameter types of callables
that already allowed to fix some of the known problems, such as ignoring their
arity (PY-16994), but I'm going to extract a separate API entity for that, since
matching of callable signatures is a much more complicated task involving
compatibility of different types of parameters (positional-only, keyword-only,
defaults, varargs, etc.).
Another positive side effect of these changes is that substitution of type
parameters during type inference became more consistent, and we no longer lose
useful type information by replacing all unbound type parameters with Any. It's
particularly visible in type checker errors where we stopped dropping unbound type
parameters from messages about mismatched parameter-argument types.
Among other improvements in this changeset are proper scoping for
TypeVarTuples, consistent with other type parameters, and recognizing TypeVarTuples
and unpacked tuples in types of *args parameters in function bodies, e.g.
`*args: *Ts` translates to "args" parameter having the type `tuple[*Ts]`.
Confusing PyNoMatchedType used only for reporting of missing arguments for *args
parameters annotated with unpacked tuples in the type checker inspection, e.g.
def f(*args: *tuple[int, str]): ...
f(42) # a type checker error about a missing argument for str
was also removed from the type system in favor of a simpler approach with handling
such errors directly in the inspection. We might need such a general type in
the future, but it has to be well thought-through.
GitOrigin-RevId: 63db6202254205863657f014632d141d340fe147
This way they have access to type parameters of these functions, and PyReferenceImpl.getResultsFromProcessor
can properly perform flow-sensitive resolve for them now that READ instructions for expressions
inside annotations and WRITE instructions for type parameters are in the same CFG. It allowed to
remove all the special logic for type parameters from PyReferenceImpl as well as from
PyUnboundLocalVariableInspection.
I had to special-case these annotations in PyResolveUtil.scopeCrawlUp, though, to "lift" their
original scope back from what ScopeUtil.getScopeOwner returns, making names defined in a class scope,
e.g., nested classes, visible to them.
GitOrigin-RevId: b2e78211565300dbcd7c2e2b246a7a8e14bb0e8f
Both of these are evaluated in the outer scope, e.g., in a scope of a containing class or a module,
with the corresponding instructions already processed there, and were duplicated in the CFG
of function bodies.
GitOrigin-RevId: 4460cbef6446a311f0ab15ffe3407bfbb73286e2
Previously, we acknowledged them in PythonIndentingProcessor.adjustBraceLevel, inserting
synthetic STATEMENT_BREAK in front of them to stop recovery in the parser, similarly
to how we handle other kinds of incomplete brackets, but the state inside PyLexerFStringHelper
was not reset, so it kept trying to find matching closing brackets, quotes and interpreting
colons as PyTokenTypes.FSTRING_FRAGMENT_FORMAT_START instead of just PyTokenTypes.COLON.
The state in PyLexerFStringHelper and PythonIndentingProcessor became out of sync, which
led to assertion violations.
It's not an optimal solution, since now these tokens are listed both in
PythonTokenSetContributor.getUnbalancedBracesRecoveryTokens and in Python.flex lexer
specification, and we need to keep them in sync. Also, PythonTokenSetContributor
can provide additional tokens from other languages, such as Cython. But it's simple
and seems "good enough" to patch the problem in the release.
GitOrigin-RevId: 4e156314cc02aba0634d5d9e3008177f49105051
PEP 498 required f-strings to be recognizable by existing tooling, such as syntax highlighters,
by prohibiting re-using quotes of the same kind and having line breaks inside expression fragments.
We used to detect these problems already at the lexer level, correctly replacing violating quotes
with FSTRING_END token, and appending STATEMENT_BREAK tokens to illegal line breaks inside expressions,
depending on the lexer's state. Now, thanks to a general f-string grammar in PEP 701, most of this
bookkeeping could be moved from the lexer to the CompatibilityVisitor (to still be reported
for previous versions of the language and by the compatibility inspection).
Previously forbidden backslashes and line comments are now also detected by the CompatibilityVisitor
instead of the version-agnostic FStringAnnotator.
One side effect of the new grammar is that parser recovery in pre-3.12 version of Python became
slightly worse. For instance, something like `f'{foo'` used to be recognized as an f-string
with an incomplete fragment lacking its closing brace. Now, it's parsed as an incomplete
f-string, lacking its own closing quote, containing an incomplete string literal inside
an incomplete fragment. What's more, parsing of this fragment's expression doesn't terminate
until the end of a file, because STATEMENT_BREAK is never produced by PythonIndentingProcessor
while it's inside an f-string fragment, and every quote is considered a new string literal.
Examples of parsing tests affected by this are:
PythonParsingTest.testFStringFragmentIncompleteTypeConversionBeforeClosingQuote
PythonParsingTest.testFStringIncompleteFragmentWithTypeConversion
PythonParsingTest.testFStringIncompleteFragment
I also had to simplify some scenarios from PythonHighlightingTest, removing snippets
with incomplete fragments or moving such examples to the very end of a file.
It's not clear how to handle these situations not overcomplicating the lexer.
(cherry picked from commit 03ba6d7fba1b45a84aa92221e6a452645a765205)
IJ-MR-115763
GitOrigin-RevId: cd36470d9cae353fe3caeb2d3b628d8743b46cbb
