Unify stubs for all Python language levels (PY-26392)

The cause of the problem was the difference in parsing for different
language levels.

This commit improves error recovery and adds support for stubs for
syntax elements unavailable in older Python versions.
Cases that are covered are necessary for stubs generation
for Python 3.6 standard library. They are discovered by automatic
validation executed by stubs generator in StubsGenerator.kt.

Those are the cases that were fixed:
* A stub for single star argument in Py2 was absent
* Type annotations in Py2 were absent
* Print with end argument in Py2 broke parser
* Exec function as argument in Py2 broke parser
* Async keyword after decorator broke parser

All the cases are listed as test cases in PyUnifiedStubsTest

P.S. It is possible though that more cases will be revealed in future.
This commit is contained in:
Dmitry Trofimov
2017-10-09 21:23:29 +02:00
parent 6fed513e8d
commit 210bf9b175
15 changed files with 240 additions and 52 deletions
@@ -0,0 +1,4 @@
class AsyncIterator(AsyncIterable):
@abstractmethod
async def __anext__(self):
raise StopAsyncIteration
+3 -6
View File
@@ -30,13 +30,10 @@ PyFile:AsyncFor.py
PyPassStatement
PsiElement(Py:PASS_KEYWORD)('pass')
PsiWhiteSpace('\n\n\n')
PyExpressionStatement
PyReferenceExpression: async
PsiElement(Py:IDENTIFIER)('async')
PsiErrorElement:End of statement expected
<empty list>
PsiWhiteSpace(' ')
PyForStatement
PsiErrorElement:'async' keyword is not expected here
PsiElement(Py:IDENTIFIER)('async')
PsiWhiteSpace(' ')
PyForPart
PsiElement(Py:FOR_KEYWORD)('for')
PsiWhiteSpace(' ')
+3 -6
View File
@@ -25,13 +25,10 @@ PyFile:AsyncWith.py
PyPassStatement
PsiElement(Py:PASS_KEYWORD)('pass')
PsiWhiteSpace('\n\n')
PyExpressionStatement
PyReferenceExpression: async
PsiElement(Py:IDENTIFIER)('async')
PsiErrorElement:End of statement expected
<empty list>
PsiWhiteSpace(' ')
PyWithStatement
PsiErrorElement:'async' keyword is not expected here
PsiElement(Py:IDENTIFIER)('async')
PsiWhiteSpace(' ')
PsiElement(Py:WITH_KEYWORD)('with')
PsiWhiteSpace(' ')
PyWithItem
+6
View File
@@ -0,0 +1,6 @@
def foo():
baz(exec, 1)
def bar():
pass
@@ -0,0 +1,6 @@
def bar():
print("header:", end=" ")
def foo():
pass
+6
View File
@@ -0,0 +1,6 @@
def foo(arg1, *, kwarg1):
pass
def bar():
pass