mirror of
https://gitflic.ru/project/openide/openide.git
synced 2026-01-06 03:21:12 +07:00
ML 4 Completion Performance Plugin
Update experiment configuration Add fixed generator execution order Fix some build issues Remove redundant ArtifactGenerator and update related classes Enable performance only on Nightly users Enable performance only on Nightly users Remove unused interfaces and update related classes Fix Kotlin style-related comments Fix references in documentation Allow other contributors to run Improve code organization and add documentation to turboComplete Merge remote-tracking branch 'origin/master' into marin/plugin-ml-4-completion-performace # Conflicts: # community/plugins/completion-ml-ranking/resources/experiment.json Run completion contributors after live templates Remove unnecessary functions from performance parameters Merge branch 'master' into marin/plugin-ml-4-completion-performace Update experiment list for Kotlin Increase experiment version number Clarify usage of local debug model Register A/B experiment Merge branch 'master' into marin/plugin-ml-4-completion-performace # Conflicts: # build/src/org/jetbrains/intellij/build/UltimateLibraryLicenses.kt # community/plugins/completion-ml-ranking/src/com/intellij/completion/ml/performance/MLCompletionPerformanceTracker.kt Remove unnecessary nullability indicators Remove redundant class Make the opener internal Revert change in modules.xml Utilize delegation when necessary Merge branch 'master' into marin/plugin-ml-4-completion-performace # Conflicts: # community/plugins/completion-ml-ranking/src/com/intellij/completion/ml/performance/MLCompletionPerformanceTracker.kt Merge branch 'master' into marin/plugin-ml-4-completion-performace Merge branch 'master' into marin/plugin-ml-4-completion-performace Merge branch 'master' into marin/plugin-ml-4-completion-performace # Conflicts: # community/platform/analysis-api/src/com/intellij/codeInsight/completion/CompletionParameters.java # community/plugins/completion-ml-ranking/src/com/intellij/completion/ml/performance/MLCompletionPerformanceTracker.kt Remove unnecessary symbols from Kotlin's kind generation session Disable performance on TeamCity Fix some style issues Merge branch 'master' into marin/plugin-ml-4-completion-performace Merge branch 'master' into marin/plugin-ml-4-completion-performace Restore EP name Add dependency on completion ml ranking plugin Change scope of kotlin's completion dependency Accord API core package name with the naming convention Accord API core package name with the naming convention Revert non-standard-root-packages.txt to default Remove java performance from community modules Remove duplicating logic Merge branch 'master' into marin/plugin-ml-4-completion-performace Reorganize plugin's structure Merge branch 'master' into marin/plugin-ml-4-completion-performace Fix some comments & build issues Fix some build issues Fix some build issues Fix some build issues Fix some build issues Fix some build issues Fix some build issues Merge remote-tracking branch 'origin/master' into marin/plugin-ml-4-completion-performace Revert unintended change Fix some style issues Rename .java to .kt Merge branch 'master' into marin/plugin-ml-4-completion-performace Add core to main and ultimate iml files Fix test Use model with updated kinds names Avoid unwanted generator execution for artifact generation Fix JavaKindVariety Revert not indented changes Actualize kotlin completion usage Merge branch 'master' into marin/plugin-ml-4-completion-performace Initialize completion kind name with Enum Add core module to CommunityRepositoryModules Update model & remove legacy support Support legacy completion kinds ranking Fix kind features computation for ordering Fix style Reintroduce CompletionKind Fix fast lookup appearance Rename main interfaces Remove plugin's core dependency from completion-ml-ranking Cache completion kind features in lookup Bundle core into community Merge branch 'master' into marin/plugin-ml-4-completion-performace # Conflicts: # intellij.idea.ultimate.main.iml Add todo Add completion kind features Try fixing build schema Try fixing build schema Try fixing build schema Add performance logging Add kotlin module dependency to build Add performance to the community Put early lookup opener to the experiment Add dependency on Kotlin by default Add plugin to the build Split java generator into the most frequent kinds Rename api module to core Add plugin to the build Add performance completion to community Update Kotlin's ML model version Initialize completion performance for Java Merge remote-tracking branch 'origin/marin/plugin-ml-4-completion-performace' into marin/plugin-ml-4-completion-performace # Conflicts: # community/plugins/turboComplete/src/com/intellij/codeInsight/completion/KindExecutingCompletionContributor.kt # intellij.idea.ultimate.main.iml Change default policy to buffering Use local models for testing Record actual performance status to context Refactor plugin structure Fix completion kind collection Fix completion kind collection Add registry configuration options Implement kind collection & split reference for Kotlin Implement fast completion for Kotlin Initialize performance plugin Use local models for testing Record actual performance status to context Refactor plugin structure Fix completion kind collection Fix completion kind collection Add registry configuration options Implement kind collection & split reference for Kotlin Implement fast completion for Kotlin Initialize performance plugin Merge-request: IJ-MR-103789 Merged-by: Gleb Marin <Gleb.Marin@jetbrains.com> GitOrigin-RevId: 1859b97a729b6c123fa22d6d9b6518e836ac6dec
This commit is contained in:
committed by
intellij-monorepo-bot
parent
e1b2d92857
commit
a71d5fc374
3
.idea/modules.xml
generated
3
.idea/modules.xml
generated
@@ -1053,6 +1053,9 @@
|
||||
<module fileurl="file://$PROJECT_DIR$/jps/standalone-builder/intellij.tools.jps.build.standalone.iml" filepath="$PROJECT_DIR$/jps/standalone-builder/intellij.tools.jps.build.standalone.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/tools/launcher-generator/intellij.tools.launcherGenerator.iml" filepath="$PROJECT_DIR$/tools/launcher-generator/intellij.tools.launcherGenerator.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/platform/testFramework/bootstrap/intellij.tools.testsBootstrap.iml" filepath="$PROJECT_DIR$/platform/testFramework/bootstrap/intellij.tools.testsBootstrap.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/plugins/turboComplete/intellij.turboComplete.iml" filepath="$PROJECT_DIR$/plugins/turboComplete/intellij.turboComplete.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/plugins/turboComplete/languages/kotlin/intellij.turboComplete.languages.kotlin.iml" filepath="$PROJECT_DIR$/plugins/turboComplete/languages/kotlin/intellij.turboComplete.languages.kotlin.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/plugins/turboComplete/intellij.turboComplete.tests.iml" filepath="$PROJECT_DIR$/plugins/turboComplete/intellij.turboComplete.tests.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/plugins/ui-designer-core/intellij.uiDesigner.iml" filepath="$PROJECT_DIR$/plugins/ui-designer-core/intellij.uiDesigner.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/plugins/changeReminder/intellij.vcs.changeReminder.iml" filepath="$PROJECT_DIR$/plugins/changeReminder/intellij.vcs.changeReminder.iml" />
|
||||
<module fileurl="file://$PROJECT_DIR$/plugins/git4idea/intellij.vcs.git.iml" filepath="$PROJECT_DIR$/plugins/git4idea/intellij.vcs.git.iml" />
|
||||
|
||||
@@ -101,6 +101,7 @@ val IDEA_BUNDLED_PLUGINS: PersistentList<String> = DEFAULT_BUNDLED_PLUGINS + per
|
||||
"intellij.keymap.visualStudio",
|
||||
"intellij.keymap.netbeans",
|
||||
"intellij.performanceTesting",
|
||||
"intellij.turboComplete",
|
||||
)
|
||||
|
||||
val CE_CLASS_VERSIONS: PersistentMap<String, String> = BASE_CLASS_VERSIONS.putAll(persistentHashMapOf(
|
||||
|
||||
@@ -282,6 +282,12 @@ object CommunityRepositoryModules {
|
||||
},
|
||||
plugin("intellij.editorconfig") { spec ->
|
||||
spec.withProjectLibrary("ec4j-core")
|
||||
},
|
||||
plugin(
|
||||
"intellij.turboComplete",
|
||||
) { spec ->
|
||||
spec.bundlingRestrictions.includeInEapOnly = true
|
||||
spec.withModule("intellij.turboComplete.languages.kotlin")
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.codeInsight.completion
|
||||
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.openapi.application.ApplicationManager
|
||||
import com.intellij.openapi.application.invokeLater
|
||||
import org.jetbrains.annotations.ApiStatus
|
||||
|
||||
/**
|
||||
* A singleton that provides the functionality to open the code completion lookups pop-up.
|
||||
*
|
||||
* @see [LookupImpl]
|
||||
* @see [CompletionProgressIndicator]
|
||||
*/
|
||||
@ApiStatus.Internal
|
||||
object CompletionLookupOpener {
|
||||
private fun LookupImpl.shownTimestampInitialized() = this.shownTimestampMillis != 0L
|
||||
|
||||
/**
|
||||
* Schedules a request to open lookup on the AWT event dispatching thread under Write Intent lock.
|
||||
* The request expires, if the lookup is already opened.
|
||||
*/
|
||||
fun showLookup(parameters: CompletionParameters) {
|
||||
val process = parameters.process
|
||||
if (process !is CompletionProgressIndicator) {
|
||||
return
|
||||
}
|
||||
val lookup = process.lookup
|
||||
if (lookup.shownTimestampInitialized()) return
|
||||
ApplicationManager.getApplication().invokeLater(process::showLookup) { lookup.shownTimestampInitialized() }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.codeInsight.completion
|
||||
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.openapi.progress.ProgressManager
|
||||
import com.intellij.patterns.ElementPattern
|
||||
import com.intellij.util.Consumer
|
||||
import org.jetbrains.annotations.ApiStatus
|
||||
|
||||
@ApiStatus.Internal
|
||||
class FilteringResultSet(
|
||||
private val base: CompletionResultSet,
|
||||
private val filter: (CompletionContributor) -> Boolean
|
||||
) : CompletionResultSet(base.prefixMatcher, base.consumer, base.myContributor) {
|
||||
override fun addElement(element: LookupElement) {
|
||||
base.addElement(element)
|
||||
}
|
||||
|
||||
override fun withPrefixMatcher(matcher: PrefixMatcher): CompletionResultSet {
|
||||
return FilteringResultSet(base.withPrefixMatcher(matcher), filter)
|
||||
}
|
||||
|
||||
override fun withPrefixMatcher(prefix: String): CompletionResultSet {
|
||||
return FilteringResultSet(base.withPrefixMatcher(prefix), filter)
|
||||
}
|
||||
|
||||
override fun withRelevanceSorter(sorter: CompletionSorter): CompletionResultSet {
|
||||
return FilteringResultSet(base.withRelevanceSorter(sorter), filter)
|
||||
}
|
||||
|
||||
override fun addLookupAdvertisement(text: String) {
|
||||
base.addLookupAdvertisement(text)
|
||||
}
|
||||
|
||||
override fun caseInsensitive(): CompletionResultSet {
|
||||
return FilteringResultSet(base.caseInsensitive(), filter)
|
||||
}
|
||||
|
||||
override fun restartCompletionOnPrefixChange(prefixCondition: ElementPattern<String>?) {
|
||||
base.restartCompletionWhenNothingMatches()
|
||||
}
|
||||
|
||||
override fun restartCompletionWhenNothingMatches() {
|
||||
base.restartCompletionWhenNothingMatches()
|
||||
}
|
||||
|
||||
override fun runRemainingContributors(parameters: CompletionParameters,
|
||||
consumer: Consumer<in CompletionResult>,
|
||||
stop: Boolean,
|
||||
customSorter: CompletionSorter?) {
|
||||
if (stop) {
|
||||
stopHere()
|
||||
}
|
||||
val batchConsumer = object : BatchConsumer<CompletionResult?> {
|
||||
override fun startBatch() {
|
||||
this@FilteringResultSet.startBatch()
|
||||
}
|
||||
|
||||
override fun endBatch() {
|
||||
this@FilteringResultSet.endBatch()
|
||||
}
|
||||
|
||||
override fun consume(result: CompletionResult?) {
|
||||
consumer.consume(result)
|
||||
}
|
||||
}
|
||||
myCompletionService.getVariantsFromContributors(parameters, myContributor, prefixMatcher, batchConsumer, customSorter, filter)
|
||||
}
|
||||
|
||||
companion object {
|
||||
private fun CompletionService.getVariantsFromContributors(parameters: CompletionParameters,
|
||||
from: CompletionContributor,
|
||||
matcher: PrefixMatcher,
|
||||
consumer: Consumer<in CompletionResult?>,
|
||||
customSorter: CompletionSorter?,
|
||||
filter: (CompletionContributor) -> Boolean) {
|
||||
val contributors = CompletionContributor.forParameters(parameters)
|
||||
for (i in contributors.indexOf(from) + 1 until contributors.size) {
|
||||
ProgressManager.checkCanceled()
|
||||
val contributor = contributors[i]
|
||||
if (!filter(contributor)) {
|
||||
continue
|
||||
}
|
||||
var result: CompletionResultSet = FilteringResultSet(createResultSet(parameters, consumer, contributor, matcher), filter)
|
||||
if (customSorter != null) {
|
||||
result = result.withRelevanceSorter(customSorter)
|
||||
}
|
||||
getVariantsFromContributor(parameters, contributor, result)
|
||||
if (result.isStopped) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.codeInsight.completion
|
||||
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.patterns.ElementPattern
|
||||
import org.jetbrains.annotations.ApiStatus
|
||||
|
||||
@ApiStatus.Internal
|
||||
class PolicyObeyingResultSet(
|
||||
private val originalResult: CompletionResultSet,
|
||||
private val policyHolder: () -> ElementsAddingPolicy
|
||||
) : CompletionResultSet(originalResult.prefixMatcher, originalResult.consumer, originalResult.myContributor) {
|
||||
|
||||
override fun addElement(element: LookupElement) {
|
||||
policyHolder().addElement(originalResult, element)
|
||||
}
|
||||
|
||||
override fun addAllElements(elements: MutableIterable<LookupElement>) {
|
||||
policyHolder().addAllElements(originalResult, elements)
|
||||
}
|
||||
|
||||
override fun withPrefixMatcher(matcher: PrefixMatcher): CompletionResultSet {
|
||||
return PolicyObeyingResultSet(originalResult.withPrefixMatcher(matcher), policyHolder)
|
||||
}
|
||||
|
||||
override fun withPrefixMatcher(prefix: String): CompletionResultSet {
|
||||
return PolicyObeyingResultSet(originalResult.withPrefixMatcher(prefix), policyHolder)
|
||||
}
|
||||
|
||||
override fun withRelevanceSorter(sorter: CompletionSorter): CompletionResultSet {
|
||||
return PolicyObeyingResultSet(originalResult.withRelevanceSorter(sorter), policyHolder)
|
||||
}
|
||||
|
||||
override fun addLookupAdvertisement(text: String) {
|
||||
originalResult.addLookupAdvertisement(text)
|
||||
}
|
||||
|
||||
override fun caseInsensitive(): CompletionResultSet {
|
||||
return PolicyObeyingResultSet(originalResult.caseInsensitive(), policyHolder)
|
||||
}
|
||||
|
||||
override fun restartCompletionOnPrefixChange(prefixCondition: ElementPattern<String>?) {
|
||||
originalResult.restartCompletionOnPrefixChange(prefixCondition)
|
||||
}
|
||||
|
||||
override fun restartCompletionWhenNothingMatches() {
|
||||
originalResult.restartCompletionWhenNothingMatches()
|
||||
}
|
||||
|
||||
override fun isStopped(): Boolean {
|
||||
return originalResult.isStopped
|
||||
}
|
||||
|
||||
override fun stopHere() {
|
||||
policyHolder().onResultStop(originalResult)
|
||||
originalResult.stopHere()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.codeInsight.completion.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import org.jetbrains.annotations.ApiStatus
|
||||
|
||||
/**
|
||||
* Policy, that controls how exactly elements should be
|
||||
* added to the result set
|
||||
*/
|
||||
@ApiStatus.Internal
|
||||
interface ElementsAddingPolicy {
|
||||
/**
|
||||
* Called when the policy should come into effect
|
||||
*
|
||||
* @see #onDeactivate
|
||||
*/
|
||||
fun onActivate(result: CompletionResultSet) {}
|
||||
|
||||
/**
|
||||
* Called when result's {@link com.intellij.codeInsight.completion.CompletionResultSet#stopHere} was called
|
||||
*/
|
||||
fun onResultStop(result: CompletionResultSet) {}
|
||||
|
||||
/**
|
||||
* Called when another [element] should be added to [result]
|
||||
*/
|
||||
fun addElement(result: CompletionResultSet, element: LookupElement)
|
||||
|
||||
/**
|
||||
* Called when all [elements] should be added to [result]
|
||||
*/
|
||||
fun addAllElements(result: CompletionResultSet, elements: MutableIterable<LookupElement>)
|
||||
|
||||
/**
|
||||
* Called when the policy should end its action
|
||||
*
|
||||
* @see #onActivate
|
||||
*/
|
||||
fun onDeactivate(result: CompletionResultSet) {}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.codeInsight.completion.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.PolicyObeyingResultSet
|
||||
import org.jetbrains.annotations.ApiStatus
|
||||
import com.intellij.util.containers.Stack
|
||||
|
||||
/**
|
||||
* An intermediary instance, that controls the policy of in which manner
|
||||
* should completion results be added to the [originalResult]
|
||||
*/
|
||||
@ApiStatus.Internal
|
||||
class PolicyController(protected val originalResult: CompletionResultSet) : () -> ElementsAddingPolicy {
|
||||
private val policies: Stack<ElementsAddingPolicy> = Stack()
|
||||
|
||||
/**
|
||||
* Make the [policy] rule how elements are added to the [originalResult]
|
||||
* If there is already an active policy A in the controller, than it
|
||||
* will be put on the stack. So that when the newly added policy will
|
||||
* be popped, the policy A will be in action again.
|
||||
*
|
||||
* @see popPolicy
|
||||
*/
|
||||
fun pushPolicy(policy: ElementsAddingPolicy) {
|
||||
policies.push(policy)
|
||||
policy.onActivate(originalResult)
|
||||
}
|
||||
|
||||
/**
|
||||
* Revoke currently active policy
|
||||
*
|
||||
* @throws NoActivePolicyException if there is no active policy
|
||||
* @see [pushPolicy]
|
||||
*/
|
||||
fun popPolicy() {
|
||||
verifyNotEmptyStack()
|
||||
val policyToDeactivate = policies.pop()
|
||||
policyToDeactivate.onDeactivate(originalResult)
|
||||
}
|
||||
|
||||
/**
|
||||
* @return A result set, that will be obeying to this controller
|
||||
*/
|
||||
fun getObeyingResultSet(): CompletionResultSet {
|
||||
return PolicyObeyingResultSet(originalResult, this)
|
||||
}
|
||||
|
||||
/**
|
||||
* Invoke the given action
|
||||
*/
|
||||
fun <T> invokeWithPolicy(policy: ElementsAddingPolicy, action: () -> T): T {
|
||||
pushPolicy(policy)
|
||||
try {
|
||||
return action()
|
||||
}
|
||||
finally {
|
||||
popPolicy()
|
||||
}
|
||||
}
|
||||
|
||||
override fun invoke(): ElementsAddingPolicy {
|
||||
verifyNotEmptyStack()
|
||||
return policies.peek()!!
|
||||
}
|
||||
|
||||
private fun verifyNotEmptyStack() {
|
||||
if (policies.isEmpty()) {
|
||||
throw NoActivePolicyException()
|
||||
}
|
||||
}
|
||||
|
||||
public class NoActivePolicyException : Exception("ElementsAddingPolicyController does not have an active policy")
|
||||
}
|
||||
@@ -20,5 +20,6 @@
|
||||
<orderEntry type="module" module-name="intellij.platform.testFramework" scope="TEST" />
|
||||
<orderEntry type="library" scope="TEST" name="assertJ" level="project" />
|
||||
<orderEntry type="module" module-name="intellij.platform.util.text.matching" />
|
||||
<orderEntry type="module" module-name="intellij.platform.lang.impl" />
|
||||
</component>
|
||||
</module>
|
||||
@@ -0,0 +1,11 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
/**
|
||||
* Data class representing a kind of [SuggestionGenerator]'s suggestions.
|
||||
*
|
||||
* Each completion kind belongs to a kind variety.
|
||||
* The completion kind's name is defined statically, it should be unique among
|
||||
* the corresponding [KindVariety].
|
||||
*/
|
||||
data class CompletionKind(val name: Enum<*>, val variety: KindVariety)
|
||||
@@ -0,0 +1,20 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
class ConditionalConsumer(
|
||||
base: SuggestionGeneratorConsumer,
|
||||
private val shouldExecuteKind: (SuggestionGenerator) -> Boolean,
|
||||
) : DelegatingConsumer(base) {
|
||||
override fun pass(suggestionGenerator: SuggestionGenerator) {
|
||||
val generator = SuggestionGenerator.withApplicability(
|
||||
suggestionGenerator.kind,
|
||||
suggestionGenerator.result,
|
||||
suggestionGenerator.parameters,
|
||||
suggestionGenerator::generateCompletionVariants
|
||||
) {
|
||||
shouldExecuteKind(suggestionGenerator)
|
||||
}
|
||||
|
||||
super.pass(generator)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
open class DelegatingConsumer(private val base: SuggestionGeneratorConsumer) : SuggestionGeneratorConsumer by base
|
||||
@@ -0,0 +1,53 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.openapi.extensions.ExtensionPointName
|
||||
/**
|
||||
* Collects all [SuggestionGenerator] of the same variety,
|
||||
* without executing them.
|
||||
*
|
||||
* The purpose of this class it to just acknowledge existence of
|
||||
* the generators, so then we can reorder the generators, and
|
||||
* run them in the order we want.
|
||||
*/
|
||||
interface KindCollector {
|
||||
/**
|
||||
* The variety of completion suggestions generated by collected [SuggestionGenerator]s
|
||||
*
|
||||
* An analogue of [com.intellij.codeInsight.completion.CompletionContributor],
|
||||
* but instead of executing all suggestion generators immediately, it delegates their execution
|
||||
*/
|
||||
val kindVariety: KindVariety
|
||||
|
||||
/**
|
||||
* Checks if this collector is relevant within the given parameters
|
||||
*/
|
||||
fun shouldBeCalled(parameters: CompletionParameters): Boolean
|
||||
|
||||
/**
|
||||
* Collects and adds [SuggestionGenerator]s to the consumer
|
||||
*
|
||||
* An analogue of [com.intellij.codeInsight.completion.CompletionContributor.fillCompletionVariants]
|
||||
*
|
||||
* @param parameters The completion parameters containing information about the current completion request.
|
||||
* @param generatorConsumer The suggestion generator consumer used to collect [SuggestionGenerator]s.
|
||||
* @param result The completion result set to which the suggestions will be added.
|
||||
*/
|
||||
fun collectKinds(parameters: CompletionParameters,
|
||||
generatorConsumer: SuggestionGeneratorConsumer,
|
||||
result: CompletionResultSet,
|
||||
resultPolicyController: PolicyController)
|
||||
|
||||
companion object {
|
||||
@JvmStatic
|
||||
val EP_NAME: ExtensionPointName<KindCollector> = ExtensionPointName.create(
|
||||
"com.intellij.turboComplete.kindCollector")
|
||||
|
||||
fun forParameters(parameters: CompletionParameters): List<KindCollector> {
|
||||
return EP_NAME.extensionList.filter { it.shouldBeCalled(parameters) }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
|
||||
/**
|
||||
* Listens how [SuggestionGenerator]s are executed
|
||||
* Be aware, that each listener is not reinitialized, but
|
||||
* the same instance used to convey information about generators'
|
||||
* execution each time.
|
||||
*
|
||||
* The functions are called in their declaration order.
|
||||
*/
|
||||
interface KindExecutionListener {
|
||||
/**
|
||||
* Code completion was just called
|
||||
*/
|
||||
fun onInitialize(parameters: CompletionParameters)
|
||||
|
||||
/**
|
||||
* Called before any [KindCollector] has collected any kinds,
|
||||
* so the collection process just began
|
||||
*/
|
||||
fun onCollectionStarted() {}
|
||||
|
||||
/**
|
||||
* A [SuggestionGenerator] had been collected.
|
||||
* There could be multiple suggestion generators, hence,
|
||||
* the function is called as many times.
|
||||
*/
|
||||
fun onGeneratorCollected(suggestionGenerator: SuggestionGenerator) {}
|
||||
|
||||
/**
|
||||
* All kinds have been collected
|
||||
*/
|
||||
fun onCollectionFinished() {}
|
||||
|
||||
/**
|
||||
* The generator started generating variants,
|
||||
* i.e. [SuggestionGenerator.generateCompletionVariants] was called
|
||||
*/
|
||||
fun onGenerationStarted(suggestionGenerator: SuggestionGenerator) {}
|
||||
|
||||
/**
|
||||
* The generator finished generating variants,
|
||||
* i.e. [SuggestionGenerator.generateCompletionVariants] finished
|
||||
* (either by an exception, or without)
|
||||
*/
|
||||
fun onGenerationFinished(suggestionGenerator: SuggestionGenerator) {}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
|
||||
/**
|
||||
* Represents a family of completion kinds.
|
||||
* One [KindCollector] collects kinds from the same family (from Java, Kotlin K1, Kotlin K2, etc.)
|
||||
*/
|
||||
interface KindVariety {
|
||||
/**
|
||||
* Checks, if the kind variety can be collected withing the given parameters
|
||||
*/
|
||||
fun kindsCorrespondToParameters(parameters: CompletionParameters): Boolean
|
||||
|
||||
/**
|
||||
* Temporary workaround
|
||||
*
|
||||
* Currently, [SuggestionGenerator] is a "fixed"
|
||||
* version of a [com.intellij.codeInsight.completion.CompletionContributor].
|
||||
* We can't dynamically remove one contributor, so we need to filter it out, so we don't have duplicates
|
||||
* (suggestions from the contributor, and from the duplicating suggestion generator).
|
||||
* And to do this, we must know, what is the actual contributor, that we are filtering out.
|
||||
*/
|
||||
val actualCompletionContributorClass: Class<*>
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.openapi.util.Key
|
||||
|
||||
/**
|
||||
* Generates suggestions for code completion. A simplified version of
|
||||
* [com.intellij.codeInsight.completion.CompletionContributor]
|
||||
*
|
||||
* Instances of this interface are generated by [KindCollector],
|
||||
* and then executed
|
||||
*/
|
||||
interface SuggestionGenerator {
|
||||
/**
|
||||
* Kind of completion variants, which will be suggested by this particular generator
|
||||
*/
|
||||
val kind: CompletionKind
|
||||
|
||||
/**
|
||||
* Result set, to where the generated completion variants will be stored after
|
||||
* [generateCompletionVariants] has been called
|
||||
*/
|
||||
val result: CompletionResultSet
|
||||
|
||||
/**
|
||||
* Parameters, which the generated suggestions will be satisfying
|
||||
*/
|
||||
val parameters: CompletionParameters
|
||||
|
||||
/**
|
||||
* Generates completion variants, which correspond to [parameters] and puts them
|
||||
* to [result]
|
||||
*/
|
||||
fun generateCompletionVariants()
|
||||
|
||||
companion object {
|
||||
@JvmStatic
|
||||
val LOOKUP_ELEMENT_SUGGESTION_GENERATOR = Key<SuggestionGenerator>("SuggestionGenerator which generated the lookup element")
|
||||
|
||||
@JvmStatic
|
||||
fun fromGenerator(kind: CompletionKind,
|
||||
parameters: CompletionParameters,
|
||||
result: CompletionResultSet,
|
||||
generateVariants: () -> Unit): SuggestionGenerator {
|
||||
return object : SuggestionGenerator {
|
||||
override val kind = kind
|
||||
override val result = result
|
||||
override val parameters = parameters
|
||||
|
||||
override fun generateCompletionVariants() {
|
||||
generateVariants()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@JvmStatic
|
||||
fun withApplicability(kind: CompletionKind,
|
||||
result: CompletionResultSet,
|
||||
parameters: CompletionParameters,
|
||||
generator: () -> Unit,
|
||||
isApplicable: () -> Boolean): SuggestionGenerator {
|
||||
return fromGenerator(
|
||||
kind,
|
||||
parameters,
|
||||
result
|
||||
) { if (isApplicable()) generator() else Unit }
|
||||
}
|
||||
|
||||
val LookupElement.suggestionGenerator
|
||||
get() = this.getUserData(LOOKUP_ELEMENT_SUGGESTION_GENERATOR)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
/**
|
||||
* An interface for classes that consume suggestion generators,
|
||||
* generated by [KindCollector]
|
||||
*/
|
||||
interface SuggestionGeneratorConsumer {
|
||||
fun pass(suggestionGenerator: SuggestionGenerator)
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.platform.ml.impl.turboComplete.addingPolicy.CollectionFillingPolicy
|
||||
import com.intellij.platform.ml.impl.turboComplete.addingPolicy.PassDirectlyPolicy
|
||||
|
||||
/**
|
||||
* Generates completion suggestions for a certain type of completion kind and stores
|
||||
* a cached artifact for later use.
|
||||
*
|
||||
* For example, we have two [SuggestionGenerator]s A and B.
|
||||
* While working, A generates an artifact, that is used later by B.
|
||||
* And the ML model decided, that B must be executed earlier.
|
||||
*
|
||||
* If we execute B first, then we want to preserve the recommended order of the
|
||||
* generators, and we don't want A to add completion variants to the lookup first.
|
||||
* Instead, A could be a [SuggestionGeneratorWithArtifact]. Then
|
||||
* 1. B asks A to create the artifact - [getArtifact]
|
||||
* 2. B generates completion variants
|
||||
* (it's A's order now to generate variants)
|
||||
* 3. A will only collect put cached lookup elements to the result set
|
||||
*/
|
||||
abstract class SuggestionGeneratorWithArtifact<T>(override val kind: CompletionKind,
|
||||
override val result: CompletionResultSet,
|
||||
private val resultPolicyController: PolicyController,
|
||||
override val parameters: CompletionParameters) : SuggestionGenerator {
|
||||
|
||||
private var cachedArtifact: T? = null
|
||||
private var cachedLookupElements: MutableList<LookupElement>? = null
|
||||
|
||||
fun getArtifact(): T {
|
||||
return cachedArtifact ?: run {
|
||||
val generatedLookupElements = mutableListOf<LookupElement>()
|
||||
val generatedArtifact = resultPolicyController.invokeWithPolicy(CollectionFillingPolicy(generatedLookupElements)) {
|
||||
generateVariantsAndArtifact()
|
||||
}
|
||||
cachedLookupElements = generatedLookupElements
|
||||
cachedArtifact = generatedArtifact
|
||||
generatedArtifact
|
||||
}
|
||||
}
|
||||
|
||||
override fun generateCompletionVariants() {
|
||||
cachedLookupElements?.let {
|
||||
if (it.isEmpty()) return
|
||||
resultPolicyController.invokeWithPolicy(PassDirectlyPolicy()) {
|
||||
result.addAllElements(it)
|
||||
}
|
||||
it.clear()
|
||||
} ?: run {
|
||||
cachedArtifact = resultPolicyController.invokeWithPolicy(PassDirectlyPolicy()) {
|
||||
generateVariantsAndArtifact()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
abstract fun generateVariantsAndArtifact(): T
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
|
||||
/**
|
||||
* Fill the given collection, when an element is added
|
||||
*/
|
||||
class CollectionFillingPolicy(private val addedElements: MutableCollection<LookupElement>) : ElementsAddingPolicy {
|
||||
|
||||
override fun addElement(result: CompletionResultSet, element: LookupElement) {
|
||||
addedElements.add(element)
|
||||
}
|
||||
|
||||
override fun addAllElements(result: CompletionResultSet, elements: MutableIterable<LookupElement>) {
|
||||
addedElements.addAll(elements)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
|
||||
/**
|
||||
* Pass all elements directly to the result set
|
||||
*/
|
||||
class PassDirectlyPolicy : ElementsAddingPolicy {
|
||||
override fun addElement(result: CompletionResultSet, element: LookupElement) {
|
||||
result.addElement(element)
|
||||
}
|
||||
|
||||
override fun addAllElements(result: CompletionResultSet, elements: MutableIterable<LookupElement>) {
|
||||
result.addAllElements(elements)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete.ranking
|
||||
|
||||
/**
|
||||
* Listens to the ranking process of [com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator]
|
||||
*
|
||||
* The same instance of the listener used during all the application lifetime.
|
||||
* The callbacks are called in the order of their declaration
|
||||
*/
|
||||
interface KindRankingListener {
|
||||
/**
|
||||
* The ranking process started
|
||||
*/
|
||||
fun onRankingStarted() {}
|
||||
|
||||
/**
|
||||
* The kinds were ranked
|
||||
*/
|
||||
fun onRanked(ranked: List<RankedKind>) {}
|
||||
|
||||
/**
|
||||
* The ranking process finished
|
||||
*/
|
||||
fun onRankingFinished() {}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.platform.ml.impl.turboComplete.ranking
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
|
||||
data class RankedKind(
|
||||
val kind: CompletionKind,
|
||||
val relevance: Double?,
|
||||
) {
|
||||
companion object {
|
||||
fun fromWeights(
|
||||
kindWeights: Iterable<Pair<CompletionKind, Double>>,
|
||||
negateWeight: Boolean,
|
||||
): List<RankedKind> {
|
||||
return kindWeights
|
||||
.sortedBy { (_, weight) -> if (negateWeight) -weight else weight }
|
||||
.map { (kind, weight) -> RankedKind(kind, weight) }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -57,6 +57,27 @@
|
||||
"useMLRanking": true,
|
||||
"showArrows": false,
|
||||
"calculateFeatures": true
|
||||
},
|
||||
{
|
||||
"number": 17,
|
||||
"description": "Completion performance (early lookup & MLRanking)",
|
||||
"useMLRanking": true,
|
||||
"showArrows": false,
|
||||
"calculateFeatures": true
|
||||
},
|
||||
{
|
||||
"number": 18,
|
||||
"description": "Completion performance (early lookup)",
|
||||
"useMLRanking": false,
|
||||
"showArrows": false,
|
||||
"calculateFeatures": true
|
||||
},
|
||||
{
|
||||
"number": 19,
|
||||
"description": "Completion performance",
|
||||
"useMLRanking": false,
|
||||
"showArrows": false,
|
||||
"calculateFeatures": true
|
||||
}
|
||||
],
|
||||
"languages": [
|
||||
@@ -75,8 +96,8 @@
|
||||
{
|
||||
"id": "kotlin",
|
||||
"experimentBucketsCount": 8,
|
||||
"includeGroups": [ 7, 8, 11, 12, 13 ],
|
||||
"shouldLogElementFeatures": false
|
||||
"includeGroups": [ 7, 8, 8, 12, 13, 17, 17, 17 ],
|
||||
"shouldLogElementFeatures": true
|
||||
},
|
||||
{
|
||||
"id": "scala",
|
||||
|
||||
@@ -7,6 +7,7 @@ import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.ElementFeatureProvider
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.openapi.util.Key
|
||||
|
||||
class CommonElementLocationFeatures : ElementFeatureProvider {
|
||||
override fun getName(): String = "common"
|
||||
@@ -34,9 +35,15 @@ class CommonElementLocationFeatures : ElementFeatureProvider {
|
||||
}
|
||||
|
||||
element.getUserData(BaseCompletionService.LOOKUP_ELEMENT_CONTRIBUTOR)?.let {
|
||||
result["contributor"] = MLFeatureValue.className(it::class.java)
|
||||
val actualCompletionContributor: Class<*>? = element.getUserData(LOOKUP_ORIGINAL_ELEMENT_CONTRIBUTOR_TYPE)
|
||||
result["contributor"] = MLFeatureValue.className(actualCompletionContributor ?: it::class.java)
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
companion object {
|
||||
// For the TurboComplete plugin, to not confuse the ML model with new unknown contributors
|
||||
val LOOKUP_ORIGINAL_ELEMENT_CONTRIBUTOR_TYPE = Key<Class<*>>("original contributor of the element")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -40,8 +40,6 @@ class CommonLocationFeatures : ContextFeatureProvider {
|
||||
"chars_in_line_after_caret" to MLFeatureValue.float(lineEndOffset - caretOffset)
|
||||
)
|
||||
|
||||
result["is_completion_performance_mode"] = MLFeatureValue.binary(false)
|
||||
|
||||
if (DumbService.isDumb(lookup.project)) {
|
||||
result["dumb_mode"] = MLFeatureValue.binary(true)
|
||||
}
|
||||
|
||||
@@ -10,12 +10,25 @@ import java.util.concurrent.atomic.LongAdder
|
||||
|
||||
|
||||
class MLCompletionPerformanceTracker {
|
||||
private val tracker: MeasuredTracker = MeasuredTracker(OTelTracker())
|
||||
private val measuredTracker = MeasuredTracker()
|
||||
private val tracker = DelegatingTracker(listOf(OTelTracker(), measuredTracker))
|
||||
private val elementProvidersMeasurer: ConcurrentHashMap<String, LongAdder> = ConcurrentHashMap()
|
||||
|
||||
private var sortingCount = 0
|
||||
private var totalMlContribution: Long = 0L
|
||||
|
||||
private val metricCollectors: MutableList<PerformanceMetricCollector> = mutableListOf()
|
||||
|
||||
fun addMetricCollector(metricCollectorFactory: PerformanceMetricCollectorFactory) {
|
||||
val performanceTracker = object : PerformanceTracker {
|
||||
override fun addByKey(key: String, timeMs: Long) {
|
||||
tracker.addByKey("${metricCollectorFactory.performanceMetricName}.$key", timeMs)
|
||||
}
|
||||
}
|
||||
val metricCollector = metricCollectorFactory.createMetricCollector(performanceTracker)
|
||||
metricCollectors.add(metricCollector)
|
||||
}
|
||||
|
||||
fun totalMLTimeContribution(): Long = totalMlContribution
|
||||
|
||||
fun sortingPerformed(itemsCount: Int, totalTime: Long) {
|
||||
@@ -55,8 +68,9 @@ class MLCompletionPerformanceTracker {
|
||||
}
|
||||
|
||||
fun measurements(): Map<String, Long> {
|
||||
metricCollectors.forEach { it.onFinishCollecting() }
|
||||
flushElementProvidersContribution()
|
||||
return tracker.measurements()
|
||||
return measuredTracker.measurements()
|
||||
}
|
||||
|
||||
private fun flushElementProvidersContribution() {
|
||||
@@ -70,11 +84,7 @@ class MLCompletionPerformanceTracker {
|
||||
}
|
||||
}
|
||||
|
||||
private interface PerfTracker {
|
||||
fun addByKey(key: String, timeMs: Long)
|
||||
}
|
||||
|
||||
private class OTelTracker : PerfTracker {
|
||||
private class OTelTracker : PerformanceTracker {
|
||||
private val meter = TelemetryTracer.getMeter(CompletionRanking)
|
||||
private val key2counter: MutableMap<String, LongCounter> = mutableMapOf()
|
||||
override fun addByKey(key: String, timeMs: Long) {
|
||||
@@ -84,10 +94,15 @@ class MLCompletionPerformanceTracker {
|
||||
}
|
||||
}
|
||||
|
||||
private class MeasuredTracker(private val delegate: PerfTracker) : PerfTracker {
|
||||
private class DelegatingTracker(private val trackers: List<PerformanceTracker>) : PerformanceTracker {
|
||||
override fun addByKey(key: String, timeMs: Long) {
|
||||
trackers.forEach { it.addByKey(key, timeMs) }
|
||||
}
|
||||
}
|
||||
|
||||
private class MeasuredTracker : PerformanceTracker {
|
||||
private val measurements: ConcurrentHashMap<String, LongAdder> = ConcurrentHashMap()
|
||||
override fun addByKey(key: String, timeMs: Long) {
|
||||
delegate.addByKey(key, timeMs)
|
||||
measurements.computeIfAbsent(key) { LongAdder() }.add(timeMs)
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.completion.ml.performance
|
||||
|
||||
interface PerformanceMetricCollector {
|
||||
fun onFinishCollecting() {}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.completion.ml.performance
|
||||
|
||||
interface PerformanceMetricCollectorFactory {
|
||||
val performanceMetricName: String
|
||||
|
||||
fun createMetricCollector(tracker: PerformanceTracker): PerformanceMetricCollector
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.completion.ml.performance
|
||||
|
||||
interface PerformanceTracker {
|
||||
fun addByKey(key: String, timeMs: Long)
|
||||
}
|
||||
@@ -2,17 +2,17 @@
|
||||
package com.intellij.completion.ml.ranker.local
|
||||
|
||||
import com.intellij.internal.ml.DecisionFunction
|
||||
import com.intellij.openapi.diagnostic.Logger
|
||||
import com.intellij.openapi.diagnostic.logger
|
||||
import com.intellij.openapi.util.registry.Registry
|
||||
import com.intellij.util.concurrency.SequentialTaskExecutor
|
||||
import java.util.*
|
||||
import java.util.concurrent.Future
|
||||
import java.util.zip.ZipFile
|
||||
|
||||
object MLCompletionLocalModelsUtil {
|
||||
private const val REGISTRY_PATH_KEY = "completion.ml.path.to.zip.model"
|
||||
private val LOG = Logger.getInstance(MLCompletionLocalModelsUtil::class.java)
|
||||
class MLCompletionLocalModelsLoader(private val registryPathKey: String) {
|
||||
private val executor = SequentialTaskExecutor.createSequentialApplicationPoolExecutor("MLCompletionTxtModelsUtil pool executor")
|
||||
@Volatile private var localModel: LocalModalInfo? = null
|
||||
@Volatile
|
||||
private var localModel: LocalModalInfo? = null
|
||||
|
||||
fun getModel(languageId: String): DecisionFunction? {
|
||||
if (!isPathToTheModelSet()) {
|
||||
@@ -25,7 +25,7 @@ object MLCompletionLocalModelsUtil {
|
||||
}
|
||||
|
||||
val resLocalModel = localModel ?: return null
|
||||
return if (languageId.toLowerCase() in resLocalModel.languages) {
|
||||
return if (languageId.lowercase(Locale.getDefault()) in resLocalModel.languages) {
|
||||
resLocalModel.decisionFunction
|
||||
}
|
||||
else {
|
||||
@@ -38,14 +38,14 @@ object MLCompletionLocalModelsUtil {
|
||||
*/
|
||||
private fun scheduleInitModel(): Future<*> = executor.submit { initModelFromPathToZipSynchronously() }
|
||||
|
||||
private fun isPathToTheModelSet() = Registry.get(REGISTRY_PATH_KEY).isChangedFromDefault
|
||||
private fun isPathToTheModelSet() = Registry.get(registryPathKey).isChangedFromDefault
|
||||
|
||||
private fun isPathToTheModelChanged() = Registry.stringValue(REGISTRY_PATH_KEY) != localModel?.path
|
||||
private fun isPathToTheModelChanged() = Registry.stringValue(registryPathKey) != localModel?.path
|
||||
|
||||
private fun initModelFromPathToZipSynchronously() {
|
||||
localModel = null
|
||||
val startTime = System.currentTimeMillis()
|
||||
val pathToZip = Registry.stringValue(REGISTRY_PATH_KEY)
|
||||
val pathToZip = Registry.stringValue(registryPathKey)
|
||||
localModel = loadModel(pathToZip)
|
||||
val endTime = System.currentTimeMillis()
|
||||
LOG.info("ML Completion local model initialization took: ${endTime - startTime} ms.")
|
||||
@@ -58,11 +58,16 @@ object MLCompletionLocalModelsUtil {
|
||||
val (decisionFunction, languages) = loader.loadModel(file)
|
||||
return LocalModalInfo(decisionFunction, pathToZip, languages.toSet())
|
||||
}
|
||||
} catch (t: Throwable) {
|
||||
}
|
||||
catch (t: Throwable) {
|
||||
LOG.error(t)
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
private data class LocalModalInfo(val decisionFunction: DecisionFunction, val path: String, val languages: Set<String>)
|
||||
|
||||
companion object {
|
||||
private val LOG = logger<MLCompletionLocalModelsLoader>()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package com.intellij.completion.ml.sorting
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.ml.CompletionEnvironment
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatureProvider
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.codeInsight.lookup.Lookup
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.completion.ml.storage.MutableLookupStorage
|
||||
import com.intellij.openapi.extensions.impl.ExtensionProcessingHelper
|
||||
import com.intellij.openapi.progress.ProgressManager
|
||||
import com.intellij.openapi.util.UserDataHolderBase
|
||||
import java.util.concurrent.TimeUnit
|
||||
import kotlin.system.measureTimeMillis
|
||||
|
||||
object ContextFactorCalculator {
|
||||
fun calculateContextFactors(lookup: LookupImpl, parameters: CompletionParameters, storage: MutableLookupStorage) {
|
||||
val environment = MyEnvironment(lookup, parameters)
|
||||
val contextFeatures = mutableMapOf<String, MLFeatureValue>()
|
||||
ExtensionProcessingHelper.forEachExtensionSafe(ContextFeatureProvider.forLanguage(storage.language)) { provider ->
|
||||
ProgressManager.checkCanceled()
|
||||
val providerName = provider.name
|
||||
val timeSpent = measureTimeMillis {
|
||||
val features = provider.calculateFeatures(environment)
|
||||
for ((featureName, value) in features) {
|
||||
contextFeatures["ml_ctx_${providerName}_$featureName"] = value
|
||||
}
|
||||
}
|
||||
storage.performanceTracker.contextFeaturesCalculated(providerName, TimeUnit.NANOSECONDS.toMillis(timeSpent))
|
||||
}
|
||||
for (contextFeatureProvider in AdditionalContextFeatureProvider.forLanguage(storage.language)) {
|
||||
contextFeatures.putAll(contextFeatureProvider.calculateFeatures(contextFeatures))
|
||||
}
|
||||
storage.initContextFactors(contextFeatures, environment)
|
||||
}
|
||||
}
|
||||
|
||||
private class MyEnvironment(
|
||||
private val lookup: LookupImpl,
|
||||
private val parameters: CompletionParameters
|
||||
) : CompletionEnvironment, UserDataHolderBase() {
|
||||
override fun getLookup(): Lookup = lookup
|
||||
|
||||
override fun getParameters(): CompletionParameters = parameters
|
||||
}
|
||||
@@ -4,19 +4,11 @@ package com.intellij.completion.ml.sorting
|
||||
import com.intellij.codeInsight.completion.CompletionContributor
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.ml.CompletionEnvironment
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatureProvider
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.codeInsight.lookup.Lookup
|
||||
import com.intellij.codeInsight.lookup.LookupManager
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.completion.ml.CompletionMLPolicy
|
||||
import com.intellij.openapi.progress.ProgressManager
|
||||
import com.intellij.openapi.project.DumbAware
|
||||
import com.intellij.openapi.util.UserDataHolderBase
|
||||
import com.intellij.completion.ml.storage.MutableLookupStorage
|
||||
import com.intellij.openapi.extensions.impl.ExtensionProcessingHelper
|
||||
import java.util.concurrent.TimeUnit
|
||||
|
||||
class ContextFeaturesContributor : CompletionContributor(), DumbAware {
|
||||
override fun fillCompletionVariants(parameters: CompletionParameters, result: CompletionResultSet) {
|
||||
@@ -29,42 +21,10 @@ class ContextFeaturesContributor : CompletionContributor(), DumbAware {
|
||||
storage.disableReRanking()
|
||||
}
|
||||
if (storage.shouldComputeFeatures() && !storage.isContextFactorsInitialized()) {
|
||||
calculateContextFactors(lookup, parameters, storage)
|
||||
ContextFactorCalculator.calculateContextFactors(lookup, parameters, storage)
|
||||
}
|
||||
}
|
||||
}
|
||||
super.fillCompletionVariants(parameters, result)
|
||||
}
|
||||
|
||||
|
||||
private fun calculateContextFactors(lookup: LookupImpl, parameters: CompletionParameters, storage: MutableLookupStorage) {
|
||||
val environment = MyEnvironment(lookup, parameters)
|
||||
val contextFeatures = mutableMapOf<String, MLFeatureValue>()
|
||||
ExtensionProcessingHelper.forEachExtensionSafe(ContextFeatureProvider.forLanguage(storage.language)) { provider ->
|
||||
ProgressManager.checkCanceled()
|
||||
val providerName = provider.name
|
||||
val start = System.nanoTime()
|
||||
val features = provider.calculateFeatures(environment)
|
||||
for ((featureName, value) in features) {
|
||||
contextFeatures["ml_ctx_${providerName}_$featureName"] = value
|
||||
}
|
||||
|
||||
val timeSpent = System.nanoTime() - start
|
||||
storage.performanceTracker.contextFeaturesCalculated(providerName, TimeUnit.NANOSECONDS.toMillis(timeSpent))
|
||||
}
|
||||
for (contextFeatureProvider in AdditionalContextFeatureProvider.forLanguage(storage.language)) {
|
||||
contextFeatures.putAll(contextFeatureProvider.calculateFeatures(contextFeatures))
|
||||
}
|
||||
storage.initContextFactors(contextFeatures, environment)
|
||||
}
|
||||
|
||||
|
||||
private class MyEnvironment(
|
||||
private val lookup: LookupImpl,
|
||||
private val parameters: CompletionParameters
|
||||
) : CompletionEnvironment, UserDataHolderBase() {
|
||||
override fun getLookup(): Lookup = lookup
|
||||
|
||||
override fun getParameters(): CompletionParameters = parameters
|
||||
}
|
||||
}
|
||||
@@ -4,7 +4,7 @@ package com.intellij.completion.ml.sorting
|
||||
import com.intellij.completion.ml.experiment.ExperimentStatus
|
||||
import com.intellij.completion.ml.ranker.ExperimentModelProvider
|
||||
import com.intellij.completion.ml.ranker.ExperimentModelProvider.Companion.match
|
||||
import com.intellij.completion.ml.ranker.local.MLCompletionLocalModelsUtil
|
||||
import com.intellij.completion.ml.ranker.local.MLCompletionLocalModelsLoader
|
||||
import com.intellij.completion.ml.settings.CompletionMLRankingSettings
|
||||
import com.intellij.internal.ml.completion.DecoratingItemsPolicy
|
||||
import com.intellij.internal.ml.completion.RankingModelProvider
|
||||
@@ -18,9 +18,11 @@ import org.jetbrains.annotations.TestOnly
|
||||
object RankingSupport {
|
||||
private val LOG = logger<RankingSupport>()
|
||||
private var enabledInTests: Boolean = false
|
||||
private val localDebugModelLoader = MLCompletionLocalModelsLoader("completion.ml.path.to.zip.model")
|
||||
|
||||
fun getRankingModel(language: Language): RankingModelWrapper? {
|
||||
MLCompletionLocalModelsUtil.getModel(language.id)?.let { return LanguageRankingModel(it, DecoratingItemsPolicy.DISABLED) }
|
||||
tryLoadLocalDebugModel(language)?.let { return it }
|
||||
|
||||
val provider = findProviderSafe(language)
|
||||
return if (provider != null && shouldSortByML(language, provider)) tryGetModel(provider) else null
|
||||
}
|
||||
@@ -73,6 +75,12 @@ object RankingSupport {
|
||||
return shouldSort
|
||||
}
|
||||
|
||||
private fun tryLoadLocalDebugModel(language: Language): LanguageRankingModel? {
|
||||
return localDebugModelLoader.getModel(language.id)?.let {
|
||||
return LanguageRankingModel(it, DecoratingItemsPolicy.DISABLED)
|
||||
}
|
||||
}
|
||||
|
||||
@TestOnly
|
||||
fun enableInTests(parentDisposable: Disposable) {
|
||||
enabledInTests = true
|
||||
|
||||
@@ -38,5 +38,6 @@
|
||||
<orderEntry type="module" module-name="intellij.java.impl" />
|
||||
<orderEntry type="module" module-name="kotlin.code-insight.api" />
|
||||
<orderEntry type="module" module-name="intellij.platform.util.text.matching" />
|
||||
<orderEntry type="module" module-name="intellij.platform.ml.impl" />
|
||||
</component>
|
||||
</module>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,173 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package org.jetbrains.kotlin.idea.completion
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.CompletionType
|
||||
import com.intellij.codeInsight.completion.PrefixMatcher
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.codeInsight.completion.impl.CamelHumpMatcher
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.openapi.extensions.InternalIgnoreDependencyViolation
|
||||
import com.intellij.openapi.util.ThrowableComputable
|
||||
import com.intellij.openapi.util.registry.Registry
|
||||
import com.intellij.patterns.PsiJavaPatterns.elementType
|
||||
import com.intellij.patterns.PsiJavaPatterns.psiElement
|
||||
import com.intellij.platform.ml.impl.turboComplete.ConditionalConsumer
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindCollector
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGeneratorConsumer
|
||||
import com.intellij.psi.PsiComment
|
||||
import com.intellij.util.indexing.DumbModeAccessType
|
||||
import org.jetbrains.kotlin.idea.completion.implCommon.stringTemplates.StringTemplateCompletion
|
||||
import org.jetbrains.kotlin.idea.completion.smart.SmartCompletionSession
|
||||
import org.jetbrains.kotlin.idea.completion.stringTemplates.wrapLookupElementForStringTemplateAfterDotCompletion
|
||||
import org.jetbrains.kotlin.lexer.KtTokens
|
||||
import org.jetbrains.kotlin.psi.KtFile
|
||||
import org.jetbrains.kotlin.psi.KtNameReferenceExpression
|
||||
import org.jetbrains.kotlin.psi.psiUtil.getNonStrictParentOfType
|
||||
|
||||
/**
|
||||
* Collects [com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator]s for Kotlin K1 code completion.
|
||||
* It is an analogue of [KotlinCompletionContributor]
|
||||
*/
|
||||
@InternalIgnoreDependencyViolation
|
||||
class KotlinKindCollector : KindCollector {
|
||||
override val kindVariety = KotlinKindVariety
|
||||
|
||||
private val AFTER_NUMBER_LITERAL = psiElement().afterLeafSkipping(
|
||||
psiElement().withText(""),
|
||||
psiElement().withElementType(elementType().oneOf(KtTokens.FLOAT_LITERAL, KtTokens.INTEGER_LITERAL))
|
||||
)
|
||||
|
||||
private val AFTER_INTEGER_LITERAL_AND_DOT = psiElement().afterLeafSkipping(
|
||||
psiElement().withText("."),
|
||||
psiElement().withElementType(elementType().oneOf(KtTokens.INTEGER_LITERAL))
|
||||
)
|
||||
|
||||
override fun shouldBeCalled(parameters: CompletionParameters): Boolean {
|
||||
val position = parameters.position
|
||||
val parametersOriginFile = parameters.originalFile
|
||||
return position.containingFile is KtFile && parametersOriginFile is KtFile
|
||||
}
|
||||
|
||||
override fun collectKinds(
|
||||
parameters: CompletionParameters,
|
||||
generatorConsumer: SuggestionGeneratorConsumer,
|
||||
result: CompletionResultSet,
|
||||
resultPolicyController: PolicyController
|
||||
) {
|
||||
if (!shouldBeCalled(parameters)) return
|
||||
|
||||
StringTemplateCompletion.correctParametersForInStringTemplateCompletion(parameters)?.let { correctedParameters ->
|
||||
generateCompletionKinds(correctedParameters, generatorConsumer, result, resultPolicyController,
|
||||
::wrapLookupElementForStringTemplateAfterDotCompletion)
|
||||
return
|
||||
}
|
||||
|
||||
DumbModeAccessType.RELIABLE_DATA_ONLY.ignoreDumbMode(ThrowableComputable {
|
||||
generateCompletionKinds(parameters, generatorConsumer, result, resultPolicyController, null)
|
||||
})
|
||||
}
|
||||
|
||||
private fun generateCompletionKinds(parameters: CompletionParameters,
|
||||
suggestionGeneratorConsumer: SuggestionGeneratorConsumer,
|
||||
result: CompletionResultSet,
|
||||
resultPolicyController: PolicyController,
|
||||
lookupElementPostProcessor: ((LookupElement) -> LookupElement)?
|
||||
) {
|
||||
val position = parameters.position
|
||||
if (position.getNonStrictParentOfType<PsiComment>() != null) {
|
||||
// don't stop here, allow other contributors to run
|
||||
return
|
||||
}
|
||||
|
||||
if (shouldSuppressCompletion(parameters, result.prefixMatcher)) {
|
||||
result.stopHere()
|
||||
return
|
||||
}
|
||||
|
||||
if (PackageDirectiveCompletion.perform(parameters, result)) {
|
||||
result.stopHere()
|
||||
return
|
||||
}
|
||||
|
||||
for (extension in KotlinCompletionExtension.EP_NAME.extensionList) {
|
||||
if (extension.perform(parameters, result)) return
|
||||
}
|
||||
|
||||
fun addPostProcessor(session: CompletionSession) {
|
||||
if (lookupElementPostProcessor != null) {
|
||||
session.addLookupElementPostProcessor(lookupElementPostProcessor)
|
||||
}
|
||||
}
|
||||
|
||||
result.restartCompletionWhenNothingMatches()
|
||||
|
||||
val configuration = CompletionSessionConfiguration(parameters)
|
||||
if (parameters.completionType == CompletionType.BASIC) {
|
||||
val session = BasicCompletionKindGenerationSession(configuration, parameters, resultPolicyController,
|
||||
suggestionGeneratorConsumer)
|
||||
addPostProcessor(session)
|
||||
|
||||
if (parameters.isAutoPopup && session.shouldDisableAutoPopup()) {
|
||||
result.stopHere()
|
||||
return
|
||||
}
|
||||
|
||||
val primaryKindsAddedSomething = session.complete()
|
||||
if (!primaryKindsAddedSomething && parameters.invocationCount < 2) {
|
||||
val newConfiguration = CompletionSessionConfiguration(
|
||||
useBetterPrefixMatcherForNonImportedClasses = false,
|
||||
nonAccessibleDeclarations = false,
|
||||
javaGettersAndSetters = true,
|
||||
javaClassesNotToBeUsed = false,
|
||||
staticMembers = parameters.invocationCount > 0,
|
||||
dataClassComponentFunctions = true,
|
||||
excludeEnumEntries = configuration.excludeEnumEntries,
|
||||
)
|
||||
|
||||
val newSession = BasicCompletionKindGenerationSession(
|
||||
newConfiguration, parameters, resultPolicyController,
|
||||
ConditionalConsumer(suggestionGeneratorConsumer) {
|
||||
session.somethingAddedToResult
|
||||
}
|
||||
)
|
||||
|
||||
addPostProcessor(newSession)
|
||||
newSession.complete()
|
||||
}
|
||||
}
|
||||
else {
|
||||
val session = SmartCompletionSession(configuration, parameters, result)
|
||||
addPostProcessor(session)
|
||||
session.complete()
|
||||
}
|
||||
}
|
||||
|
||||
private fun shouldSuppressCompletion(parameters: CompletionParameters, prefixMatcher: PrefixMatcher): Boolean {
|
||||
val position = parameters.position
|
||||
val invocationCount = parameters.invocationCount
|
||||
|
||||
if (prefixMatcher is CamelHumpMatcher && prefixMatcher.isTypoTolerant) return true
|
||||
|
||||
// no completion inside number literals
|
||||
if (AFTER_NUMBER_LITERAL.accepts(position)) return true
|
||||
|
||||
// no completion auto-popup after integer and dot
|
||||
if (invocationCount == 0 && prefixMatcher.prefix.isEmpty() && AFTER_INTEGER_LITERAL_AND_DOT.accepts(position)) return true
|
||||
|
||||
if (invocationCount == 0 && Registry.`is`("kotlin.disable.auto.completion.inside.expression", false)) {
|
||||
val originalPosition = parameters.originalPosition
|
||||
val originalExpression = originalPosition?.getNonStrictParentOfType<KtNameReferenceExpression>()
|
||||
val expression = position.getNonStrictParentOfType<KtNameReferenceExpression>()
|
||||
|
||||
if (expression != null && originalExpression != null &&
|
||||
!expression.getReferencedName().startsWith(originalExpression.getReferencedName())
|
||||
) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package org.jetbrains.kotlin.idea.completion
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
import org.jetbrains.kotlin.idea.KotlinLanguage
|
||||
|
||||
enum class KotlinCompletionKindName {
|
||||
DSL_FUNCTION,
|
||||
SMART_ADDITIONAL_ITEM,
|
||||
REFERENCE_BASIC,
|
||||
REFERENCE_EXTENSION,
|
||||
PACKAGE_NAME,
|
||||
NAMED_ARGUMENT,
|
||||
EXTENSION_FUNCTION_TYPE_VALUE,
|
||||
CONTEXT_VARIABLE_TYPE_SC,
|
||||
CONTEXT_VARIABLE_TYPE_REFERENCE,
|
||||
STATIC_MEMBER_FROM_IMPORTS,
|
||||
NON_IMPORTED,
|
||||
DEBUGGER_VARIANTS,
|
||||
STATIC_MEMBER_OBJECT_MEMBER,
|
||||
STATIC_MEMBER_EXPLICIT_INHERITED,
|
||||
STATIC_MEMBER_INACCESSIBLE,
|
||||
KEYWORD_ONLY,
|
||||
OPERATOR_NAME,
|
||||
DECLARATION_NAME,
|
||||
TOP_LEVEL_CLASS_NAME,
|
||||
SUPER_QUALIFIER,
|
||||
DECLARATION_NAME_FROM_UNRESOLVED_OVERRIDE,
|
||||
PARAMETER_OR_VAR_NAME_AND_TYPE,
|
||||
}
|
||||
|
||||
object KotlinKindVariety : KindVariety {
|
||||
override fun kindsCorrespondToParameters(parameters: CompletionParameters): Boolean {
|
||||
return parameters.position.language == KotlinLanguage.INSTANCE
|
||||
}
|
||||
|
||||
override val actualCompletionContributorClass: Class<*>
|
||||
get() = KotlinCompletionContributor::class.java
|
||||
}
|
||||
@@ -119,6 +119,27 @@ class ReferenceVariantsCollector(
|
||||
consumer(extensions)
|
||||
}
|
||||
|
||||
data class ReferenceVariantsCollectors(
|
||||
val basic: Lazy<ReferenceVariants>,
|
||||
val extensions: Lazy<ReferenceVariants>
|
||||
)
|
||||
|
||||
fun makeReferenceVariantsCollectors(descriptorKindFilter: DescriptorKindFilter): ReferenceVariantsCollectors {
|
||||
val config = configuration(descriptorKindFilter)
|
||||
|
||||
val basic = lazy {
|
||||
assert(!isCollectingFinished)
|
||||
collectBasicVariants(config).fixDescriptors()
|
||||
}
|
||||
|
||||
val extensions = lazy {
|
||||
assert(!isCollectingFinished)
|
||||
collectExtensionVariants(config, basic.value).fixDescriptors()
|
||||
}
|
||||
|
||||
return ReferenceVariantsCollectors(basic, extensions)
|
||||
}
|
||||
|
||||
private fun collectBasicVariants(filterConfiguration: FilterConfiguration): ReferenceVariants {
|
||||
val variants = doCollectBasicVariants(filterConfiguration)
|
||||
collectedImported += variants.imported
|
||||
|
||||
20
plugins/turboComplete/intellij.turboComplete.iml
Normal file
20
plugins/turboComplete/intellij.turboComplete.iml
Normal file
@@ -0,0 +1,20 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<module type="JAVA_MODULE" version="4">
|
||||
<component name="NewModuleRootManager" inherit-compiler-output="true">
|
||||
<exclude-output />
|
||||
<content url="file://$MODULE_DIR$">
|
||||
<sourceFolder url="file://$MODULE_DIR$/resources" type="java-resource" />
|
||||
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
|
||||
</content>
|
||||
<orderEntry type="inheritedJdk" />
|
||||
<orderEntry type="sourceFolder" forTests="false" />
|
||||
<orderEntry type="library" name="kotlin-stdlib" level="project" />
|
||||
<orderEntry type="module" module-name="intellij.platform.analysis" />
|
||||
<orderEntry type="module" module-name="intellij.platform.analysis.impl" />
|
||||
<orderEntry type="module" module-name="intellij.platform.lang" />
|
||||
<orderEntry type="module" module-name="intellij.platform.statistics" />
|
||||
<orderEntry type="module" module-name="intellij.completionMlRanking" />
|
||||
<orderEntry type="module" module-name="intellij.platform.lang.impl" />
|
||||
<orderEntry type="module" module-name="intellij.platform.ml.impl" />
|
||||
</component>
|
||||
</module>
|
||||
16
plugins/turboComplete/intellij.turboComplete.tests.iml
Normal file
16
plugins/turboComplete/intellij.turboComplete.tests.iml
Normal file
@@ -0,0 +1,16 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<module type="JAVA_MODULE" version="4">
|
||||
<component name="NewModuleRootManager" inherit-compiler-output="true">
|
||||
<exclude-output />
|
||||
<content url="file://$MODULE_DIR$/test">
|
||||
<sourceFolder url="file://$MODULE_DIR$/test" isTestSource="true" />
|
||||
</content>
|
||||
<orderEntry type="inheritedJdk" />
|
||||
<orderEntry type="sourceFolder" forTests="false" />
|
||||
<orderEntry type="library" scope="TEST" name="JUnit5" level="project" />
|
||||
<orderEntry type="module" module-name="intellij.turboComplete" scope="TEST" />
|
||||
<orderEntry type="library" scope="TEST" name="kotlin-stdlib" level="project" />
|
||||
<orderEntry type="module" module-name="intellij.platform.analysis" scope="TEST" />
|
||||
<orderEntry type="module" module-name="intellij.platform.ml.impl" scope="TEST" />
|
||||
</component>
|
||||
</module>
|
||||
@@ -0,0 +1,29 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<module type="JAVA_MODULE" version="4">
|
||||
<component name="NewModuleRootManager" inherit-compiler-output="true">
|
||||
<exclude-output />
|
||||
<content url="file://$MODULE_DIR$">
|
||||
<sourceFolder url="file://$MODULE_DIR$/resources" type="java-resource" />
|
||||
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
|
||||
</content>
|
||||
<orderEntry type="inheritedJdk" />
|
||||
<orderEntry type="sourceFolder" forTests="false" />
|
||||
<orderEntry type="library" name="kotlin-stdlib" level="project" />
|
||||
<orderEntry type="module" module-name="kotlin.completion.impl-k1" scope="PROVIDED" />
|
||||
<orderEntry type="module" module-name="intellij.platform.ml.impl" />
|
||||
<orderEntry type="module-library">
|
||||
<library name="completion-performance-kotlin" type="repository">
|
||||
<properties maven-id="org.jetbrains.intellij.deps.completion:performance-kotlin:0.0.6" />
|
||||
<CLASSES>
|
||||
<root url="jar://$MAVEN_REPOSITORY$/org/jetbrains/intellij/deps/completion/performance-kotlin/0.0.6/performance-kotlin-0.0.6.jar!/" />
|
||||
</CLASSES>
|
||||
<JAVADOC />
|
||||
<SOURCES>
|
||||
<root url="jar://$MAVEN_REPOSITORY$/org/jetbrains/intellij/deps/completion/performance-kotlin/0.0.6/performance-kotlin-0.0.6-sources.jar!/" />
|
||||
</SOURCES>
|
||||
</library>
|
||||
</orderEntry>
|
||||
<orderEntry type="module" module-name="intellij.turboComplete" />
|
||||
<orderEntry type="module" module-name="intellij.platform.ide.impl" />
|
||||
</component>
|
||||
</module>
|
||||
@@ -0,0 +1,13 @@
|
||||
<idea-plugin package="com.intellij.turboComplete.languages.kotlin">
|
||||
<dependencies>
|
||||
<plugin id="org.jetbrains.kotlin"/>
|
||||
</dependencies>
|
||||
|
||||
<extensions defaultExtensionNs="com.intellij.turboComplete">
|
||||
<kindCollector implementation="org.jetbrains.kotlin.idea.completion.KotlinKindCollector"/>
|
||||
<suggestionGeneratorExecutorProvider implementation="com.intellij.turboComplete.languages.kotlin.MLKotlinSuggestionGeneratorExecutorProvider"/>
|
||||
</extensions>
|
||||
<extensions defaultExtensionNs="com.intellij">
|
||||
<registryKey key="ml.completion.performance.localModel.kotlin" defaultValue="no path" description="Use local model for kotlin completion kind ordering"/>
|
||||
</extensions>
|
||||
</idea-plugin>
|
||||
@@ -0,0 +1,2 @@
|
||||
completion.kind.sorter.ml.kotlin=Kotlin
|
||||
presentation.tail.for.0.in.1="\ for {0} in {1}"
|
||||
@@ -0,0 +1,37 @@
|
||||
package com.intellij.turboComplete.languages.kotlin
|
||||
|
||||
import com.intellij.internal.ml.catboost.CatBoostJarCompletionModelProvider
|
||||
import com.intellij.lang.Language
|
||||
import com.intellij.turboComplete.SortingExecutorPreferences
|
||||
import com.intellij.turboComplete.SortingExecutorProvider
|
||||
import com.intellij.turboComplete.ranking.MLKindSorterProvider
|
||||
import com.intellij.turboComplete.ranking.provideLocalIfAny
|
||||
import org.jetbrains.kotlin.idea.completion.KotlinKindVariety
|
||||
|
||||
class MLKotlinSuggestionGeneratorExecutorProvider : SortingExecutorProvider(
|
||||
SortingExecutorPreferences(
|
||||
policyForMostRelevant = SortingExecutorPreferences.MostRelevantKindPolicy.PASS_TO_RESULT,
|
||||
policyForNoneKind = SortingExecutorPreferences.NoneKindPolicy.PASS_TO_RESULT,
|
||||
executeMostRelevantWhenPassed = 3
|
||||
)
|
||||
) {
|
||||
override val sorterProvider = run {
|
||||
val mlSorterProvider = MLKindSorterProvider(KotlinKindVariety) { MlKotlinMultipleReferencesSorterProvider().model }
|
||||
val maybeLocalSorterProvider = mlSorterProvider
|
||||
.provideLocalIfAny("kotlin", "ml.completion.performance.localModel.kotlin")
|
||||
|
||||
maybeLocalSorterProvider
|
||||
}
|
||||
}
|
||||
|
||||
class MlKotlinMultipleReferencesSorterProvider : CatBoostJarCompletionModelProvider(
|
||||
TurboCompleteKotlinBundle.message(
|
||||
"completion.kind.sorter.ml.kotlin"),
|
||||
"performance_kotlin_features", "performance_kotlin_model") {
|
||||
|
||||
override fun isLanguageSupported(language: Language): Boolean = language.id.compareTo("kotlin", ignoreCase = true) == 0
|
||||
|
||||
override fun isEnabledByDefault(): Boolean {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package com.intellij.turboComplete.languages.kotlin
|
||||
|
||||
import com.intellij.AbstractBundle
|
||||
import com.intellij.DynamicBundle
|
||||
import org.jetbrains.annotations.Nls
|
||||
import org.jetbrains.annotations.PropertyKey
|
||||
|
||||
class TurboCompleteKotlinBundle : DynamicBundle(TURBO_COMPLETE_JAVA_BUNDLE) {
|
||||
companion object {
|
||||
private val ourInstance: AbstractBundle = TurboCompleteKotlinBundle()
|
||||
|
||||
private const val TURBO_COMPLETE_JAVA_BUNDLE = "messages.TurboCompleteKotlin"
|
||||
|
||||
@Nls
|
||||
@JvmStatic
|
||||
fun message(key: @PropertyKey(resourceBundle = TURBO_COMPLETE_JAVA_BUNDLE) String, vararg params: Any): String {
|
||||
return ourInstance.getMessage(key, *params)
|
||||
}
|
||||
}
|
||||
}
|
||||
83
plugins/turboComplete/resources/META-INF/plugin.xml
Normal file
83
plugins/turboComplete/resources/META-INF/plugin.xml
Normal file
@@ -0,0 +1,83 @@
|
||||
<idea-plugin>
|
||||
<id>com.intellij.turboComplete</id>
|
||||
<name>Turbo Complete</name>
|
||||
<vendor>JetBrains</vendor>
|
||||
<category>Other Tools</category>
|
||||
|
||||
<description>
|
||||
<![CDATA[
|
||||
Experimental approach to speed up code completion.
|
||||
<br><br>
|
||||
The plugin implements a <a href="https://youtrack.jetbrains.com/issue/MLP-17/ML-for-IDE-Performance">new approach</a>
|
||||
to improve code completion's performance perception.
|
||||
Currently, it works only for Kotlin.
|
||||
If you are experiencing problems with code completion, you could disable
|
||||
the plugin.
|
||||
]]>
|
||||
</description>
|
||||
|
||||
<resource-bundle>messages.TurboComplete</resource-bundle>
|
||||
|
||||
<extensionPoints>
|
||||
<extensionPoint
|
||||
interface="com.intellij.platform.ml.impl.turboComplete.KindCollector"
|
||||
dynamic="true"
|
||||
name="kindCollector"/>
|
||||
<extensionPoint
|
||||
interface="com.intellij.turboComplete.analysis.PipelineListener"
|
||||
dynamic="true"
|
||||
name="analysis.pipelineListener"/>
|
||||
<extensionPoint
|
||||
interface="com.intellij.turboComplete.SuggestionGeneratorExecutorProvider"
|
||||
dynamic="true"
|
||||
name="suggestionGeneratorExecutorProvider"/>
|
||||
<extensionPoint
|
||||
interface="com.intellij.turboComplete.features.kind.KindFeatureProvider"
|
||||
dynamic="true"
|
||||
name="features.kind.provider"/>
|
||||
</extensionPoints>
|
||||
|
||||
<extensions defaultExtensionNs="com.intellij">
|
||||
<completion.contributor language="any"
|
||||
id="kind_ordering_factors"
|
||||
implementationClass="com.intellij.turboComplete.platform.contributor.KindOrderingFeaturesContributor"
|
||||
order="first, after liveTemplates"/>
|
||||
<completion.contributor language="any"
|
||||
id="kind_executor"
|
||||
implementationClass="com.intellij.turboComplete.platform.contributor.KindExecutingCompletionContributor"
|
||||
order="first, after kind_ordering_factors, before duplicate_remover"/>
|
||||
<completion.contributor language="any"
|
||||
id="duplicate_remover"
|
||||
implementationClass="com.intellij.turboComplete.platform.contributor.FilteringCompletionContributor"
|
||||
order="first"/>
|
||||
|
||||
<completion.ml.contextFeatures language="" implementationClass="com.intellij.turboComplete.features.context.AllKindsUsageFeatures"/>
|
||||
<completion.ml.contextFeatures language=""
|
||||
implementationClass="com.intellij.turboComplete.features.context.CompletionPerformanceStatusFeatures"/>
|
||||
|
||||
<completion.ml.elementFeatures language="" implementationClass="com.intellij.turboComplete.features.element.ElementsKindFeatures"/>
|
||||
|
||||
<turboComplete.features.kind.provider implementation="com.intellij.turboComplete.features.kind.KindCommonFeatures"/>
|
||||
<turboComplete.features.kind.provider implementation="com.intellij.turboComplete.features.kind.KindUsageFeatures"/>
|
||||
|
||||
<turboComplete.analysis.pipelineListener implementation="com.intellij.turboComplete.analysis.KindPerformanceRecorder"/>
|
||||
<turboComplete.analysis.pipelineListener implementation="com.intellij.turboComplete.analysis.PipelinePerformanceRecorder"/>
|
||||
<turboComplete.analysis.pipelineListener
|
||||
implementation="com.intellij.turboComplete.analysis.usage.KindVarietyUsageTracker$UsagePipelineListener"/>
|
||||
<turboComplete.analysis.pipelineListener implementation="com.intellij.turboComplete.platform.EarlyLookupOpener"/>
|
||||
|
||||
<registryKey key="ml.completion.performance.enable" defaultValue="true" description="Enable completion performance"/>
|
||||
<registryKey key="ml.completion.performance.showLookupEarly" defaultValue="true"
|
||||
description="Show lookup as soon as the first CompletionKind finished working"/>
|
||||
<registryKey key="ml.completion.performance.experiment" defaultValue="true"
|
||||
description="Perform an A/B experiment turning on and off performance"/>
|
||||
<registryKey key="ml.completion.performance.executeImmediately" defaultValue="false"
|
||||
description="Immediately execute suggestion generators"/>
|
||||
</extensions>
|
||||
|
||||
<content>
|
||||
<module name="intellij.turboComplete.languages.kotlin"/>
|
||||
</content>
|
||||
|
||||
<depends>com.intellij.completion.ml.ranking</depends>
|
||||
</idea-plugin>
|
||||
@@ -0,0 +1,82 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.completion.ml.common.CurrentProjectInfo
|
||||
import com.intellij.completion.ml.experiment.ExperimentStatus
|
||||
import com.intellij.openapi.util.registry.Registry
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindCollector
|
||||
|
||||
data class CompletionPerformanceParameters(
|
||||
val enabled: Boolean,
|
||||
val showLookupEarly: Boolean,
|
||||
val fixedGeneratorsOrder: Boolean,
|
||||
) {
|
||||
companion object {
|
||||
private fun parametrizeNoPerformance() = CompletionPerformanceParameters(false, false, false)
|
||||
|
||||
private fun parametrizeForUnitTesting() = CompletionPerformanceParameters(true, false, false)
|
||||
|
||||
private val isTeamCity = System.getenv("TEAMCITY_VERSION") != null
|
||||
|
||||
private enum class Experiment(val number: Int) {
|
||||
PERFORMANCE_LOOKUP_RANKING(17),
|
||||
PERFORMANCE_LOOKUP(18),
|
||||
ONLY_PERFORMANCE(19);
|
||||
|
||||
companion object {
|
||||
fun numbers() = Experiment.values().map { it.number }
|
||||
|
||||
fun fromVersion(version: Int) = Experiment.values().find { it.number == version }
|
||||
}
|
||||
}
|
||||
|
||||
private fun registryParameters(): CompletionPerformanceParameters? {
|
||||
if (Registry.`is`("ml.completion.performance.experiment")) return null
|
||||
|
||||
val enablePerformance = Registry.`is`("ml.completion.performance.enable")
|
||||
val showLookupEarly = Registry.`is`("ml.completion.performance.showLookupEarly")
|
||||
val executeImmediately = Registry.`is`("ml.completion.performance.executeImmediately")
|
||||
|
||||
return CompletionPerformanceParameters(
|
||||
enablePerformance,
|
||||
enablePerformance && showLookupEarly,
|
||||
executeImmediately,
|
||||
)
|
||||
}
|
||||
|
||||
private fun experimentParameters(parameters: CompletionParameters): CompletionPerformanceParameters {
|
||||
if (!CurrentProjectInfo.getInstance(parameters.position.project).isIdeaProject) {
|
||||
return parametrizeNoPerformance()
|
||||
}
|
||||
|
||||
val status = ExperimentStatus.getInstance().forLanguage(parameters.position.language)
|
||||
if (!status.inExperiment || status.version !in Experiment.numbers()) {
|
||||
return parametrizeNoPerformance()
|
||||
}
|
||||
|
||||
return when (Experiment.fromVersion(status.version)) {
|
||||
Experiment.PERFORMANCE_LOOKUP_RANKING, Experiment.PERFORMANCE_LOOKUP -> CompletionPerformanceParameters(true, true, false)
|
||||
Experiment.ONLY_PERFORMANCE -> CompletionPerformanceParameters(true, false, false)
|
||||
else -> parametrizeNoPerformance()
|
||||
}
|
||||
}
|
||||
|
||||
private fun canEnablePerformance(parameters: CompletionParameters): Boolean {
|
||||
val hasKindExecutorProvider = { SuggestionGeneratorExecutorProvider.hasAnyToCall(parameters) }
|
||||
val hasAtLeastOneGenerator = { KindCollector.forParameters(parameters).any() }
|
||||
return hasKindExecutorProvider() && hasAtLeastOneGenerator()
|
||||
}
|
||||
|
||||
fun fromCompletionPreferences(parameters: CompletionParameters): CompletionPerformanceParameters {
|
||||
if (isTeamCity || !canEnablePerformance(parameters)) {
|
||||
return parametrizeNoPerformance()
|
||||
}
|
||||
|
||||
if (parameters.isTestingMode) {
|
||||
return parametrizeForUnitTesting()
|
||||
}
|
||||
|
||||
return registryParameters() ?: experimentParameters(parameters)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindExecutionListener
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
|
||||
interface DelegatingKindExecutionListener<T : KindExecutionListener> : KindExecutionListener {
|
||||
val delegatedListeners: MutableList<T>
|
||||
|
||||
override fun onInitialize(parameters: CompletionParameters) = delegatedListeners.forEach { it.onInitialize(parameters) }
|
||||
|
||||
override fun onCollectionStarted() = delegatedListeners.forEach { it.onCollectionStarted() }
|
||||
|
||||
override fun onGeneratorCollected(suggestionGenerator: SuggestionGenerator) = delegatedListeners.forEach {
|
||||
it.onGeneratorCollected(suggestionGenerator)
|
||||
}
|
||||
|
||||
override fun onGenerationStarted(suggestionGenerator: SuggestionGenerator) = delegatedListeners.forEach {
|
||||
it.onGenerationStarted(suggestionGenerator)
|
||||
}
|
||||
|
||||
override fun onGenerationFinished(suggestionGenerator: SuggestionGenerator) = delegatedListeners.forEach {
|
||||
it.onGenerationFinished(suggestionGenerator)
|
||||
}
|
||||
|
||||
override fun onCollectionFinished() = delegatedListeners.forEach { it.onCollectionFinished() }
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.addingPolicy.PassDirectlyPolicy
|
||||
|
||||
class ImmediateExecutor(override val parameters: CompletionParameters,
|
||||
override val policyController: PolicyController) : SuggestionGeneratorExecutor {
|
||||
override fun createNoneKindPolicy() = PassDirectlyPolicy()
|
||||
|
||||
override fun executeAll() {}
|
||||
|
||||
override fun pass(suggestionGenerator: SuggestionGenerator) {
|
||||
suggestionGenerator.generateCompletionVariants()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
|
||||
enum class NullableKindName {
|
||||
NONE_KIND
|
||||
}
|
||||
|
||||
class NullableKindVariety(private val baseKindVariety: KindVariety) : KindVariety by baseKindVariety {
|
||||
|
||||
companion object {
|
||||
fun KindVariety.withNullableKind() = NullableKindVariety(this)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindCollector
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindExecutionListener
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGeneratorConsumer
|
||||
import com.intellij.turboComplete.analysis.PipelineListener
|
||||
|
||||
private class GenerationReportingSuggestionGenerator(
|
||||
private val suggestionGenerator: SuggestionGenerator,
|
||||
private val listener: KindExecutionListener,
|
||||
) : SuggestionGenerator by suggestionGenerator {
|
||||
override fun generateCompletionVariants() {
|
||||
listener.onGenerationStarted(suggestionGenerator)
|
||||
try {
|
||||
suggestionGenerator.generateCompletionVariants()
|
||||
}
|
||||
finally {
|
||||
listener.onGenerationFinished(suggestionGenerator)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
class ReportingSuggestionGeneratorExecutor(
|
||||
private val baseExecutor: SuggestionGeneratorExecutor,
|
||||
private val listener: KindExecutionListener,
|
||||
) : SuggestionGeneratorExecutor by baseExecutor {
|
||||
fun reportGenerateCompletionKinds(generator: KindCollector,
|
||||
parameters: CompletionParameters,
|
||||
completionKindsConsumer: SuggestionGeneratorConsumer,
|
||||
result: CompletionResultSet,
|
||||
resultPolicyController: PolicyController) {
|
||||
listener.onCollectionStarted()
|
||||
try {
|
||||
generator.collectKinds(parameters, completionKindsConsumer, result, resultPolicyController)
|
||||
}
|
||||
finally {
|
||||
listener.onCollectionFinished()
|
||||
}
|
||||
}
|
||||
|
||||
override fun pass(suggestionGenerator: SuggestionGenerator) {
|
||||
val reportingCompletionKind = GenerationReportingSuggestionGenerator(suggestionGenerator, listener)
|
||||
listener.onGeneratorCollected(suggestionGenerator)
|
||||
baseExecutor.pass(reportingCompletionKind)
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun initialize(
|
||||
parameters: CompletionParameters,
|
||||
policyController: PolicyController,
|
||||
executorProvider: SuggestionGeneratorExecutorProvider,
|
||||
): ReportingSuggestionGeneratorExecutor {
|
||||
val listener = object : DelegatingKindExecutionListener<KindExecutionListener> {
|
||||
override val delegatedListeners: MutableList<KindExecutionListener> = PipelineListener.EP_NAME.extensionList.toMutableList()
|
||||
}
|
||||
listener.onInitialize(parameters)
|
||||
|
||||
val performanceParameters = CompletionPerformanceParameters.fromCompletionPreferences(parameters)
|
||||
|
||||
val generatorExecutor = if (performanceParameters.fixedGeneratorsOrder)
|
||||
ImmediateExecutor(parameters, policyController)
|
||||
else
|
||||
executorProvider.createExecutor(parameters, policyController)
|
||||
|
||||
return ReportingSuggestionGeneratorExecutor(generatorExecutor, listener)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.openapi.components.service
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.addingPolicy.PassDirectlyPolicy
|
||||
import com.intellij.turboComplete.analysis.usage.KindVarietyUsageTracker
|
||||
import com.intellij.turboComplete.platform.addingPolicy.BufferingPolicy
|
||||
import com.intellij.turboComplete.platform.addingPolicy.ConvertToCompletionKindPolicy
|
||||
import com.intellij.turboComplete.platform.addingPolicy.addingActualContributor
|
||||
import com.intellij.turboComplete.platform.addingPolicy.addingCompletionKind
|
||||
import com.intellij.turboComplete.ranking.KindRelevanceSorter
|
||||
|
||||
class SortingExecutor(
|
||||
private val sorter: KindRelevanceSorter,
|
||||
override val parameters: CompletionParameters,
|
||||
override val policyController: PolicyController,
|
||||
private val executionPreferences: SortingExecutorPreferences
|
||||
) : SuggestionGeneratorExecutor {
|
||||
private val executedGenerators: MutableSet<SuggestionGenerator> = mutableSetOf()
|
||||
private val nonExecutedGenerators: MutableSet<SuggestionGenerator> = mutableSetOf()
|
||||
private val mostRelevantKinds: MutableList<CompletionKind> = run {
|
||||
val usageTracker = service<KindVarietyUsageTracker>()
|
||||
val onceGeneratedKinds = usageTracker.trackedKinds(sorter.kindVariety)
|
||||
sorter.sort(onceGeneratedKinds, parameters).map { it.kind }
|
||||
.take(executionPreferences.executeMostRelevantWhenPassed)
|
||||
.toMutableList()
|
||||
}
|
||||
|
||||
override fun createNoneKindPolicy() = when (executionPreferences.policyForNoneKind) {
|
||||
SortingExecutorPreferences.NoneKindPolicy.BUFFER -> BufferingPolicy()
|
||||
|
||||
SortingExecutorPreferences.NoneKindPolicy.PASS_TO_RESULT -> PassDirectlyPolicy()
|
||||
|
||||
SortingExecutorPreferences.NoneKindPolicy.CREATE_NONE_KIND -> ConvertToCompletionKindPolicy(
|
||||
this,
|
||||
CompletionKind(NullableKindName.NONE_KIND, sorter.kindVariety),
|
||||
parameters
|
||||
)
|
||||
}
|
||||
|
||||
private fun makeMostRelevantKindPolicy() = when (executionPreferences.policyForMostRelevant) {
|
||||
SortingExecutorPreferences.MostRelevantKindPolicy.BUFFER -> BufferingPolicy()
|
||||
SortingExecutorPreferences.MostRelevantKindPolicy.PASS_TO_RESULT -> PassDirectlyPolicy()
|
||||
}
|
||||
|
||||
private fun makeKindPolicy(suggestionGenerator: SuggestionGenerator): ElementsAddingPolicy {
|
||||
val plainPolicy =
|
||||
if (executedGenerators.isEmpty()) makeMostRelevantKindPolicy()
|
||||
else PassDirectlyPolicy()
|
||||
|
||||
return plainPolicy
|
||||
.addingActualContributor(suggestionGenerator)
|
||||
.addingCompletionKind(suggestionGenerator)
|
||||
}
|
||||
|
||||
private fun invokeCompletionKind(suggestionGenerator: SuggestionGenerator) {
|
||||
val policy = makeKindPolicy(suggestionGenerator)
|
||||
policyController.invokeWithPolicy(policy) {
|
||||
suggestionGenerator.generateCompletionVariants()
|
||||
}
|
||||
executedGenerators.add(suggestionGenerator)
|
||||
nonExecutedGenerators.remove(suggestionGenerator)
|
||||
}
|
||||
|
||||
override fun executeAll() {
|
||||
if (nonExecutedGenerators.isEmpty()) return
|
||||
val executionOrder = sorter.sortGenerators(nonExecutedGenerators.toList(), parameters)
|
||||
executionOrder.forEach {
|
||||
invokeCompletionKind(it)
|
||||
}
|
||||
}
|
||||
|
||||
override fun pass(suggestionGenerator: SuggestionGenerator) {
|
||||
nonExecutedGenerators.add(suggestionGenerator)
|
||||
if (mostRelevantKinds.getOrNull(0) == suggestionGenerator.kind) {
|
||||
invokeCompletionKind(suggestionGenerator)
|
||||
mostRelevantKinds.removeFirst()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
data class SortingExecutorPreferences(
|
||||
val policyForMostRelevant: MostRelevantKindPolicy,
|
||||
val policyForNoneKind: NoneKindPolicy,
|
||||
val executeMostRelevantWhenPassed: Int,
|
||||
) {
|
||||
enum class MostRelevantKindPolicy {
|
||||
BUFFER,
|
||||
PASS_TO_RESULT,
|
||||
}
|
||||
|
||||
enum class NoneKindPolicy {
|
||||
BUFFER,
|
||||
PASS_TO_RESULT,
|
||||
CREATE_NONE_KIND,
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.turboComplete.analysis.DelegatingPipelineListener
|
||||
import com.intellij.turboComplete.analysis.PipelineListener
|
||||
import com.intellij.turboComplete.ranking.KindRelevanceSorter
|
||||
import com.intellij.turboComplete.ranking.KindSorterProvider
|
||||
import com.intellij.turboComplete.ranking.ReportingKindSorter
|
||||
|
||||
abstract class SortingExecutorProvider(
|
||||
private val executionPreferences: SortingExecutorPreferences
|
||||
) : SuggestionGeneratorExecutorProvider {
|
||||
abstract val sorterProvider: KindSorterProvider
|
||||
override fun shouldBeCalled(parameters: CompletionParameters): Boolean {
|
||||
return sorterProvider.kindVariety.kindsCorrespondToParameters(parameters)
|
||||
}
|
||||
|
||||
private fun createSorter(): KindRelevanceSorter {
|
||||
val rankingListeners: MutableList<PipelineListener> = PipelineListener.EP_NAME.extensionList.toMutableList()
|
||||
val delegatingListener = object : DelegatingPipelineListener {
|
||||
override val delegatedListeners: MutableList<PipelineListener>
|
||||
get() = rankingListeners
|
||||
}
|
||||
return ReportingKindSorter(delegatingListener, sorterProvider)
|
||||
}
|
||||
|
||||
override fun createExecutor(
|
||||
parameters: CompletionParameters,
|
||||
policyController: PolicyController,
|
||||
): SuggestionGeneratorExecutor {
|
||||
return SortingExecutor(createSorter(), parameters, policyController, executionPreferences)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGeneratorConsumer
|
||||
|
||||
interface SuggestionGeneratorExecutor : SuggestionGeneratorConsumer {
|
||||
val parameters: CompletionParameters
|
||||
val policyController: PolicyController
|
||||
|
||||
fun createNoneKindPolicy(): ElementsAddingPolicy
|
||||
|
||||
fun executeAll()
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package com.intellij.turboComplete
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.openapi.extensions.ExtensionPointName
|
||||
|
||||
interface SuggestionGeneratorExecutorProvider {
|
||||
fun shouldBeCalled(parameters: CompletionParameters): Boolean
|
||||
|
||||
fun createExecutor(
|
||||
parameters: CompletionParameters,
|
||||
policyController: PolicyController,
|
||||
): SuggestionGeneratorExecutor
|
||||
|
||||
companion object {
|
||||
private val EP_NAME: ExtensionPointName<SuggestionGeneratorExecutorProvider> =
|
||||
ExtensionPointName("com.intellij.turboComplete.suggestionGeneratorExecutorProvider")
|
||||
|
||||
fun hasAnyToCall(parameters: CompletionParameters): Boolean {
|
||||
return EP_NAME.extensionList.any { it.shouldBeCalled(parameters) }
|
||||
}
|
||||
|
||||
fun findOneMatching(parameters: CompletionParameters): SuggestionGeneratorExecutorProvider {
|
||||
val allExecutorProviders = EP_NAME.extensionList.filter { it.shouldBeCalled(parameters) }
|
||||
if (allExecutorProviders.size > 1) {
|
||||
throw IllegalStateException(
|
||||
"Found more than one matching CompletionKindExecutorProvider: ${allExecutorProviders.map { it.javaClass.name }}")
|
||||
}
|
||||
return allExecutorProviders[0]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.turboComplete.DelegatingKindExecutionListener
|
||||
import com.intellij.turboComplete.ranking.DelegatingKindRankingListener
|
||||
|
||||
interface DelegatingPipelineListener
|
||||
: PipelineListener,
|
||||
DelegatingKindRankingListener<PipelineListener>,
|
||||
DelegatingKindExecutionListener<PipelineListener>
|
||||
@@ -0,0 +1,18 @@
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.completion.ml.performance.PerformanceTracker
|
||||
|
||||
class KindPerformanceRecorder : PerformanceRecorder<SingleLifetimeKindsRecorder>() {
|
||||
override val performanceMetricName = "turboComplete.kindPerf"
|
||||
|
||||
override fun createPipelineListener() = SingleLifetimeKindsRecorder()
|
||||
|
||||
override fun capturePerformance(pipelineListener: SingleLifetimeKindsRecorder, tracker: PerformanceTracker) {
|
||||
val completionKindLifetimes = pipelineListener.captureCompletionKindLifetimeDurations()
|
||||
completionKindLifetimes.entries.forEach { (completionKind, performance) ->
|
||||
tracker.addByKey("init.${completionKind.name}", performance.created)
|
||||
performance.executed?.let { tracker.addByKey("run.${completionKind.name}", it) }
|
||||
performance.executedAsFirst?.let { tracker.addByKey("runFirst.${completionKind.name}", it) }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.codeInsight.lookup.impl.LookupManagerImpl
|
||||
import com.intellij.completion.ml.performance.PerformanceMetricCollector
|
||||
import com.intellij.completion.ml.performance.PerformanceMetricCollectorFactory
|
||||
import com.intellij.completion.ml.performance.PerformanceTracker
|
||||
import com.intellij.completion.ml.storage.MutableLookupStorage
|
||||
|
||||
abstract class PerformanceRecorder<T : PipelineListener>
|
||||
: DelegatingPipelineListener,
|
||||
PerformanceMetricCollectorFactory {
|
||||
|
||||
private lateinit var currentListener: T
|
||||
|
||||
abstract fun createPipelineListener(): T
|
||||
|
||||
abstract fun capturePerformance(pipelineListener: T, tracker: PerformanceTracker)
|
||||
|
||||
override val delegatedListeners: MutableList<PipelineListener>
|
||||
get() = mutableListOf(currentListener)
|
||||
|
||||
override fun onInitialize(parameters: CompletionParameters) {
|
||||
val lookup = LookupManagerImpl.getActiveLookup(parameters.editor) as? LookupImpl ?: return
|
||||
val lookupStorage = MutableLookupStorage.get(lookup)!!
|
||||
lookupStorage.performanceTracker.addMetricCollector(this)
|
||||
currentListener = createPipelineListener()
|
||||
super.onInitialize(parameters)
|
||||
}
|
||||
|
||||
override fun createMetricCollector(tracker: PerformanceTracker): PerformanceMetricCollector {
|
||||
return CompletionKindMetricsCollector(tracker)
|
||||
}
|
||||
|
||||
inner class CompletionKindMetricsCollector(private val tracker: PerformanceTracker) : PerformanceMetricCollector {
|
||||
override fun onFinishCollecting() {
|
||||
capturePerformance(currentListener, tracker)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.openapi.extensions.ExtensionPointName
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindExecutionListener
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.KindRankingListener
|
||||
|
||||
interface PipelineListener : KindExecutionListener, KindRankingListener {
|
||||
companion object {
|
||||
val EP_NAME: ExtensionPointName<PipelineListener> = ExtensionPointName.create(
|
||||
"com.intellij.turboComplete.analysis.pipelineListener")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.completion.ml.performance.PerformanceTracker
|
||||
|
||||
class PipelinePerformanceRecorder : PerformanceRecorder<SinglePipelineRecorder>() {
|
||||
override val performanceMetricName = "turboComplete.pipeline"
|
||||
|
||||
override fun createPipelineListener() = SinglePipelineRecorder()
|
||||
|
||||
override fun capturePerformance(pipelineListener: SinglePipelineRecorder, tracker: PerformanceTracker) {
|
||||
val chronologyDurations = pipelineListener.captureChronologyDurations()
|
||||
tracker.addByKey("gen", chronologyDurations.generation)
|
||||
tracker.addByKey("rank", chronologyDurations.ranking)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.RankedKind
|
||||
import kotlin.properties.Delegates
|
||||
|
||||
class SingleLifetimeKindsRecorder : PipelineListener {
|
||||
private var collectionStarted by Delegates.notNull<Long>()
|
||||
private var generatorCollected: MutableMap<CompletionKind, Long?> = mutableMapOf()
|
||||
private var kindRanked: MutableMap<CompletionKind, Long?> = mutableMapOf()
|
||||
private var generationStarted: MutableMap<CompletionKind, Long?> = mutableMapOf()
|
||||
private var generationFinished: MutableMap<CompletionKind, Long?> = mutableMapOf()
|
||||
private var firstExecutedKind: CompletionKind? = null
|
||||
|
||||
override fun onInitialize(parameters: CompletionParameters) {}
|
||||
|
||||
override fun onCollectionStarted() {
|
||||
collectionStarted = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onGeneratorCollected(suggestionGenerator: SuggestionGenerator) {
|
||||
generatorCollected[suggestionGenerator.kind] = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onRanked(ranked: List<RankedKind>) {
|
||||
val time = System.currentTimeMillis()
|
||||
ranked.forEach { kindRanked[it.kind] = time }
|
||||
}
|
||||
|
||||
override fun onGenerationStarted(suggestionGenerator: SuggestionGenerator) {
|
||||
generationStarted[suggestionGenerator.kind] = System.currentTimeMillis()
|
||||
if (firstExecutedKind == null) {
|
||||
firstExecutedKind = suggestionGenerator.kind
|
||||
}
|
||||
}
|
||||
|
||||
override fun onGenerationFinished(suggestionGenerator: SuggestionGenerator) {
|
||||
generationFinished[suggestionGenerator.kind] = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
fun captureCompletionKindLifetimeDurations(): Map<CompletionKind, GeneratorLifetimeDurations> {
|
||||
fun durationBetween(start: Long?, finish: Long?): Long? {
|
||||
return finish?.let { notNullFinish -> start?.let { notNullStart -> notNullFinish - notNullStart } }
|
||||
}
|
||||
return generatorCollected.keys.associateWith {
|
||||
GeneratorLifetimeDurations(
|
||||
created = durationBetween(collectionStarted, generatorCollected[it])!!,
|
||||
executed = durationBetween(generationStarted[it], generationFinished[it]),
|
||||
executedAsFirst = if (it == firstExecutedKind)
|
||||
durationBetween(generationStarted[it], generationFinished[it])
|
||||
else
|
||||
null
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
data class GeneratorLifetimeDurations(
|
||||
val created: Long,
|
||||
val executed: Long?,
|
||||
val executedAsFirst: Long?
|
||||
)
|
||||
@@ -0,0 +1,60 @@
|
||||
package com.intellij.turboComplete.analysis
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import kotlin.properties.Delegates
|
||||
|
||||
class SinglePipelineRecorder : PipelineListener {
|
||||
private var collectionStarted by Delegates.notNull<Long>()
|
||||
private var collectionFinished by Delegates.notNull<Long>()
|
||||
private val generatorCollected: MutableMap<CompletionKind, Long> = mutableMapOf()
|
||||
private val generatorStarted: MutableMap<CompletionKind, Long> = mutableMapOf()
|
||||
private val generatorFinished: MutableMap<CompletionKind, Long> = mutableMapOf()
|
||||
private var rankingDurations: MutableList<Long> = mutableListOf()
|
||||
private var rankingStarted: Long? = null
|
||||
|
||||
override fun onInitialize(parameters: CompletionParameters) {
|
||||
}
|
||||
|
||||
override fun onCollectionStarted() {
|
||||
collectionStarted = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onGeneratorCollected(suggestionGenerator: SuggestionGenerator) {
|
||||
generatorCollected[suggestionGenerator.kind] = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onCollectionFinished() {
|
||||
collectionFinished = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onGenerationStarted(suggestionGenerator: SuggestionGenerator) {
|
||||
generatorStarted[suggestionGenerator.kind] = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onGenerationFinished(suggestionGenerator: SuggestionGenerator) {
|
||||
generatorFinished[suggestionGenerator.kind] = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
fun captureChronologyDurations(): PipelineChronologyDurations {
|
||||
return PipelineChronologyDurations(
|
||||
generation = collectionFinished - collectionStarted,
|
||||
ranking = rankingDurations.sum(),
|
||||
)
|
||||
}
|
||||
|
||||
override fun onRankingStarted() {
|
||||
rankingStarted = System.currentTimeMillis()
|
||||
}
|
||||
|
||||
override fun onRankingFinished() {
|
||||
rankingDurations.add(System.currentTimeMillis() - rankingStarted!!)
|
||||
rankingStarted = null
|
||||
}
|
||||
|
||||
data class PipelineChronologyDurations(
|
||||
val generation: Long,
|
||||
val ranking: Long,
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package com.intellij.turboComplete.analysis.usage
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
|
||||
data class CombinedKindUsageStatistics(
|
||||
val created: Int,
|
||||
val generated: KindStatistics,
|
||||
val generatedInRow: KindStatistics,
|
||||
val recentGenerated: ValuePerPeriod<KindStatistics>,
|
||||
) {
|
||||
val recentProbGenerateCorrect: ValuePerPeriod<Double>
|
||||
get() {
|
||||
return if (recentGenerated.period != 0)
|
||||
ValuePerPeriod(
|
||||
recentGenerated.period,
|
||||
recentGenerated.value.correct.toDouble() / recentGenerated.period
|
||||
)
|
||||
else {
|
||||
ValuePerPeriod(0, 0.0)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
class CombinedKindUsageTracker(override val kind: CompletionKind,
|
||||
recentWindowSize: Int,
|
||||
maxUsageWindowSize: Int) : KindUsageTracker<CombinedKindUsageStatistics> {
|
||||
private val createdTracker = KindCreatedTracker(kind)
|
||||
private val generatedTracker = KindGenerationTracker(kind)
|
||||
private val generationInRawTracker = KindGenerationInRawTracker(kind, maxUsageWindowSize)
|
||||
private val recentGeneratedTracker = KindRecentUsageTracker(kind, recentWindowSize) {
|
||||
KindGenerationTracker(it)
|
||||
}
|
||||
private val allTrackers = listOf(
|
||||
createdTracker,
|
||||
generatedTracker,
|
||||
generationInRawTracker,
|
||||
recentGeneratedTracker,
|
||||
)
|
||||
|
||||
override fun trackCreated() = allTrackers.forEach { it.trackCreated() }
|
||||
|
||||
override fun trackGenerated(correct: Boolean) = allTrackers.forEach { it.trackGenerated(correct) }
|
||||
|
||||
override fun getSummary(): CombinedKindUsageStatistics {
|
||||
return CombinedKindUsageStatistics(
|
||||
created = createdTracker.getSummary(),
|
||||
generated = generatedTracker.getSummary(),
|
||||
generatedInRow = generationInRawTracker.getSummary(),
|
||||
recentGenerated = recentGeneratedTracker.getSummary(),
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package com.intellij.turboComplete.analysis.usage
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import java.util.*
|
||||
|
||||
data class ValuePerPeriod<T>(
|
||||
val period: Int,
|
||||
val value: T,
|
||||
)
|
||||
|
||||
private data class RecentUsage<T>(
|
||||
val isCreation: Boolean,
|
||||
val action: (KindUsageTracker<T>) -> Unit,
|
||||
)
|
||||
|
||||
class KindRecentUsageTracker<T>(override val kind: CompletionKind,
|
||||
private val windowSize: Int,
|
||||
val baseTrackerProvider: (CompletionKind) -> KindUsageTracker<T>
|
||||
) : KindUsageTracker<ValuePerPeriod<T>> {
|
||||
private val recentUsages: Queue<RecentUsage<T>> = LinkedList()
|
||||
private var recentCreations: Int = 0
|
||||
private var recentUsagesStatistics: T? = null
|
||||
|
||||
private fun popCreationAndFollowingActions() {
|
||||
assert(recentUsages.peek().isCreation)
|
||||
assert(recentUsages.remove().isCreation)
|
||||
recentCreations -= 1
|
||||
while (recentUsages.isNotEmpty() && !recentUsages.peek().isCreation) {
|
||||
recentUsages.remove()
|
||||
}
|
||||
}
|
||||
|
||||
private fun trackRecentUsage(action: RecentUsage<T>) {
|
||||
recentUsagesStatistics = null
|
||||
recentUsages.add(action)
|
||||
if (action.isCreation) {
|
||||
recentCreations += 1
|
||||
}
|
||||
while (recentCreations > windowSize) {
|
||||
popCreationAndFollowingActions()
|
||||
}
|
||||
}
|
||||
|
||||
override fun trackCreated() {
|
||||
trackRecentUsage(RecentUsage(true) { it.trackCreated() })
|
||||
}
|
||||
|
||||
override fun trackGenerated(correct: Boolean) {
|
||||
trackRecentUsage(RecentUsage(false) { it.trackGenerated(correct) })
|
||||
}
|
||||
|
||||
private fun computeSummary(): T {
|
||||
val recentUsagesTracker = baseTrackerProvider(kind)
|
||||
recentUsages.forEach { it.action(recentUsagesTracker) }
|
||||
return recentUsagesTracker.getSummary()
|
||||
}
|
||||
|
||||
override fun getSummary(): ValuePerPeriod<T> {
|
||||
if (recentUsagesStatistics == null) {
|
||||
recentUsagesStatistics = computeSummary()
|
||||
}
|
||||
return ValuePerPeriod(recentCreations, recentUsagesStatistics!!)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package com.intellij.turboComplete.analysis.usage
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import kotlin.math.min
|
||||
|
||||
interface KindUsageTracker<T> {
|
||||
val kind: CompletionKind
|
||||
|
||||
fun trackCreated() {}
|
||||
|
||||
fun trackGenerated(correct: Boolean) {}
|
||||
|
||||
fun getSummary(): T
|
||||
}
|
||||
|
||||
class KindCreatedTracker(override val kind: CompletionKind) : KindUsageTracker<Int> {
|
||||
private var created = 0
|
||||
|
||||
override fun trackCreated() {
|
||||
created += 1
|
||||
}
|
||||
|
||||
override fun getSummary() = created
|
||||
}
|
||||
|
||||
data class KindStatistics(
|
||||
val correct: Int,
|
||||
val notCorrect: Int
|
||||
)
|
||||
|
||||
class KindGenerationTracker(override val kind: CompletionKind) : KindUsageTracker<KindStatistics> {
|
||||
private var generatedCorrect = 0
|
||||
private var generatedNotCorrect = 0
|
||||
|
||||
override fun trackGenerated(correct: Boolean) {
|
||||
if (correct)
|
||||
generatedCorrect += 1
|
||||
else
|
||||
generatedNotCorrect += 1
|
||||
}
|
||||
|
||||
override fun getSummary() = KindStatistics(generatedCorrect, generatedNotCorrect)
|
||||
}
|
||||
|
||||
class KindGenerationInRawTracker(override val kind: CompletionKind, private val maxPeriod: Int?) : KindUsageTracker<KindStatistics> {
|
||||
private var generatedCorrectInRow = 0
|
||||
private var generatedNotCorrectInRow = 0
|
||||
|
||||
override fun trackGenerated(correct: Boolean) {
|
||||
if (correct) {
|
||||
generatedCorrectInRow += 1
|
||||
generatedNotCorrectInRow = 0
|
||||
}
|
||||
else {
|
||||
generatedCorrectInRow = 0
|
||||
generatedNotCorrectInRow += 1
|
||||
}
|
||||
}
|
||||
|
||||
override fun getSummary(): KindStatistics {
|
||||
return maxPeriod?.let {
|
||||
KindStatistics(min(it, generatedCorrectInRow), min(it, generatedNotCorrectInRow))
|
||||
} ?: KindStatistics(generatedCorrectInRow, generatedNotCorrectInRow)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package com.intellij.turboComplete.analysis.usage
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.lookup.LookupEvent
|
||||
import com.intellij.codeInsight.lookup.LookupListener
|
||||
import com.intellij.codeInsight.lookup.LookupManager
|
||||
import com.intellij.openapi.components.Service
|
||||
import com.intellij.openapi.components.service
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator.Companion.suggestionGenerator
|
||||
import com.intellij.turboComplete.analysis.PipelineListener
|
||||
|
||||
@Service
|
||||
class KindVarietyUsageTracker {
|
||||
private val kindStatisticsPerVariety: MutableMap<CompletionKind, CombinedKindUsageTracker> = mutableMapOf()
|
||||
|
||||
private val recentPeriodLength = 30
|
||||
|
||||
fun kindStatistics(kind: CompletionKind): CombinedKindUsageStatistics {
|
||||
return kindStatisticsPerVariety
|
||||
.getOrDefault(kind, CombinedKindUsageTracker(kind, recentPeriodLength, recentPeriodLength))
|
||||
.getSummary()
|
||||
}
|
||||
|
||||
fun trackedKinds(variety: KindVariety): List<CompletionKind> {
|
||||
return kindStatisticsPerVariety.keys.filter { it.variety == variety }
|
||||
}
|
||||
|
||||
fun trackedKinds(parameters: CompletionParameters): List<CompletionKind> {
|
||||
return kindStatisticsPerVariety.keys.filter { it.variety.kindsCorrespondToParameters(parameters) }
|
||||
}
|
||||
|
||||
private fun kindStatisticsTracker(kind: CompletionKind): CombinedKindUsageTracker {
|
||||
return kindStatisticsPerVariety.getOrPut(kind) {
|
||||
CombinedKindUsageTracker(kind, recentPeriodLength, recentPeriodLength)
|
||||
}
|
||||
}
|
||||
|
||||
class UsagePipelineListener : PipelineListener {
|
||||
override fun onInitialize(parameters: CompletionParameters) {
|
||||
LookupManager.getActiveLookup(parameters.editor)?.addLookupListener(UsageLookupListener())
|
||||
}
|
||||
|
||||
override fun onGeneratorCollected(suggestionGenerator: SuggestionGenerator) {
|
||||
service<KindVarietyUsageTracker>()
|
||||
.kindStatisticsTracker(suggestionGenerator.kind)
|
||||
.trackCreated()
|
||||
}
|
||||
}
|
||||
|
||||
class UsageLookupListener : LookupListener {
|
||||
override fun itemSelected(event: LookupEvent) {
|
||||
val tracker = service<KindVarietyUsageTracker>()
|
||||
val correctSuggestionGenerator = event.item?.suggestionGenerator ?: return
|
||||
|
||||
tracker.kindStatisticsPerVariety.forEach { (kind, kindStatisticsTracker) ->
|
||||
kindStatisticsTracker.trackGenerated(correct = kind == correctSuggestionGenerator.kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package com.intellij.turboComplete.features.context
|
||||
|
||||
import com.intellij.codeInsight.completion.ml.CompletionEnvironment
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatureProvider
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.openapi.components.service
|
||||
import com.intellij.turboComplete.analysis.usage.KindVarietyUsageTracker
|
||||
|
||||
class AllKindsUsageFeatures : ContextFeatureProvider {
|
||||
override fun getName() = "kind_usage"
|
||||
|
||||
override fun calculateFeatures(environment: CompletionEnvironment): Map<String, MLFeatureValue> {
|
||||
val usageTracker = service<KindVarietyUsageTracker>()
|
||||
val features = mutableMapOf<String, MLFeatureValue>()
|
||||
|
||||
val trackedKinds = usageTracker.trackedKinds(environment.parameters)
|
||||
|
||||
features.putAll(
|
||||
trackedKinds
|
||||
.map { kind -> kind to usageTracker.kindStatistics(kind) }
|
||||
.filter { it.second.generated.correct > 0 }
|
||||
.flatMap { (kind, usage) ->
|
||||
listOf(
|
||||
"correct_prob_${kind.name}" to MLFeatureValue.float(usage.recentProbGenerateCorrect.value),
|
||||
"correct_in_row_${kind.name}" to MLFeatureValue.numerical(usage.generatedInRow.correct),
|
||||
"incorrect_in_row_${kind.name}" to MLFeatureValue.numerical(usage.generatedInRow.notCorrect),
|
||||
)
|
||||
}
|
||||
)
|
||||
return features
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package com.intellij.turboComplete.features.context
|
||||
|
||||
import com.intellij.codeInsight.completion.ml.CompletionEnvironment
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatureProvider
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.turboComplete.CompletionPerformanceParameters
|
||||
|
||||
class CompletionPerformanceStatusFeatures : ContextFeatureProvider {
|
||||
override fun getName() = "common_completion_kind"
|
||||
|
||||
override fun calculateFeatures(environment: CompletionEnvironment): Map<String, MLFeatureValue> {
|
||||
val performanceParameters = CompletionPerformanceParameters.fromCompletionPreferences(environment.parameters)
|
||||
return mutableMapOf(
|
||||
"performance_enabled" to MLFeatureValue.binary(performanceParameters.enabled),
|
||||
"show_lookup_early" to MLFeatureValue.binary(performanceParameters.showLookupEarly),
|
||||
"fixed_generators_order" to MLFeatureValue.binary(performanceParameters.fixedGeneratorsOrder)
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package com.intellij.turboComplete.features.element
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.ElementFeatureProvider
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator.Companion.suggestionGenerator
|
||||
import com.intellij.turboComplete.features.kind.FeaturesComputer
|
||||
|
||||
class ElementsKindFeatures : ElementFeatureProvider {
|
||||
override fun getName() = "completion_kind"
|
||||
|
||||
override fun calculateFeatures(element: LookupElement,
|
||||
location: CompletionLocation,
|
||||
contextFeatures: ContextFeatures): Map<String, MLFeatureValue> {
|
||||
val suggestionGenerator = element.suggestionGenerator ?: return mutableMapOf()
|
||||
return FeaturesComputer.getKindFeatures(suggestionGenerator.kind, location, contextFeatures).toMutableMap()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package com.intellij.turboComplete.features.kind
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.openapi.util.Key
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
|
||||
class CompletionKindFeaturesManager {
|
||||
private val computedFeatures: MutableMap<CompletionKind, Map<String, MLFeatureValue>> = mutableMapOf()
|
||||
|
||||
private fun computeKindFeatures(kind: CompletionKind,
|
||||
location: CompletionLocation,
|
||||
contextFeatures: ContextFeatures): Map<String, MLFeatureValue> {
|
||||
return KindFeatureProvider.EP_NAME.extensionList.flatMap { provider ->
|
||||
provider.calculateFeatures(kind, location, contextFeatures).entries.map { (featureName, featureValue) ->
|
||||
"${provider.getName()}_$featureName" to featureValue
|
||||
}
|
||||
}.toMap()
|
||||
}
|
||||
|
||||
fun getOrCompute(kind: CompletionKind,
|
||||
location: CompletionLocation,
|
||||
contextFeatures: ContextFeatures): Map<String, MLFeatureValue> {
|
||||
return computedFeatures.getOrPut(kind) {
|
||||
computeKindFeatures(kind, location, contextFeatures)
|
||||
}
|
||||
}
|
||||
|
||||
companion object {
|
||||
private val LOOKUP_COMPLETION_KIND_FEATURES_MANAGER = Key<CompletionKindFeaturesManager>("Manager of CompletionKinds' features")
|
||||
|
||||
val LookupImpl.completionKindFeaturesManager: CompletionKindFeaturesManager
|
||||
get() = this.getUserData(LOOKUP_COMPLETION_KIND_FEATURES_MANAGER) ?: run {
|
||||
val manager = CompletionKindFeaturesManager()
|
||||
this.putUserData(LOOKUP_COMPLETION_KIND_FEATURES_MANAGER, manager)
|
||||
manager
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
package com.intellij.turboComplete.features.kind
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.codeInsight.lookup.LookupManager
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.turboComplete.features.kind.CompletionKindFeaturesManager.Companion.completionKindFeaturesManager
|
||||
|
||||
object FeaturesComputer {
|
||||
fun getKindFeatures(kind: CompletionKind, location: CompletionLocation, contextFeatures: ContextFeatures): Map<String, MLFeatureValue> {
|
||||
val lookup = LookupManager.getActiveLookup(location.completionParameters.editor) as? LookupImpl ?: return emptyMap()
|
||||
val featuresManager = lookup.completionKindFeaturesManager
|
||||
return featuresManager.getOrCompute(kind, location, contextFeatures)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
package com.intellij.turboComplete.features.kind
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
|
||||
class KindCommonFeatures : KindFeatureProvider {
|
||||
override fun getName() = "common"
|
||||
|
||||
override fun calculateFeatures(kind: CompletionKind,
|
||||
location: CompletionLocation,
|
||||
contextFeatures: ContextFeatures): Map<String, MLFeatureValue> {
|
||||
return mutableMapOf(
|
||||
"is_name" to MLFeatureValue.categorical(kind.name)
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package com.intellij.turboComplete.features.kind
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.openapi.extensions.ExtensionPointName
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
|
||||
interface KindFeatureProvider {
|
||||
fun getName(): String
|
||||
|
||||
fun calculateFeatures(kind: CompletionKind,
|
||||
location: CompletionLocation,
|
||||
contextFeatures: ContextFeatures): Map<String, MLFeatureValue>
|
||||
|
||||
companion object {
|
||||
val EP_NAME: ExtensionPointName<KindFeatureProvider> = ExtensionPointName.create(
|
||||
"com.intellij.turboComplete.features.kind.provider")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
package com.intellij.turboComplete.features.kind
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.ml.ContextFeatures
|
||||
import com.intellij.codeInsight.completion.ml.MLFeatureValue
|
||||
import com.intellij.openapi.components.service
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.turboComplete.analysis.usage.KindVarietyUsageTracker
|
||||
|
||||
class KindUsageFeatures : KindFeatureProvider {
|
||||
override fun getName() = "usage"
|
||||
|
||||
override fun calculateFeatures(kind: CompletionKind,
|
||||
location: CompletionLocation,
|
||||
contextFeatures: ContextFeatures): Map<String, MLFeatureValue> {
|
||||
val usageTracker = service<KindVarietyUsageTracker>()
|
||||
val usage = usageTracker.kindStatistics(kind)
|
||||
return mutableMapOf(
|
||||
"correct_prob" to MLFeatureValue.float(usage.recentProbGenerateCorrect.value),
|
||||
"correct_in_row" to MLFeatureValue.numerical(usage.generatedInRow.correct),
|
||||
"incorrect_in_row" to MLFeatureValue.numerical(usage.generatedInRow.notCorrect),
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
package com.intellij.turboComplete.platform
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLookupOpener
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.RankedKind
|
||||
import com.intellij.turboComplete.CompletionPerformanceParameters
|
||||
import com.intellij.turboComplete.analysis.PipelineListener
|
||||
|
||||
class EarlyLookupOpener : PipelineListener {
|
||||
private val order: MutableList<CompletionKind> = mutableListOf()
|
||||
|
||||
override fun onInitialize(parameters: CompletionParameters) {
|
||||
}
|
||||
|
||||
override fun onRanked(ranked: List<RankedKind>) {
|
||||
order.addAll(ranked.map { it.kind })
|
||||
}
|
||||
|
||||
override fun onGenerationFinished(suggestionGenerator: SuggestionGenerator) {
|
||||
val rankIndex = order.indexOf(suggestionGenerator.kind)
|
||||
if (rankIndex != 0) return
|
||||
showLookup(suggestionGenerator.parameters)
|
||||
order.clear()
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun showLookup(parameters: CompletionParameters) {
|
||||
val performanceParameters = CompletionPerformanceParameters.fromCompletionPreferences(parameters)
|
||||
if (!performanceParameters.showLookupEarly) return
|
||||
CompletionLookupOpener.showLookup(parameters)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package com.intellij.turboComplete.platform.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.completion.ml.common.CommonElementLocationFeatures
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
|
||||
class ActualContributorPuttingPolicy(
|
||||
base: ElementsAddingPolicy,
|
||||
private val actualCompletionContributorClass: Class<*>,
|
||||
) : ElementDecoratingPolicy(base) {
|
||||
override fun decorate(element: LookupElement) {
|
||||
element.putUserData(CommonElementLocationFeatures.LOOKUP_ORIGINAL_ELEMENT_CONTRIBUTOR_TYPE,
|
||||
actualCompletionContributorClass)
|
||||
}
|
||||
}
|
||||
|
||||
fun ElementsAddingPolicy.addingActualContributor(suggestionGenerator: SuggestionGenerator): ActualContributorPuttingPolicy {
|
||||
return ActualContributorPuttingPolicy(this, suggestionGenerator.kind.variety.actualCompletionContributorClass)
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package com.intellij.turboComplete.platform.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
|
||||
class BufferingPolicy : ElementsAddingPolicy {
|
||||
private val buffer: MutableList<LookupElement> = mutableListOf()
|
||||
|
||||
override fun onResultStop(result: CompletionResultSet) {
|
||||
flushBuffer(result)
|
||||
}
|
||||
|
||||
override fun addElement(result: CompletionResultSet, element: LookupElement) {
|
||||
buffer.add(element)
|
||||
}
|
||||
|
||||
override fun addAllElements(result: CompletionResultSet, elements: MutableIterable<LookupElement>) {
|
||||
buffer.addAll(elements)
|
||||
}
|
||||
|
||||
override fun onDeactivate(result: CompletionResultSet) {
|
||||
flushBuffer(result)
|
||||
}
|
||||
|
||||
private fun flushBuffer(result: CompletionResultSet) {
|
||||
result.addAllElements(buffer)
|
||||
buffer.clear()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
package com.intellij.turboComplete.platform.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGeneratorConsumer
|
||||
|
||||
class ConvertToCompletionKindPolicy(
|
||||
private val suggestionGeneratorConsumer: SuggestionGeneratorConsumer,
|
||||
private val convertedKind: CompletionKind,
|
||||
private val parameters: CompletionParameters
|
||||
) : ElementsAddingPolicy {
|
||||
private val buffer = mutableListOf<LookupElement>()
|
||||
|
||||
override fun onResultStop(result: CompletionResultSet) {
|
||||
createCompletionKindFromBuffer(result)
|
||||
}
|
||||
|
||||
override fun addElement(result: CompletionResultSet, element: LookupElement) {
|
||||
buffer.add(element)
|
||||
}
|
||||
|
||||
override fun addAllElements(result: CompletionResultSet, elements: MutableIterable<LookupElement>) {
|
||||
buffer.addAll(elements)
|
||||
}
|
||||
|
||||
override fun onDeactivate(result: CompletionResultSet) {
|
||||
createCompletionKindFromBuffer(result)
|
||||
}
|
||||
|
||||
private fun createCompletionKindFromBuffer(result: CompletionResultSet) {
|
||||
if (buffer.isEmpty()) return
|
||||
|
||||
val bufferCopy = mutableListOf<LookupElement>()
|
||||
bufferCopy.addAll(buffer)
|
||||
buffer.clear()
|
||||
suggestionGeneratorConsumer.pass(SuggestionGenerator.fromGenerator(
|
||||
convertedKind,
|
||||
parameters,
|
||||
result
|
||||
) {
|
||||
result.addAllElements(bufferCopy)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
package com.intellij.turboComplete.platform.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
|
||||
abstract class ElementDecoratingPolicy(protected val base: ElementsAddingPolicy) : ElementsAddingPolicy {
|
||||
override fun addElement(result: CompletionResultSet, element: LookupElement) {
|
||||
decorate(element)
|
||||
base.addElement(result, element)
|
||||
}
|
||||
|
||||
override fun addAllElements(result: CompletionResultSet, elements: MutableIterable<LookupElement>) {
|
||||
elements.forEach { decorate(it) }
|
||||
base.addAllElements(result, elements)
|
||||
}
|
||||
|
||||
override fun onActivate(result: CompletionResultSet) = base.onActivate(result)
|
||||
|
||||
override fun onDeactivate(result: CompletionResultSet) = base.onDeactivate(result)
|
||||
|
||||
abstract fun decorate(element: LookupElement)
|
||||
|
||||
override fun onResultStop(result: CompletionResultSet) = base.onResultStop(result)
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
// Copyright 2000-2023 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license.
|
||||
package com.intellij.turboComplete.platform.addingPolicy
|
||||
|
||||
import com.intellij.codeInsight.completion.addingPolicy.ElementsAddingPolicy
|
||||
import com.intellij.codeInsight.lookup.LookupElement
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
|
||||
class SuggestionGeneratorPuttingPolicy(
|
||||
base: ElementsAddingPolicy,
|
||||
private val suggestionGenerator: SuggestionGenerator,
|
||||
) : ElementDecoratingPolicy(base) {
|
||||
override fun decorate(element: LookupElement) {
|
||||
element.putUserData(SuggestionGenerator.LOOKUP_ELEMENT_SUGGESTION_GENERATOR, suggestionGenerator)
|
||||
}
|
||||
}
|
||||
|
||||
fun ElementsAddingPolicy.addingCompletionKind(suggestionGenerator: SuggestionGenerator): ElementsAddingPolicy {
|
||||
return SuggestionGeneratorPuttingPolicy(this, suggestionGenerator)
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
package com.intellij.turboComplete.platform.contributor
|
||||
|
||||
import com.intellij.codeInsight.completion.*
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindCollector
|
||||
import com.intellij.turboComplete.CompletionPerformanceParameters
|
||||
|
||||
class FilteringCompletionContributor : CompletionContributor() {
|
||||
private val filter: (CompletionContributor) -> Boolean = RejectedListFilter(KindCollector.EP_NAME.extensionList.map { it.kindVariety.actualCompletionContributorClass })
|
||||
|
||||
override fun fillCompletionVariants(parameters: CompletionParameters, result: CompletionResultSet) {
|
||||
val performanceParameters = CompletionPerformanceParameters.fromCompletionPreferences(parameters)
|
||||
if (!performanceParameters.enabled) return
|
||||
|
||||
// TODO: Verify if that works
|
||||
|
||||
FilteringResultSet(result, filter).runRemainingContributors(parameters,true)
|
||||
result.stopHere()
|
||||
}
|
||||
}
|
||||
|
||||
private class RejectedListFilter(private val rejectedTypes: List<Class<*>>) : (CompletionContributor) -> Boolean {
|
||||
override fun invoke(contributor: CompletionContributor): Boolean {
|
||||
return contributor.javaClass !in rejectedTypes
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package com.intellij.turboComplete.platform.contributor
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionContributor
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.completion.PolicyObeyingResultSet
|
||||
import com.intellij.codeInsight.completion.addingPolicy.PolicyController
|
||||
import com.intellij.openapi.util.ThrowableComputable
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindCollector
|
||||
import com.intellij.turboComplete.CompletionPerformanceParameters
|
||||
import com.intellij.turboComplete.ReportingSuggestionGeneratorExecutor
|
||||
import com.intellij.turboComplete.SuggestionGeneratorExecutorProvider
|
||||
import com.intellij.util.indexing.DumbModeAccessType
|
||||
|
||||
class KindExecutingCompletionContributor : CompletionContributor() {
|
||||
override fun fillCompletionVariants(parameters: CompletionParameters, result: CompletionResultSet) {
|
||||
val performanceParameters = CompletionPerformanceParameters.fromCompletionPreferences(parameters)
|
||||
if (!performanceParameters.enabled) return
|
||||
|
||||
val executorProvider = SuggestionGeneratorExecutorProvider.findOneMatching(parameters)
|
||||
|
||||
val policyController = PolicyController(result)
|
||||
val obeyingResult = PolicyObeyingResultSet(result, policyController)
|
||||
|
||||
val executor = ReportingSuggestionGeneratorExecutor.initialize(
|
||||
parameters, policyController, executorProvider
|
||||
)
|
||||
|
||||
for (generator in KindCollector.forParameters(parameters)) {
|
||||
val policyWhileGenerating = executor.createNoneKindPolicy()
|
||||
policyController.invokeWithPolicy(policyWhileGenerating) {
|
||||
executor.reportGenerateCompletionKinds(generator, parameters, executor, obeyingResult, policyController)
|
||||
}
|
||||
if (obeyingResult.isStopped) break
|
||||
}
|
||||
|
||||
DumbModeAccessType.RELIABLE_DATA_ONLY.ignoreDumbMode(ThrowableComputable {
|
||||
executor.executeAll()
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package com.intellij.turboComplete.platform.contributor
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionContributor
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.completion.CompletionResultSet
|
||||
import com.intellij.codeInsight.lookup.LookupManager
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.completion.ml.sorting.ContextFactorCalculator
|
||||
import com.intellij.completion.ml.storage.MutableLookupStorage
|
||||
import com.intellij.openapi.project.DumbAware
|
||||
import com.intellij.turboComplete.CompletionPerformanceParameters
|
||||
|
||||
class KindOrderingFeaturesContributor : CompletionContributor(), DumbAware {
|
||||
override fun fillCompletionVariants(parameters: CompletionParameters, result: CompletionResultSet) {
|
||||
val performanceParameters = CompletionPerformanceParameters.fromCompletionPreferences(parameters)
|
||||
if (!performanceParameters.enabled) return
|
||||
|
||||
val lookup = LookupManager.getActiveLookup(parameters.editor) as? LookupImpl
|
||||
if (lookup != null) {
|
||||
val storage = MutableLookupStorage.get(lookup)
|
||||
if (storage != null) {
|
||||
MutableLookupStorage.saveAsUserData(parameters, storage)
|
||||
if (!storage.isContextFactorsInitialized()) {
|
||||
ContextFactorCalculator.calculateContextFactors(lookup, parameters, storage)
|
||||
}
|
||||
}
|
||||
}
|
||||
super.fillCompletionVariants(parameters, result)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
import com.intellij.platform.ml.impl.turboComplete.SuggestionGenerator
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.RankedKind
|
||||
|
||||
interface KindRelevanceSorter {
|
||||
val kindVariety: KindVariety
|
||||
|
||||
fun sort(kinds: List<CompletionKind>, parameters: CompletionParameters): List<RankedKind>
|
||||
|
||||
fun sortGenerators(suggestionGenerators: List<SuggestionGenerator>, parameters: CompletionParameters): List<SuggestionGenerator> {
|
||||
val completionKindNames = suggestionGenerators.map { it.kind }
|
||||
val order = sort(completionKindNames, parameters)
|
||||
.map { ranked ->
|
||||
val actualCompletionKind = suggestionGenerators.find { it.kind == ranked.kind }!!
|
||||
actualCompletionKind
|
||||
}
|
||||
return order
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.KindRankingListener
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.RankedKind
|
||||
|
||||
interface DelegatingKindRankingListener<T : KindRankingListener> : KindRankingListener {
|
||||
val delegatedListeners: MutableList<T>
|
||||
|
||||
override fun onRankingStarted() {
|
||||
delegatedListeners.forEach { it.onRankingStarted() }
|
||||
}
|
||||
|
||||
override fun onRanked(ranked: List<RankedKind>) {
|
||||
delegatedListeners.forEach { it.onRanked(ranked) }
|
||||
}
|
||||
|
||||
override fun onRankingFinished() {
|
||||
delegatedListeners.forEach { it.onRankingFinished() }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
|
||||
interface KindSorterProvider {
|
||||
val kindVariety: KindVariety
|
||||
|
||||
fun createSorter(): KindRelevanceSorter
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.completion.ml.ranker.local.MLCompletionLocalModelsLoader
|
||||
import com.intellij.internal.ml.DecisionFunction
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
|
||||
class LocalOrDefaultSorterProvider(
|
||||
private val languageId: String,
|
||||
modelRegistryKey: String,
|
||||
private val defaultDecisionFunctionProvider: () -> DecisionFunction,
|
||||
override val kindVariety: KindVariety,
|
||||
) : KindSorterProvider {
|
||||
private val localModelLoader = MLCompletionLocalModelsLoader(modelRegistryKey)
|
||||
|
||||
override fun createSorter(): KindRelevanceSorter {
|
||||
return MLKindSorter(loadLocalIfAny(languageId, defaultDecisionFunctionProvider), kindVariety)
|
||||
}
|
||||
|
||||
private fun loadLocalIfAny(languageId: String, default: () -> DecisionFunction): DecisionFunction {
|
||||
return localModelLoader.getModel(languageId) ?: default()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionLocation
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.codeInsight.lookup.LookupManager
|
||||
import com.intellij.codeInsight.lookup.impl.LookupImpl
|
||||
import com.intellij.completion.ml.features.ContextFeaturesStorage
|
||||
import com.intellij.completion.ml.sorting.LanguageRankingModel
|
||||
import com.intellij.completion.ml.sorting.RankingFeatures
|
||||
import com.intellij.completion.ml.storage.MutableLookupStorage
|
||||
import com.intellij.internal.ml.DecisionFunction
|
||||
import com.intellij.internal.ml.completion.DecoratingItemsPolicy
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.RankedKind
|
||||
import com.intellij.turboComplete.features.kind.FeaturesComputer
|
||||
|
||||
class MLKindSorter(decisionFunction: DecisionFunction, override val kindVariety: KindVariety) : KindRelevanceSorter {
|
||||
|
||||
private val model = LanguageRankingModel(decisionFunction, DecoratingItemsPolicy.DISABLED)
|
||||
|
||||
override fun sort(kinds: List<CompletionKind>,
|
||||
parameters: CompletionParameters): List<RankedKind> {
|
||||
|
||||
val lookup = LookupManager.getActiveLookup(parameters.editor) as? LookupImpl
|
||||
|
||||
requireNotNull(lookup) { "Unable to perform Completion Kind ordering when Lookup is not LookupImpl" }
|
||||
|
||||
val lookupStorage = MutableLookupStorage.get(lookup)
|
||||
|
||||
requireNotNull(lookupStorage)
|
||||
require(lookupStorage.isContextFactorsInitialized())
|
||||
|
||||
val kindsWeights = kinds.map { it to predict(it, lookupStorage, parameters) }
|
||||
|
||||
return RankedKind.fromWeights(
|
||||
kindsWeights,
|
||||
true
|
||||
)
|
||||
}
|
||||
|
||||
private fun predict(kind: CompletionKind, lookupStorage: MutableLookupStorage, parameters: CompletionParameters): Double {
|
||||
val rankingFeatures = RankingFeatures(
|
||||
lookupStorage.userFactors,
|
||||
lookupStorage.contextFactors,
|
||||
emptyMap(), emptyMap(), emptySet()
|
||||
)
|
||||
|
||||
val kindFeatures = FeaturesComputer.getKindFeatures(kind, CompletionLocation(parameters), lookupStorage.contextProvidersResult())
|
||||
|
||||
return model.score(rankingFeatures.withElementFeatures(
|
||||
ContextFeaturesStorage(kindFeatures).asMap(),
|
||||
emptyMap(),
|
||||
))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.internal.ml.DecisionFunction
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
|
||||
class MLKindSorterProvider(
|
||||
override val kindVariety: KindVariety,
|
||||
val decisionFunctionProvider: () -> DecisionFunction,
|
||||
) : KindSorterProvider {
|
||||
override fun createSorter(): KindRelevanceSorter {
|
||||
return MLKindSorter(decisionFunctionProvider(), kindVariety)
|
||||
}
|
||||
}
|
||||
|
||||
fun MLKindSorterProvider.provideLocalIfAny(languageId: String, registryKey: String): KindSorterProvider {
|
||||
return LocalOrDefaultSorterProvider(languageId, registryKey, this.decisionFunctionProvider, this.kindVariety)
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package com.intellij.turboComplete.ranking
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.KindRankingListener
|
||||
import com.intellij.platform.ml.impl.turboComplete.ranking.RankedKind
|
||||
|
||||
class ReportingKindSorter(
|
||||
private val listener: KindRankingListener,
|
||||
private val sorterProvider: KindSorterProvider
|
||||
) : KindRelevanceSorter {
|
||||
override val kindVariety: KindVariety
|
||||
get() = sorterProvider.kindVariety
|
||||
|
||||
override fun sort(kinds: List<CompletionKind>, parameters: CompletionParameters): List<RankedKind> {
|
||||
val sorter = sorterProvider.createSorter()
|
||||
listener.onRankingStarted()
|
||||
return try {
|
||||
val kindsSorted = sorter.sort(kinds, parameters)
|
||||
listener.onRanked(kindsSorted)
|
||||
kindsSorted
|
||||
}
|
||||
finally {
|
||||
listener.onRankingFinished()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
package com.intellij.codeInsight.completion.ml.performance.analysis.usage
|
||||
|
||||
import com.intellij.codeInsight.completion.CompletionParameters
|
||||
import com.intellij.platform.ml.impl.turboComplete.CompletionKind
|
||||
import com.intellij.platform.ml.impl.turboComplete.KindVariety
|
||||
import com.intellij.turboComplete.NullableKindName
|
||||
import com.intellij.turboComplete.analysis.usage.CombinedKindUsageStatistics
|
||||
import com.intellij.turboComplete.analysis.usage.CombinedKindUsageTracker
|
||||
import com.intellij.turboComplete.analysis.usage.KindStatistics
|
||||
import com.intellij.turboComplete.analysis.usage.ValuePerPeriod
|
||||
import org.junit.jupiter.api.Assertions.assertEquals
|
||||
import org.junit.jupiter.api.Test
|
||||
|
||||
|
||||
class CombinedKindUsageTrackerTest {
|
||||
private enum class Event {
|
||||
CREATED,
|
||||
GENERATED_CORRECT,
|
||||
GENERATED_NOT_CORRECT,
|
||||
}
|
||||
|
||||
private fun makeMockCompletionKind() = CompletionKind(
|
||||
NullableKindName.NONE_KIND,
|
||||
|
||||
object : KindVariety {
|
||||
|
||||
override fun kindsCorrespondToParameters(parameters: CompletionParameters) = true
|
||||
|
||||
override val actualCompletionContributorClass: Class<*>
|
||||
get() = throw NotImplementedError()
|
||||
}
|
||||
)
|
||||
|
||||
private fun getEventsStatistics(windowSize: Int, events: List<Event>): CombinedKindUsageStatistics {
|
||||
val kind = makeMockCompletionKind()
|
||||
val tracker = CombinedKindUsageTracker(kind, windowSize, windowSize)
|
||||
events.forEach {
|
||||
when (it) {
|
||||
Event.CREATED -> tracker.trackCreated()
|
||||
Event.GENERATED_CORRECT -> tracker.trackGenerated(true)
|
||||
Event.GENERATED_NOT_CORRECT -> tracker.trackGenerated(false)
|
||||
}
|
||||
}
|
||||
return tracker.getSummary()
|
||||
}
|
||||
|
||||
private fun validateStates(windowSize: Int, events: List<Pair<Event, CombinedKindUsageStatistics>>) {
|
||||
val kind = makeMockCompletionKind()
|
||||
val tracker = CombinedKindUsageTracker(kind, windowSize, windowSize)
|
||||
events.forEach { (event, expectedStatistics) ->
|
||||
when (event) {
|
||||
Event.CREATED -> tracker.trackCreated()
|
||||
Event.GENERATED_CORRECT -> tracker.trackGenerated(true)
|
||||
Event.GENERATED_NOT_CORRECT -> tracker.trackGenerated(false)
|
||||
}
|
||||
assertEquals(expectedStatistics, tracker.getSummary())
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `test total created`() {
|
||||
val windowSize = 3
|
||||
|
||||
fun assertCreated(expected: Int, events: List<Event>) {
|
||||
assertEquals(expected, getEventsStatistics(windowSize, events).created)
|
||||
}
|
||||
|
||||
assertCreated(0, listOf())
|
||||
assertCreated(1, listOf(Event.CREATED))
|
||||
assertCreated(2, listOf(Event.CREATED, Event.CREATED))
|
||||
assertCreated(15, List(15) { Event.CREATED })
|
||||
assertCreated(1, listOf(Event.CREATED, Event.GENERATED_CORRECT))
|
||||
assertCreated(1, listOf(Event.CREATED, Event.GENERATED_CORRECT, Event.GENERATED_NOT_CORRECT))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `test total generated correct`() {
|
||||
val windowSize = 3
|
||||
|
||||
fun assertCorrect(expected: Int, events: List<Event>) {
|
||||
assertEquals(expected, getEventsStatistics(windowSize, events).generated.correct)
|
||||
}
|
||||
|
||||
assertCorrect(0, listOf())
|
||||
assertCorrect(0, listOf(Event.CREATED))
|
||||
assertCorrect(0, listOf(Event.GENERATED_NOT_CORRECT))
|
||||
assertCorrect(1, listOf(Event.GENERATED_CORRECT))
|
||||
assertCorrect(2, listOf(Event.GENERATED_NOT_CORRECT, Event.GENERATED_CORRECT, Event.CREATED, Event.CREATED, Event.GENERATED_CORRECT))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `test total generated not correct`() {
|
||||
val windowSize = 3
|
||||
|
||||
fun assertCorrect(expected: Int, events: List<Event>) {
|
||||
assertEquals(expected, getEventsStatistics(windowSize, events).generated.notCorrect)
|
||||
}
|
||||
|
||||
assertCorrect(0, listOf())
|
||||
assertCorrect(0, listOf(Event.CREATED))
|
||||
assertCorrect(1, listOf(Event.GENERATED_NOT_CORRECT))
|
||||
assertCorrect(0, listOf(Event.GENERATED_CORRECT))
|
||||
assertCorrect(1, listOf(Event.GENERATED_NOT_CORRECT, Event.GENERATED_CORRECT, Event.CREATED, Event.CREATED, Event.GENERATED_CORRECT))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `test total not correct in row`() {
|
||||
val windowSize = 3
|
||||
|
||||
fun assertNotCorrectInRow(expected: Int, events: List<Event>) {
|
||||
assertEquals(expected, getEventsStatistics(windowSize, events).generatedInRow.notCorrect)
|
||||
}
|
||||
|
||||
assertNotCorrectInRow(0, listOf())
|
||||
assertNotCorrectInRow(0, listOf(Event.CREATED))
|
||||
assertNotCorrectInRow(0, listOf(Event.GENERATED_CORRECT))
|
||||
assertNotCorrectInRow(1, listOf(Event.GENERATED_NOT_CORRECT))
|
||||
assertNotCorrectInRow(0, listOf(Event.GENERATED_NOT_CORRECT, Event.GENERATED_CORRECT, Event.CREATED, Event.CREATED,
|
||||
Event.GENERATED_CORRECT))
|
||||
assertNotCorrectInRow(3, List(4) { Event.GENERATED_NOT_CORRECT })
|
||||
assertNotCorrectInRow(2, List(2) { Event.GENERATED_NOT_CORRECT } + listOf(Event.CREATED))
|
||||
assertNotCorrectInRow(3, List(2) { Event.GENERATED_NOT_CORRECT } + listOf(Event.CREATED, Event.GENERATED_NOT_CORRECT))
|
||||
assertNotCorrectInRow(0, List(4) { Event.GENERATED_NOT_CORRECT } + listOf(Event.CREATED, Event.GENERATED_CORRECT))
|
||||
|
||||
assertNotCorrectInRow(2, listOf(
|
||||
Event.CREATED,
|
||||
Event.CREATED,
|
||||
Event.GENERATED_CORRECT,
|
||||
Event.GENERATED_NOT_CORRECT,
|
||||
Event.CREATED,
|
||||
Event.GENERATED_NOT_CORRECT,
|
||||
Event.GENERATED_NOT_CORRECT,
|
||||
Event.GENERATED_CORRECT,
|
||||
Event.GENERATED_NOT_CORRECT,
|
||||
Event.GENERATED_NOT_CORRECT,
|
||||
))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `test recently generated correct`() {
|
||||
val windowSize = 2
|
||||
|
||||
fun assertCreated(expected: Int, events: List<Event>) {
|
||||
assertEquals(expected, getEventsStatistics(windowSize, events).recentGenerated.value.correct)
|
||||
}
|
||||
|
||||
assertCreated(0, listOf())
|
||||
assertCreated(0, listOf(Event.CREATED))
|
||||
assertCreated(1, listOf(Event.CREATED, Event.GENERATED_CORRECT))
|
||||
assertCreated(3, listOf(Event.CREATED, Event.GENERATED_CORRECT, Event.GENERATED_CORRECT, Event.GENERATED_CORRECT))
|
||||
assertCreated(0, listOf(Event.CREATED, Event.GENERATED_CORRECT, Event.GENERATED_CORRECT, Event.CREATED, Event.CREATED))
|
||||
assertCreated(0, listOf(Event.CREATED, Event.GENERATED_CORRECT, Event.GENERATED_CORRECT, Event.CREATED, Event.CREATED, Event.CREATED))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `test event sequence`() {
|
||||
validateStates(2, listOf(
|
||||
Event.CREATED to CombinedKindUsageStatistics(
|
||||
1, generated = KindStatistics(0, 0), generatedInRow = KindStatistics(0, 0), ValuePerPeriod(1, KindStatistics(0, 0))
|
||||
),
|
||||
Event.GENERATED_CORRECT to CombinedKindUsageStatistics(
|
||||
1, generated = KindStatistics(1, 0), generatedInRow = KindStatistics(1, 0), ValuePerPeriod(1, KindStatistics(1, 0))
|
||||
),
|
||||
Event.CREATED to CombinedKindUsageStatistics(
|
||||
2, generated = KindStatistics(1, 0), generatedInRow = KindStatistics(1, 0), ValuePerPeriod(2, KindStatistics(1, 0))
|
||||
),
|
||||
Event.GENERATED_CORRECT to CombinedKindUsageStatistics(
|
||||
2, generated = KindStatistics(2, 0), generatedInRow = KindStatistics(2, 0), ValuePerPeriod(2, KindStatistics(2, 0))
|
||||
),
|
||||
Event.CREATED to CombinedKindUsageStatistics(
|
||||
3, generated = KindStatistics(2, 0), generatedInRow = KindStatistics(2, 0), ValuePerPeriod(2, KindStatistics(1, 0))
|
||||
),
|
||||
Event.GENERATED_CORRECT to CombinedKindUsageStatistics(
|
||||
3, generated = KindStatistics(3, 0), generatedInRow = KindStatistics(2, 0), ValuePerPeriod(2, KindStatistics(2, 0))
|
||||
),
|
||||
Event.CREATED to CombinedKindUsageStatistics(
|
||||
4, generated = KindStatistics(3, 0), generatedInRow = KindStatistics(2, 0), ValuePerPeriod(2, KindStatistics(1, 0))
|
||||
),
|
||||
Event.GENERATED_NOT_CORRECT to CombinedKindUsageStatistics(
|
||||
4, generated = KindStatistics(3, 1), generatedInRow = KindStatistics(0, 1), ValuePerPeriod(2, KindStatistics(1, 1))
|
||||
),
|
||||
Event.CREATED to CombinedKindUsageStatistics(
|
||||
5, generated = KindStatistics(3, 1), generatedInRow = KindStatistics(0, 1), ValuePerPeriod(2, KindStatistics(0, 1))
|
||||
),
|
||||
Event.GENERATED_CORRECT to CombinedKindUsageStatistics(
|
||||
5, generated = KindStatistics(4, 1), generatedInRow = KindStatistics(1, 0), ValuePerPeriod(2, KindStatistics(1, 1))
|
||||
),
|
||||
Event.CREATED to CombinedKindUsageStatistics(
|
||||
6, generated = KindStatistics(4, 1), generatedInRow = KindStatistics(1, 0), ValuePerPeriod(2, KindStatistics(1, 0))
|
||||
),
|
||||
Event.GENERATED_CORRECT to CombinedKindUsageStatistics(
|
||||
6, generated = KindStatistics(5, 1), generatedInRow = KindStatistics(2, 0), ValuePerPeriod(2, KindStatistics(2, 0))
|
||||
),
|
||||
))
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user