diff --git a/platform/jewel/docs/standalone-code-highlighting.md b/platform/jewel/docs/standalone-code-highlighting.md index d92173d16456..881219997f89 100644 --- a/platform/jewel/docs/standalone-code-highlighting.md +++ b/platform/jewel/docs/standalone-code-highlighting.md @@ -35,7 +35,19 @@ plugins/textmate/lib/bundles/java/syntaxes/java.tmLanguage.json The fields you need: `match` (single-line pattern), `begin`/`end` (block patterns like strings and comments), and `captures` (group-to-scope mappings). Look for scopes that map to `TokenType` values: `keyword.*`, `string.*`, -`comment.*`, `storage.type`/`support.type`, `constant.*`, `constant.numeric`, and `entity.name.function`. +`comment.*`, `storage.type`/`support.type`, `support.type.property-name` (data-language keys), `constant.*`, +`constant.numeric`, and `entity.name.function`. + +Scopes with no `TokenType` equivalent are meant to be skipped: `meta.*` (structural context), `punctuation.*`, and +`invalid.*` (error highlighting, which this highlighter deliberately doesn't do). + +When a scope could plausibly map to more than one `TokenType`, check what the TextMate plugin itself does before +deciding. `plugins/textmate/src/.../highlighting/TextMateDefaultColorsProvider.java` maps each scope prefix to a +`TextAttributesKey`; `platform/core-api/src/com/intellij/openapi/editor/DefaultLanguageHighlighterColors.java` gives +that key its fallback; and `platform/platform-resources/src/DefaultColorSchemesManager.xml` holds the values for both +Default and Darcula. Two mappings there are easy to get wrong from intuition: `storage.type` is a **keyword**, not a +type, so C's `int` and `size_t` are keywords; and `keyword.operator` is `DEFAULT_OPERATION_SIGN`, a key of its own that +neither default scheme colors, which is what `TokenType.OPERATOR` exists for. If the IntelliJ bundle doesn't cover your language, check the [shikijs/textmate-grammars-themes](https://github.com/shikijs/textmate-grammars-themes) repo or the @@ -44,6 +56,19 @@ If the IntelliJ bundle doesn't cover your language, check the tmLanguage grammars use [Oniguruma](https://macromates.com/manual/en/regular_expressions) RegEx library. See [Regex engine limitations](#regex-engine-limitations) for what breaks when porting to Java's regex engine. +### Watch for patterns that rely on the grammar tree + +A tmLanguage grammar is a set of regexes **plus a tree** deciding which of them are eligible where. We only have +the regexes: our rule list is flat and every rule is tried at every position. So a pattern that was safe upstream +because it could only be reached in one context may misfire once it's tried everywhere. + +For example, CSS scopes `#tag-names` under `#selector` and `#property-values` under `#rule-list`, so they can +never both apply. Flattened, the ~20 words that are in both lists collide, and `display: table` colors `table` as +an HTML tag. + +When a pattern depends on context, re-encode that context in the pattern, or leave the rule out. Either way, say +which you did in the file header, and mark any lookaround you add as yours rather than the bundle's. + Add a comment at the top of the file pointing to the source: ```kotlin @@ -110,18 +135,30 @@ internal object BuiltInLanguageGrammars { } ``` -When opening the PR, make sure to test all possible keywords for a language. Feel free to mix and match regexes, -sometimes TextMate won't highlight a keyword that GitHub does, and vice versa. Just make sure the highlighting is being -properly applied. Here's a very simple template for testing: +When opening the PR, run a sample of real code through the grammar and check every `TokenType` the language can +produce. Reading the patterns is not enough — the flat-model misfires described above only show up in the output. +Here's a very simple template for testing: ``` -Keywords, types, constants: +Keywords (control flow, declarations, modifiers): + + +Types: + + +Constants and language literals: + + +Builtins (support.type and support.function: stdlib types, well-known globals): Method declarations + calls: -Control flow: +Property keys (JSON/YAML/TOML keys, CSS property names, HTML attributes): + + +Operators (`+`, `==`, `&&`, `|`, and word operators like `instanceof`, `typeof`, `in`): Comments: @@ -134,6 +171,9 @@ Strings: ``` +It goes without saying but do leave out the sections a language doesn't have. Most have no property keys, and Kotlin has +no operator rules, for instance. + #### Registering as an additional language or overriding the LanguageGrammar for a built-in language To highlight a language that isn't built in or override an existing one, just pass your `LanguageGrammar` @@ -221,9 +261,11 @@ pattern is compiled once when the `TokenRule` is created. | `constant(pattern)` | group 0 → CONSTANT | Entire match colored as constant | | `number(pattern)` | group 0 → NUMBER | Entire match colored as number | | `builtin(pattern)` | group 0 → BUILTIN | Entire match colored as builtin | +| `propertyKey(pattern)` | group 0 → PROPERTY_KEY | Entire match colored as a data-language key (JSON/YAML/TOML) | +| `operator(pattern)` | group 0 → OPERATOR | Entire match colored as an operator (`+`, `==`, `&&`, `\|`) | | `functionCall(pattern)` | group 1 → FUNCTION_CALL | Group 1 must isolate the function name | | `functionDeclaration(pattern)` | group 1 → KEYWORD, group 2 → FUNCTION_CALL | Group 1 = keyword, group 2 = name | -| `typeDeclaration(pattern)` | group 1 → KEYWORD, group 2 → BUILTIN | Group 1 = keyword, group 2 = type name | +| `typeDeclaration(pattern)` | group 1 → KEYWORD, group 2 → TYPE | Group 1 = keyword, group 2 = type name | If the regex has fewer groups than the factory expects, the missing groups are silently skipped and no span is emitted. @@ -238,14 +280,26 @@ TokenRule( ## Regex engine limitations -Patterns run through Java's `java.util.regex`. Most tmLanguage patterns work, but a few `PCRE`/`Oniguruma` features -don't exist in Java: +Patterns run through Java's `java.util.regex`. Most tmLanguage patterns work as written. These do not: - **POSIX character classes** (`[[:alpha:]]`, etc.) -> use `[a-zA-Z]` etc. instead. -- **Variable-length lookbehind** -> only fixed-width works (`(?<=fun )` is fine, `(?<=fun\s+)` is not). Rewrite as - a capturing-group rule. -- **Named backreferences**, **conditional patterns**, **subroutine calls**, and **recursive patterns** -> not - supported. +- **`\p{Surrogate}`** -> use `\p{Cs}`. +- **Multi-codepoint escapes** (`\x{FEFF FFFE FFFF}`) -> split into one escape each. +- **Open-ended repetition** (`\d{,2}`) -> Oniguruma reads that as `{0,2}`; Java raises "Illegal repetition". + Write the zero. +- **`#` and spaces inside a character class under `(?x)`** -> Java strips both, Oniguruma does not. The C + bundle's printf rule contains `[#0\- +']`, where the `#` opens a comment that swallows the rest of the line + and leaves the class unclosed. Escape them: `[\#0\-\ +']`. +- **Named backreferences**, **conditional patterns**, **subroutine calls** and **recursive patterns** -> not + supported. Plain numeric backreferences (`\2`) are fine, which is what lets a heredoc rule match its own + closing delimiter. +- **`\G`** -> anchors to the search start, so it pins a rule to the cursor and breaks earliest-match-wins. Drop + it and re-encode whatever context it stood for. + +Java is more permissive than the usual rule of thumb in several places, so test before rewriting a pattern. It +accepts variable-length lookbehind, including `+` and `*` (`(?<=fun\s+)x` both compiles and matches), and +alternations of differing length inside one (`(?<=^|[;&]|then )`). It also accepts `(?-mix:...)`, possessive +quantifiers, atomic groups, `\p{Cntrl}`, `\x{85}` and nested class unions like `[\x{85}[^abc]]`. `additionalGrammars` is searched before the built-in list, so you can also use it to override a built-in grammar for an existing language. diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/BuiltInLanguageGrammarsTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/BuiltInLanguageGrammarsTest.kt new file mode 100644 index 000000000000..da4a3f869154 --- /dev/null +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/BuiltInLanguageGrammarsTest.kt @@ -0,0 +1,30 @@ +// Copyright 2000-2026 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license. +package org.jetbrains.jewel.intui.code.highlighting + +import kotlin.test.assertTrue +import org.jetbrains.jewel.intui.standalone.code.highlighting.BuiltInLanguageGrammars +import org.junit.jupiter.api.Test + +internal class BuiltInLanguageGrammarsTest { + // A \G whose backslash isn't itself escaped, so a literal \\G doesn't count as the anchor. + private val backtrackingAnchor = Regex("""(? + grammar.rules + .map { it.pattern } + .filter { backtrackingAnchor.containsMatchIn(it) } + .map { grammar.name to it } + } + + assertTrue( + offenders.isEmpty(), + "These built-in rules use \\G, which breaks the next-match cache in SimpleCodeHighlighter:\n" + + offenders.joinToString("\n") { (language, pattern) -> " $language: $pattern" }, + ) + } +} diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/CodeHighlightingTestUtils.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/CodeHighlightingTestUtils.kt index 5ec554dccf23..fc5d86ca5950 100644 --- a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/CodeHighlightingTestUtils.kt +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/CodeHighlightingTestUtils.kt @@ -21,6 +21,8 @@ internal val testColors = comment = Color.Gray, number = Color.Magenta, builtin = Color.White, + propertyKey = Color.Black, + operator = Color.DarkGray, ) /** All [SpanStyle]s covering character offset [index], in [spanStyles] order. */ diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/SimpleCodeHighlighterTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/SimpleCodeHighlighterTest.kt index 53ed314ebcbd..ef1e8860b2c3 100644 --- a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/SimpleCodeHighlighterTest.kt +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/SimpleCodeHighlighterTest.kt @@ -128,4 +128,30 @@ internal class SimpleCodeHighlighterTest { assertEquals(Color.Red, result.colorAt(0)) // Red = keyword in testColors assertNull(result.colorAt(4)) // "myFunc" — no span for missing group 2 } + + @Test + fun `a rule that only matches late in the input is still applied`() = runTest { + // The tokenizer caches each rule's next match. A rule whose first match is far from the start must + // not be dropped, and a rule that has already matched must keep matching after the cursor passes it. + val grammar = + LanguageGrammar( + name = "test", + rules = listOf(TokenRule.keyword("\\bfun\\b"), TokenRule.number("\\b\\d+\\b")), + ) + val code = "fun " + "a ".repeat(400) + "42" + val result = SimpleCodeHighlighter(testColors, listOf(grammar)).highlight(code, "test").first() + + assertEquals(Color.Red, result.colorAt(0)) // "fun" at the very start + assertEquals(Color.Magenta, result.colorAt(code.length - 1)) // "42" at the very end + } + + @Test + fun `every occurrence of a repeating rule is matched`() = runTest { + // Guards the cache invalidation: the cached match must be refreshed once the cursor passes it + val grammar = LanguageGrammar(name = "test", rules = listOf(TokenRule.keyword("\\bfun\\b"))) + val code = List(50) { "fun x" }.joinToString(" ") + val result = SimpleCodeHighlighter(testColors, listOf(grammar)).highlight(code, "test").first() + + assertEquals(50, result.spanStyles.size) + } } diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/TokenRuleTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/TokenRuleTest.kt index 2f61d1443281..842e87d10624 100644 --- a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/TokenRuleTest.kt +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/TokenRuleTest.kt @@ -49,6 +49,18 @@ internal class TokenRuleTest { assertEquals(mapOf(0 to TokenType.BUILTIN), rule.captures) } + @Test + fun `propertyKey factory colors entire match`() { + val rule = TokenRule.propertyKey("\"(?:[^\"\\\\]|\\\\.)*\"(?=\\s*:)") + assertEquals(mapOf(0 to TokenType.PROPERTY_KEY), rule.captures) + } + + @Test + fun `operator factory colors entire match`() { + val rule = TokenRule.operator("&&|\\|\\|") + assertEquals(mapOf(0 to TokenType.OPERATOR), rule.captures) + } + @Test fun `functionCall factory colors group 1`() { val rule = TokenRule.functionCall("\\b([A-Za-z_]\\w*)\\s*(?=\\()") @@ -62,8 +74,8 @@ internal class TokenRuleTest { } @Test - fun `typeDeclaration factory colors group 1 as keyword and group 2 as builtin`() { + fun `typeDeclaration factory colors group 1 as keyword and group 2 as type`() { val rule = TokenRule.typeDeclaration("\\b(class)\\s+([A-Za-z_]\\w*)") - assertEquals(mapOf(1 to TokenType.KEYWORD, 2 to TokenType.BUILTIN), rule.captures) + assertEquals(mapOf(1 to TokenType.KEYWORD, 2 to TokenType.TYPE), rule.captures) } } diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/CGrammarTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/CGrammarTest.kt new file mode 100644 index 000000000000..91e0b67ee4c8 --- /dev/null +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/CGrammarTest.kt @@ -0,0 +1,223 @@ +// Copyright 2000-2026 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license. +package org.jetbrains.jewel.intui.code.highlighting.languages + +import kotlin.test.assertEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.test.runTest +import org.jetbrains.jewel.intui.code.highlighting.colorAt +import org.jetbrains.jewel.intui.code.highlighting.testColors +import org.jetbrains.jewel.intui.standalone.code.highlighting.SimpleCodeHighlighter +import org.junit.jupiter.api.Test + +internal class CGrammarTest { + private val highlighter = SimpleCodeHighlighter(testColors) + + private suspend fun highlight(code: String, language: String = "c") = highlighter.highlight(code, language).first() + + @Test + fun `name and all aliases are recognized`() = runTest { + for (identifier in listOf("c", "h", "i", "cats", "idc")) { + assertTrue(highlight("int x;", identifier).spanStyles.isNotEmpty(), "Alias '$identifier' not recognized") + } + } + + @Test + fun `line and block comments are colored as comment`() = runTest { + assertEquals(testColors.comment, highlight("// a comment").colorAt(0)) + assertEquals(testColors.comment, highlight("/* a\nb */").colorAt(0)) + assertEquals(testColors.comment, highlight("int x; // trailing").colorAt(7)) + } + + @Test + fun `keywords and strings inside a comment are not colored separately`() = runTest { + assertEquals(testColors.comment, highlight("// int x").colorAt(3)) + assertEquals(testColors.comment, highlight("/* \"str\" */").colorAt(4)) + } + + @Test + fun `strings and character literals are colored as string`() = runTest { + assertEquals(testColors.string, highlight("puts(\"hi\");").colorAt(5)) + assertEquals(testColors.string, highlight("char c = 'a';").colorAt(9)) + assertEquals(testColors.string, highlight("puts(\"a\\\"b\");").colorAt(5)) + } + + @Test + fun `an escape inside a string stays string-colored`() = runTest { + // The bundle scopes \n and %d separately, but only reaches those rules inside the string + assertEquals(testColors.string, highlight("puts(\"%d\\n\");").colorAt(6)) + } + + @Test + fun `control flow keywords are colored as keyword`() = runTest { + for (keyword in listOf("break", "continue", "do", "else", "for", "goto", "if", "return", "while")) { + assertEquals(testColors.keyword, highlight("$keyword ").colorAt(0), "'$keyword' should be a keyword") + } + } + + @Test + fun `switch case and default are colored as keyword`() = runTest { + for (keyword in listOf("switch", "case", "default")) { + assertEquals(testColors.keyword, highlight("$keyword ").colorAt(0), "'$keyword' should be a keyword") + } + } + + @Test + fun `aggregate keywords are colored as keyword`() = runTest { + // The bundle files enum, struct and union under storage.type; we read them as declaration keywords + for (keyword in listOf("enum", "struct", "union")) { + assertEquals(testColors.keyword, highlight("$keyword S").colorAt(0), "'$keyword' should be a keyword") + } + } + + @Test + fun `storage modifiers and typedef are colored as keyword`() = runTest { + for (keyword in listOf("const", "extern", "register", "restrict", "static", "volatile", "inline")) { + assertEquals(testColors.keyword, highlight("$keyword int x").colorAt(0), "'$keyword' should be a keyword") + } + assertEquals(testColors.keyword, highlight("typedef int myint;").colorAt(0)) + } + + @Test + fun `a keyword appearing inside an identifier is not colored`() = runTest { + // The bundle's typedef rule is the bare word, so without the \b pair we add, the substring matches + assertNull(highlight("int mytypedefName;").colorAt(6)) + assertNull(highlight("int my_size_typedef;").colorAt(12)) + // Same check for the neighbours that already carried boundaries + assertNull(highlight("int constant;").colorAt(4)) + assertNull(highlight("int switcher;").colorAt(4)) + assertNull(highlight("int defaults;").colorAt(4)) + } + + @Test + fun `storage type names are colored as keyword`() = runTest { + // IntelliJ maps the whole storage.type family to its keyword key, so `int` is not a TYPE here + for (type in listOf("int", "char", "void", "float", "double", "unsigned", "short", "long", "_Bool")) { + assertEquals(testColors.keyword, highlight("$type x").colorAt(0), "'$type' should be a keyword") + } + // These are in storage.type.built-in.c as well as in a support.type table, and the bundle lists + // #storage_types first + for (type in listOf("size_t", "uint32_t", "pthread_t", "time_t", "ssize_t")) { + assertEquals(testColors.keyword, highlight("$type x").colorAt(0), "'$type' should be a keyword") + } + } + + @Test + fun `support type names are colored as builtin`() = runTest { + // support.type maps to IntelliJ's predefined-symbol key, which is our BUILTIN + assertEquals(testColors.builtin, highlight("UInt32 x").colorAt(0)) + assertEquals(testColors.builtin, highlight("my_custom_t x").colorAt(0), "support.type.posix-reserved") + } + + @Test + fun `language constants are colored as constant`() = runTest { + for (constant in listOf("NULL", "true", "false", "TRUE", "FALSE")) { + assertEquals(testColors.constant, highlight("x = $constant;").colorAt(4), "'$constant' is a constant") + } + } + + @Test + fun `predefined macros are colored as constant`() = runTest { + assertEquals(testColors.constant, highlight("__FILE__").colorAt(0)) + } + + @Test + fun `mac classic prefixes are honored`() = runTest { + assertEquals(testColors.constant, highlight("kMaxSize").colorAt(0)) + assertEquals(testColors.builtin, highlight("gState").colorAt(0)) + assertEquals(testColors.builtin, highlight("sBuffer").colorAt(0)) + } + + @Test + fun `numbers are colored as number`() = runTest { + for (number in listOf("42", "0xFF", "0b1010", "0755", "3.14", "1e-10", "3.14f", "10UL")) { + assertEquals(testColors.number, highlight("x = $number;").colorAt(4), "'$number' should be a number") + } + } + + @Test + fun `function calls and definitions color the name`() = runTest { + assertEquals(testColors.functionCall, highlight("puts(\"hi\");").colorAt(0)) + assertEquals(testColors.functionCall, highlight("int main(void) { return 0; }").colorAt(4)) + } + + @Test + fun `a keyword followed by a paren is not a function call`() = runTest { + for (keyword in listOf("if", "while", "return", "for", "switch")) { + assertEquals(testColors.keyword, highlight("$keyword (x)").colorAt(0), "'$keyword (' should stay a keyword") + } + // sizeof is keyword.operator.sizeof.c in the bundle, not a control keyword + assertEquals(testColors.operator, highlight("sizeof (x)").colorAt(0)) + } + + @Test + fun `the call rule does not slide past a word it excludes`() = runTest { + // The bundle's exclusion list is an unanchored lookahead. `catch` is on it but is not a C keyword, + // so without the leading boundary we added the rule would fail at `c` and match `atch (` instead. + val result = highlight("catch (x)") + assertNull(result.colorAt(0)) + assertNull(result.colorAt(1)) + } + + @Test + fun `member access colors the member but not the object`() = runTest { + assertEquals(testColors.builtin, highlight("p->count").colorAt(3)) + assertNull(highlight("p->count").colorAt(0)) + assertEquals(testColors.builtin, highlight("s.len").colorAt(2)) + } + + @Test + fun `member access wins over the operator rules`() = runTest { + // The match starts at the `-`, where the minus operator ties with it + assertEquals(testColors.builtin, highlight("argv[i]->len").colorAt(9)) + assertEquals(testColors.operator, highlight("a - b").colorAt(2), "a bare minus is still an operator") + } + + @Test + fun `preprocessor directives are colored as keyword`() = runTest { + for (directive in listOf("if", "ifdef", "ifndef", "elif", "else", "endif")) { + assertEquals(testColors.keyword, highlight("#$directive X\n").colorAt(0), "'#$directive'") + } + for (directive in listOf("pragma once", "error nope", "undef X", "line 5")) { + assertEquals(testColors.keyword, highlight("#$directive").colorAt(0), "'#$directive'") + } + } + + @Test + fun `include colors the directive and the path`() = runTest { + assertEquals(testColors.keyword, highlight("#include ").colorAt(0)) + assertEquals(testColors.string, highlight("#include ").colorAt(10)) + assertEquals(testColors.string, highlight("#include \"local.h\"").colorAt(10)) + } + + @Test + fun `define colors the directive and the macro name`() = runTest { + assertEquals(testColors.keyword, highlight("#define MAX 10").colorAt(0)) + assertEquals(testColors.functionCall, highlight("#define MAX 10").colorAt(8)) + assertEquals(testColors.functionCall, highlight("#define SQ(x) ((x)*(x))").colorAt(8)) + } + + @Test + fun `an if 0 block is greyed out`() = runTest { + val result = highlight("#if 0\ndead();\n#endif\n") + assertEquals(testColors.comment, result.colorAt(0)) + assertEquals(testColors.comment, result.colorAt(6)) + } + + @Test + fun `an if 1 block stays live`() = runTest { + assertEquals(testColors.functionCall, highlight("#if 1\nlive();\n#endif\n").colorAt(6)) + } + + @Test + fun `operators are colored as operator`() = runTest { + // keyword.operator.* is DEFAULT_OPERATION_SIGN in IntelliJ, a different key from keywords + for (expression in listOf("a + b", "a << b", "a == b", "a && b")) { + assertEquals(testColors.operator, highlight(expression).colorAt(2), "'$expression'") + } + assertEquals(testColors.operator, highlight("i++").colorAt(1)) + assertEquals(testColors.operator, highlight("a ? b : c").colorAt(2)) + assertEquals(testColors.operator, highlight("sizeof(x)").colorAt(0)) + } +} diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/CSSGrammarTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/CSSGrammarTest.kt new file mode 100644 index 000000000000..7a83dbe0135c --- /dev/null +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/CSSGrammarTest.kt @@ -0,0 +1,319 @@ +// Copyright 2000-2026 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license. +package org.jetbrains.jewel.intui.code.highlighting.languages + +import kotlin.test.assertEquals +import kotlin.test.assertNotEquals +import kotlin.test.assertTrue +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.test.runTest +import org.jetbrains.jewel.intui.code.highlighting.colorAt +import org.jetbrains.jewel.intui.code.highlighting.spansAt +import org.jetbrains.jewel.intui.code.highlighting.testColors +import org.jetbrains.jewel.intui.standalone.code.highlighting.SimpleCodeHighlighter +import org.junit.jupiter.api.Test + +internal class CSSGrammarTest { + private val highlighter = SimpleCodeHighlighter(testColors) + + private suspend fun highlight(code: String, language: String = "css") = + highlighter.highlight(code, language).first() + + @Test fun `name is recognized`() = runTest { assertTrue(highlight("a { color: red; }").spanStyles.isNotEmpty()) } + + @Test + fun `comment is colored as comment`() = runTest { + assertEquals(testColors.comment, highlight("/* a comment */").colorAt(0)) + } + + @Test + fun `property name is colored as property key`() = runTest { + // "color" starts at index 4 in "a { color: red; }" + assertEquals(testColors.propertyKey, highlight("a { color: red; }").colorAt(4)) + } + + @Test + fun `custom property is colored as property key`() = runTest { + // "--brand" starts at index 4 in "a { --brand: red; }" + assertEquals(testColors.propertyKey, highlight("a { --brand: red; }").colorAt(4)) + } + + @Test + fun `vendored property name is colored as property key`() = runTest { + assertEquals(testColors.propertyKey, highlight("a { -webkit-box-shadow: none; }").colorAt(4)) + } + + @Test + fun `element selector is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("a { color: red; }").colorAt(0)) + } + + @Test + fun `custom element selector is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("my-widget { color: red; }").colorAt(0)) + } + + @Test + fun `class selector is colored as type`() = runTest { + assertEquals(testColors.type, highlight(".foo { color: red; }").colorAt(0)) + } + + @Test + fun `class selector may contain digits and double hyphens`() = runTest { + assertEquals(testColors.type, highlight(".card--wide { color: red; }").colorAt(0)) + assertEquals(testColors.type, highlight(".col-6 { color: red; }").colorAt(0)) + } + + @Test + fun `id selector is colored as type`() = runTest { + assertEquals(testColors.type, highlight("#foo { color: red; }").colorAt(0)) + } + + @Test + fun `an id selector that looks like a hex color is still a selector`() = runTest { + // `#foo` above never reached the hex rule, since `o` is not a hex digit. These do. + assertEquals(testColors.type, highlight("#abc { color: red; }").colorAt(0)) + assertEquals(testColors.type, highlight("#abcdef { color: red; }").colorAt(0)) + assertEquals(testColors.type, highlight("#abc, #def { color: red; }").colorAt(6)) + assertEquals(testColors.type, highlight("a:hover #abc { color: red; }").colorAt(8)) + assertEquals(testColors.type, highlight("a { color: red; } #abc { color: red; }").colorAt(18)) + } + + @Test + fun `a hyphenated id selector is colored whole, not just its hex-looking prefix`() = runTest { + val result = highlight("#abc-def { color: red; }") + assertEquals(testColors.type, result.colorAt(0)) + assertEquals(testColors.type, result.colorAt(5)) + } + + @Test + fun `pseudo-class is colored as builtin`() = runTest { + val result = highlight("a:hover { color: red; }") + assertEquals(testColors.keyword, result.colorAt(0), "'a' is still an element selector") + assertEquals(testColors.builtin, result.colorAt(1), "':hover' should be a pseudo-class") + assertEquals(testColors.builtin, result.colorAt(2), "the name is part of the same match") + } + + @Test + fun `pseudo-element is colored as builtin`() = runTest { + assertEquals(testColors.builtin, highlight("a::before { color: red; }").colorAt(1)) + } + + @Test + fun `pseudo-class is not mistaken for a property`() = runTest { + // The grammar's property-name list does not contain `hover`, and the pseudo-class rule owns the colon + assertNotEquals(testColors.propertyKey, highlight("a:hover { color: red; }").colorAt(0)) + } + + @Test + fun `functional pseudo-class colors its nth expression as a number`() = runTest { + val result = highlight("li:nth-child(2n+1) { color: red; }") + assertEquals(testColors.builtin, result.colorAt(2)) + assertEquals(testColors.number, result.colorAt(13)) + } + + @Test + fun `functional pseudo-class colors its parity keyword as builtin`() = runTest { + assertEquals(testColors.builtin, highlight("li:nth-child(odd) { color: red; }").colorAt(13)) + } + + @Test + fun `negation pseudo-class is colored as builtin`() = runTest { + assertEquals(testColors.builtin, highlight("a:not(.x) { color: red; }").colorAt(1)) + } + + @Test + fun `a value that is also a tag name stays a value without a trailing semicolon`() = runTest { + // Regression: about twenty words are in both the tag-name and property-value lists. The bundle keeps + // them apart by nesting; flattened, `table` matched the tag rule because `\s` is in its lookahead set + // and `}` follows. Declarations ending in `;` were never affected. + for (value in listOf("table", "small", "ruby", "progress", "menu")) { + assertEquals( + testColors.builtin, + highlight("a { display: $value }").colorAt(13), + "'$value' without a trailing semicolon should still be a property value", + ) + assertEquals( + testColors.builtin, + highlight("a { display: $value; }").colorAt(13), + "'$value' with a trailing semicolon should be a property value", + ) + } + } + + @Test + fun `the same words are still tag names in selector position`() = runTest { + // The other half of the guard: it must not cost us selector highlighting + for (tag in listOf("table", "small", "ruby", "progress", "menu", "code")) { + assertEquals( + testColors.keyword, + highlight("$tag { color: red; }").colorAt(0), + "'$tag' in selector position should be a tag name", + ) + } + } + + @Test + fun `hex color is colored as constant`() = runTest { + // "#fff" starts at index 11 in "a { color: #fff; }" + assertEquals(testColors.constant, highlight("a { color: #fff; }").colorAt(11)) + } + + @Test + fun `a hex color in value position is a color, not an id`() = runTest { + val result = highlight("a { color: #abc; }") + assertEquals(testColors.constant, result.colorAt(11)) + assertNotEquals(testColors.type, result.colorAt(11), "'#abc' in a value is a color, not an id") + // Every shape the hex rule accepts, in the positions a value can take + assertEquals(testColors.constant, highlight("a { color: #abc }").colorAt(11)) + assertEquals(testColors.constant, highlight("a { color:#abc; }").colorAt(10)) + assertEquals(testColors.constant, highlight("a { color: #abcd; }").colorAt(11)) + assertEquals(testColors.constant, highlight("a { color: #aabbccdd; }").colorAt(11)) + assertEquals(testColors.constant, highlight("a { border: 1px solid #ff0000; }").colorAt(22)) + assertEquals( + testColors.constant, + highlight("a { background: linear-gradient(#abc, #def); }").colorAt(32), + "hex inside a function call", + ) + } + + @Test + fun `document-rule functions are colored as function call`() = runTest { + // support.function.document-rule.css — each name starts at index 10 in `@document ("x") { }` + for (function in listOf("url-prefix", "domain", "regexp")) { + assertEquals( + testColors.functionCall, + highlight("@document $function(\"x\") { }").colorAt(10), + "'$function' should be colored as a function call", + ) + } + } + + @Test + fun `at-rule is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("@media screen").colorAt(0)) + assertEquals(testColors.keyword, highlight("@font-face { }").colorAt(0)) + } + + @Test + fun `vendor-prefixed at-rule is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("@-webkit-keyframes spin { }").colorAt(0)) + } + + @Test + fun `important is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("!important").colorAt(0)) + assertEquals(testColors.keyword, highlight("a { color: red !important; }").colorAt(15)) + } + + @Test + fun `numbers with units are colored as number`() = runTest { + // "10px" starts at index 11 in "a { width: 10px; }" + assertEquals(testColors.number, highlight("a { width: 10px; }").colorAt(11)) + assertEquals(testColors.number, highlight("a { width: 50%; }").colorAt(11)) + assertEquals(testColors.number, highlight("a { margin: -1.5em; }").colorAt(12)) + } + + @Test + fun `unit gets its own keyword span`() = runTest { + // The bundle scopes the unit keyword.other.unit.*.css, so "rem" is both inside the number span and its own + // keyword span. "rem" starts at index 12 in "a { width: 2rem; }". + assertTrue(highlight("a { width: 2rem; }").spansAt(12).any { it.color == testColors.keyword }) + } + + @Test + fun `decimal without a leading digit is a number, not a class`() = runTest { + assertEquals(testColors.number, highlight("a { opacity: .5; }").colorAt(13)) + assertEquals(testColors.number, highlight("a { color: rgba(0, 0, 0, .5); }").colorAt(25)) + } + + @Test + fun `string is colored as string`() = runTest { + // The opening quote is at index 13 in "a { content: \"x\"; }" + assertEquals(testColors.string, highlight("a { content: \"x\"; }").colorAt(13)) + assertEquals(testColors.string, highlight("a { content: '\\201C'; }").colorAt(13)) + } + + @Test + fun `color keyword is colored as builtin`() = runTest { + assertEquals(testColors.builtin, highlight("a { color: red; }").colorAt(11)) + assertEquals(testColors.builtin, highlight("a { color: rebeccapurple; }").colorAt(11)) + assertEquals(testColors.builtin, highlight("a { color: currentColor; }").colorAt(11)) + } + + @Test + fun `property value keyword is colored as builtin`() = runTest { + assertEquals(testColors.builtin, highlight("a { display: flex; }").colorAt(13)) + assertEquals(testColors.builtin, highlight("a { list-style-type: lower-roman; }").colorAt(21)) + assertEquals(testColors.builtin, highlight("a { font-family: helvetica; }").colorAt(17)) + assertEquals(testColors.builtin, highlight("a { color: -moz-fixed; }").colorAt(11)) + } + + @Test + fun `function name is colored as function call`() = runTest { + assertEquals(testColors.functionCall, highlight("a { width: calc(1px); }").colorAt(11)) + assertEquals(testColors.functionCall, highlight("a { color: rgba(0,0,0,.5); }").colorAt(11)) + assertEquals(testColors.functionCall, highlight("a { width: min(1px, 2px); }").colorAt(11)) + assertEquals(testColors.functionCall, highlight("a { clip-path: circle(1px); }").colorAt(15)) + assertEquals(testColors.functionCall, highlight("a { transform: translateX(1px); }").colorAt(15)) + assertEquals(testColors.functionCall, highlight("a { transition-timing-function: steps(2); }").colorAt(32)) + } + + @Test + fun `gradient url and var are colored as function calls`() = runTest { + assertEquals(testColors.functionCall, highlight("a { background: linear-gradient(red, blue); }").colorAt(16)) + assertEquals(testColors.functionCall, highlight("a { background: url(x.png); }").colorAt(16)) + assertEquals(testColors.functionCall, highlight("a { color: var(--x); }").colorAt(11)) + } + + @Test + fun `attribute selector names its attribute and operator`() = runTest { + assertEquals(testColors.propertyKey, highlight("a[href] { color: red; }").colorAt(2)) + assertEquals(testColors.operator, highlight("a[href=\"x\"] { color: red; }").colorAt(6)) + } + + @Test + fun `media feature is colored as property key`() = runTest { + assertEquals(testColors.propertyKey, highlight("@media (min-width: 600px) { }").colorAt(8)) + } + + @Test + fun `media type and logical operator are colored`() = runTest { + val result = highlight("@media screen and (color) { }") + assertEquals(testColors.builtin, result.colorAt(7), "'screen' is a media type") + assertEquals(testColors.operator, result.colorAt(14), "'and' is a logical operator") + } + + @Test + fun `media feature keyword is colored as builtin`() = runTest { + assertEquals(testColors.builtin, highlight("@media (orientation: portrait) { }").colorAt(21)) + } + + @Test + fun `aspect ratio is colored as two numbers around an operator`() = runTest { + val result = highlight("@media (aspect-ratio: 16/9) { }") + assertEquals(testColors.number, result.colorAt(22)) + assertEquals(testColors.operator, result.colorAt(24)) + } + + @Test + fun `unicode range is colored as constant`() = runTest { + assertEquals(testColors.constant, highlight("a { unicode-range: U+0025-00FF; }").colorAt(19)) + } + + @Test + fun `combinator is an operator and the wildcard is a tag`() = runTest { + assertEquals(testColors.operator, highlight("a > b { color: red; }").colorAt(2)) + assertEquals(testColors.keyword, highlight("* { color: red; }").colorAt(0)) + } + + @Test + fun `an identifier the bundle does not know is left unstyled`() = runTest { + assertEquals(null, highlight("a { color: zzzz; }").colorAt(11)) + } + + @Test + fun `shorthand property names from the bundle list are recognized`() = runTest { + assertEquals(testColors.propertyKey, highlight("a { background-position-x: 0; }").colorAt(4)) + assertEquals(testColors.propertyKey, highlight("a { overflow-y: hidden; }").colorAt(4)) + } +} diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/HTMLGrammarTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/HTMLGrammarTest.kt new file mode 100644 index 000000000000..1406365db089 --- /dev/null +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/HTMLGrammarTest.kt @@ -0,0 +1,194 @@ +// Copyright 2000-2026 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license. +package org.jetbrains.jewel.intui.code.highlighting.languages + +import kotlin.test.assertEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.test.runTest +import org.jetbrains.jewel.intui.code.highlighting.colorAt +import org.jetbrains.jewel.intui.code.highlighting.testColors +import org.jetbrains.jewel.intui.standalone.code.highlighting.SimpleCodeHighlighter +import org.junit.jupiter.api.Test + +internal class HTMLGrammarTest { + private val highlighter = SimpleCodeHighlighter(testColors) + + private suspend fun highlight(code: String, language: String = "html") = + highlighter.highlight(code, language).first() + + @Test + fun `name and all aliases are recognized`() = runTest { + val identifiers = listOf("html", "htm", "xhtml", "xht", "shtml", "mdoc", "jshtm", "volt", "ejs", "rhtml") + for (identifier in identifiers) { + assertTrue(highlight("
", identifier).spanStyles.isNotEmpty(), "Alias '$identifier' not recognized") + } + } + + @Test + fun `comment is colored as comment`() = runTest { + assertEquals(testColors.comment, highlight("").colorAt(0)) + } + + @Test + fun `tag inside a comment is not colored as a tag`() = runTest { + // "<" of the inner tag is at index 5 in "" + assertEquals(testColors.comment, highlight("").colorAt(5)) + } + + @Test + fun `doctype name is colored as keyword`() = runTest { + // Only the DOCTYPE word carries entity.name.tag.html; the leading "") + assertEquals(testColors.keyword, highlighted.colorAt(2)) + assertNull(highlighted.colorAt(0)) + assertNull(highlighted.colorAt(10)) + } + + @Test + fun `xml processing instruction name is colored as keyword`() = runTest { + // "xml" starts at index 2 in `` + assertEquals(testColors.keyword, highlight("").colorAt(2)) + } + + @Test + fun `cdata content is colored as string`() = runTest { + // "raw" starts at index 10 in ""; the delimiters are punctuation + val highlighted = highlight("") + assertEquals(testColors.string, highlighted.colorAt(10)) + assertNull(highlighted.colorAt(0)) + } + + @Test + fun `tag name is colored as keyword`() = runTest { + val highlighted = highlight("
") + assertEquals(testColors.keyword, highlighted.colorAt(1)) + assertNull(highlighted.colorAt(0)) + } + + @Test + fun `closing tag name is colored as keyword`() = runTest { + // "div" starts at index 2 in "
" + assertEquals(testColors.keyword, highlight("
").colorAt(2)) + } + + @Test + fun `heading and void tag names are colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("

").colorAt(1)) + assertEquals(testColors.keyword, highlight("").colorAt(1)) + assertEquals(testColors.keyword, highlight("
").colorAt(1)) + } + + @Test + fun `script and style tag names are colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("").colorAt(2)) + assertEquals(testColors.keyword, highlight("").colorAt(2)) + } + + @Test + fun `custom element name is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("").colorAt(1)) + assertEquals(testColors.keyword, highlight("").colorAt(2)) + } + + @Test + fun `obsolete and unrecognized tag names are still colored as keyword`() = runTest { + // The bundle scopes these as entity.name.tag.html plus an invalid.* scope that this highlighter skips + assertEquals(testColors.keyword, highlight("").colorAt(1)) + assertEquals(testColors.keyword, highlight("").colorAt(1)) + assertEquals(testColors.keyword, highlight("").colorAt(1)) + } + + @Test + fun `attribute name is colored as property key`() = runTest { + // "href" starts at index 3 in `` + val highlighted = highlight("") + assertEquals(testColors.propertyKey, highlighted.colorAt(3)) + assertEquals(testColors.keyword, highlighted.colorAt(1)) + } + + @Test + fun `data and event handler attributes are colored as property key`() = runTest { + // Each of these attribute names starts at index 5 + assertEquals(testColors.propertyKey, highlight("
").colorAt(5)) + assertEquals(testColors.propertyKey, highlight("
").colorAt(5)) + assertEquals(testColors.propertyKey, highlight("
").colorAt(5)) + } + + @Test + fun `an attribute name buried in a longer name is not colored`() = runTest { + // The bundle's names carry a trailing (?![\w:-]) but no leading guard, so without the one we add + // these match a suffix and color only part of the name. + assertNull(highlight("
").colorAt(7), "html5 name as a suffix") + assertNull(highlight("
").colorAt(7), "event handler as a suffix") + assertNull(highlight("
").colorAt(6), "style as a suffix") + assertNull(highlight("
").colorAt(8), "data- as a suffix") + assertNull(highlight("
").colorAt(7), "after a hyphen") + assertNull(highlight("
").colorAt(9), "after a namespace colon") + } + + @Test + fun `valueless attributes are not colored`() = runTest { + // The `(?=\s*=)` guard that keeps attribute names out of prose also excludes boolean attributes. + // "type" is at index 7 and "required" at index 19 in ``. + val highlighted = highlight("") + assertEquals(testColors.propertyKey, highlighted.colorAt(7)) + assertNull(highlighted.colorAt(19)) + } + + @Test + fun `attribute names in text content are not colored`() = runTest { + // Regression: HTML5 attribute names are ordinary English words, and the bundle relies on the grammar + // tree to only ever try them between `<` and `>`. Without the `(?=\s*=)` guard, every one of these + // words was colored as a property key. + for (word in listOf("title", "for", "size", "value", "list", "method", "type", "open")) { + assertNull(highlight("

a $word b

").colorAt(5), "'$word' should not be colored in text content") + } + } + + @Test + fun `prose with no markup is not highlighted at all`() = runTest { + assertTrue(highlight("The list method returns a value of that type.").spanStyles.isEmpty()) + } + + @Test + fun `deprecated attribute name is not colored`() = runTest { + // The bundle's only scope for align, bgcolor and border is invalid.deprecated.*, which is skipped + assertNull(highlight("
").colorAt(5)) + } + + @Test + fun `attribute value is colored as string`() = runTest { + // The opening quote is at index 8 in `` + assertEquals(testColors.string, highlight("").colorAt(8)) + // The opening quote is at index 11 in `
` + assertEquals(testColors.string, highlight("
").colorAt(11)) + } + + @Test + fun `entities are colored as constant`() = runTest { + for (entity in listOf(" ", "&", " ", " ", " ")) { + assertEquals(testColors.constant, highlight(entity).colorAt(0), "'$entity' should be colored as constant") + } + } + + @Test + fun `text content is not highlighted`() = runTest { + // "hello" starts at index 3 in "

hello

" + val highlighted = highlight("

hello

") + assertNull(highlighted.colorAt(3)) + assertEquals(testColors.keyword, highlighted.colorAt(1)) + assertEquals(testColors.keyword, highlighted.colorAt(10)) + } + + @Test + fun `tags on separate lines are highlighted`() = runTest { + // "span" starts at index 9 and its closing name at index 17 + val highlighted = highlight("
\n x\n
") + assertEquals(testColors.keyword, highlighted.colorAt(9)) + assertEquals(testColors.keyword, highlighted.colorAt(17)) + } +} diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/JSONGrammarTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/JSONGrammarTest.kt new file mode 100644 index 000000000000..449d998ea456 --- /dev/null +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/JSONGrammarTest.kt @@ -0,0 +1,181 @@ +// Copyright 2000-2026 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license. +package org.jetbrains.jewel.intui.code.highlighting.languages + +import androidx.compose.ui.text.font.FontStyle +import kotlin.test.assertEquals +import kotlin.test.assertNotEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.test.runTest +import org.jetbrains.jewel.intui.code.highlighting.colorAt +import org.jetbrains.jewel.intui.code.highlighting.spansAt +import org.jetbrains.jewel.intui.code.highlighting.testColors +import org.jetbrains.jewel.intui.standalone.code.highlighting.SimpleCodeHighlighter +import org.junit.jupiter.api.Test + +internal class JSONGrammarTest { + private val highlighter = SimpleCodeHighlighter(testColors) + + private suspend fun highlight(code: String, language: String = "json") = + highlighter.highlight(code, language).first() + + @Test + fun `name and all aliases are recognized`() = runTest { + val identifiers = + listOf( + "json", + "4dform", + "4dproject", + "avsc", + "bowerrc", + "cssmap", + "geojson", + "gltf", + "har", + "ice", + "ipynb", + "jscsrc", + "jslintrc", + "jsmap", + "json.example", + "json-tmlanguage", + "jsonl", + "jsonld", + "mcmeta", + "sarif", + "slnlaunch", + "tact", + "tfstate", + "tfstate.backup", + "topojson", + "tsmap", + "vuerc", + "webapp", + "webmanifest", + "yy", + "yyp", + ) + for (identifier in identifiers) { + assertTrue( + highlight("""{"a": 1}""", identifier).spanStyles.isNotEmpty(), + "Alias '$identifier' not recognized", + ) + } + } + + @Test + fun `language tags are matched case-insensitively`() = runTest { + for (tag in listOf("JSON", "Json", "GeoJSON")) { + assertTrue(highlight("""{"a": 1}""", tag).spanStyles.isNotEmpty(), "Tag '$tag' not recognized") + } + } + + @Test + fun `line comment is colored as comment`() = runTest { + assertEquals(testColors.comment, highlight("// a comment").colorAt(0)) + } + + @Test + fun `block comment is colored as comment`() = runTest { + assertEquals(testColors.comment, highlight("/* block */").colorAt(0)) + } + + @Test + fun `doc-style block comment is colored as comment`() = runTest { + assertEquals(testColors.comment, highlight("/** doc */").colorAt(0)) + } + + @Test + fun `comments are italic`() = runTest { + assertEquals(FontStyle.Italic, highlight("// comment").spansAt(0).first().fontStyle) + } + + @Test + fun `constant inside comment is not colored as constant`() = runTest { + val result = highlight("// true") + assertEquals(testColors.comment, result.colorAt(3)) + assertNotEquals(testColors.constant, result.colorAt(3)) + } + + @Test + fun `object key is colored as property key`() = runTest { + assertEquals(testColors.propertyKey, highlight("""{"a": 1}""").colorAt(1)) + } + + @Test + fun `string value is colored as string`() = runTest { + assertEquals(testColors.string, highlight("""{"a": "b"}""").colorAt(6)) + } + + @Test + fun `key and value with identical text are colored differently`() = runTest { + val result = highlight("""{"a": "a"}""") + assertEquals(testColors.propertyKey, result.colorAt(1), "Key should be a property key") + assertEquals(testColors.string, result.colorAt(6), "Value should be a string") + } + + @Test + fun `colon is not part of the key span`() = runTest { + // The key rule's lookahead is zero-width, so the match ends at the closing quote (index 3) + assertNull(highlight("""{"a": 1}""").colorAt(4)) + } + + @Test + fun `whitespace between key and colon is allowed`() = runTest { + assertEquals(testColors.propertyKey, highlight("""{"a" : 1}""").colorAt(1)) + } + + @Test + fun `key that looks like a language constant is still a key`() = runTest { + val result = highlight("""{"true": 1}""") + assertEquals(testColors.propertyKey, result.colorAt(2)) + assertNotEquals(testColors.constant, result.colorAt(2)) + } + + @Test + fun `escaped quote does not end the key early`() = runTest { + // Key is `a\":b`, so the quote at index 4 must not terminate it + val result = highlight("""{"a\":b": 1}""") + assertEquals(testColors.propertyKey, result.colorAt(6)) + assertEquals(testColors.number, result.colorAt(10)) + } + + @Test + fun `string in an array is not treated as a key`() = runTest { + assertEquals(testColors.string, highlight("""["a", "b"]""").colorAt(1)) + } + + @Test + fun `language constants are colored as constant`() = runTest { + for (constant in listOf("true", "false", "null")) { + assertEquals( + testColors.constant, + highlight(constant).colorAt(0), + "'$constant' should be colored as constant", + ) + } + } + + @Test + fun `constant inside string is not colored as constant`() = runTest { + val result = highlight("""{"a": "true"}""") + assertEquals(testColors.string, result.colorAt(7)) + assertNotEquals(testColors.constant, result.colorAt(7)) + } + + @Test + fun `numbers are colored as number`() = runTest { + for (number in listOf("0", "42", "-1", "3.14", "1e10", "-1.5e+10")) { + assertEquals(testColors.number, highlight(number).colorAt(0), "'$number' should be colored as number") + } + } + + @Test + fun `numbers in arrays are colored as number`() = runTest { + // Numbers are not gated on a preceding colon + val result = highlight("[1, 2, 3]") + assertEquals(testColors.number, result.colorAt(1)) + assertEquals(testColors.number, result.colorAt(4)) + } +} diff --git a/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/JSXGrammarTest.kt b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/JSXGrammarTest.kt new file mode 100644 index 000000000000..dc5e5261619d --- /dev/null +++ b/platform/jewel/int-ui/int-ui-standalone-tests/src/test/kotlin/org/jetbrains/jewel/intui/code/highlighting/languages/JSXGrammarTest.kt @@ -0,0 +1,152 @@ +// Copyright 2000-2026 JetBrains s.r.o. and contributors. Use of this source code is governed by the Apache 2.0 license. +package org.jetbrains.jewel.intui.code.highlighting.languages + +import kotlin.test.assertEquals +import kotlin.test.assertNotEquals +import kotlin.test.assertNull +import kotlin.test.assertTrue +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.test.runTest +import org.jetbrains.jewel.intui.code.highlighting.colorAt +import org.jetbrains.jewel.intui.code.highlighting.testColors +import org.jetbrains.jewel.intui.standalone.code.highlighting.SimpleCodeHighlighter +import org.junit.jupiter.api.Test + +internal class JSXGrammarTest { + private val highlighter = SimpleCodeHighlighter(testColors) + + private suspend fun highlight(code: String, language: String = "jsx") = + highlighter.highlight(code, language).first() + + @Test + fun `name and all aliases are recognized`() = runTest { + for (identifier in listOf("jsx", "javascriptreact")) { + assertTrue(highlight("let x = 1", identifier).spanStyles.isNotEmpty(), "Alias '$identifier' not recognized") + } + } + + @Test + fun `the javascript grammar does not highlight jsx tags`() = runTest { + // Pins the split from the other side: JAVASCRIPT is registered first, so if it ever reclaims the + // "jsx" alias the tag tests below start resolving to this (unstyled) grammar instead. + assertNull(highlight("
", "javascript").colorAt(1)) + } + + @Test + fun `lowercase tag name is colored as keyword`() = runTest { + assertEquals(testColors.keyword, highlight("
").colorAt(1)) + } + + @Test + fun `closing tag name is colored as keyword`() = runTest { + // "div" starts at index 2 in "
" + assertEquals(testColors.keyword, highlight("
").colorAt(2)) + } + + @Test + fun `capitalized tag name is colored as type`() = runTest { + val result = highlight("") + assertEquals(testColors.type, result.colorAt(1)) + assertNotEquals(testColors.keyword, result.colorAt(1), "Components should not read as DOM elements") + } + + @Test + fun `dotted component name is colored as type`() = runTest { + assertEquals(testColors.type, highlight("").colorAt(1)) + } + + @Test + fun `attribute name is colored as property key`() = runTest { + // "className" starts at index 5 in `
` + assertEquals(testColors.propertyKey, highlight("
").colorAt(5)) + } + + @Test + fun `attribute value is colored as string`() = runTest { + // The opening quote is at index 15 in `
` + assertEquals(testColors.string, highlight("
").colorAt(15)) + } + + @Test + fun `attribute with an expression value is colored as property key`() = runTest { + // "onClick" starts at index 5 in "
" + assertEquals(testColors.propertyKey, highlight("
").colorAt(5)) + } + + @Test + fun `attribute rule does not fire on javascript assignments`() = runTest { + // Spacing is a convention, not a syntactic boundary, so the rule is anchored to opening-tag + // context instead: an unclosed `<` in expression position, a tag name, then whitespace. + assertNull(highlight("const x = \"y\"").colorAt(6), "spaced assignment") + assertNull(highlight("const x=\"y\";").colorAt(6), "compact assignment") + assertNull(highlight("let a=\"b\", c=\"d\";").colorAt(4), "compact let") + assertNull(highlight("let a=\"b\", c=\"d\";").colorAt(11), "second compact assignment") + assertNull(highlight("obj={k:\"v\"};").colorAt(0), "object literal") + assertNull(highlight("foo.bar=\"baz\";").colorAt(0), "member assignment") + } + + @Test + fun `a comparison operator does not open tag context`() = runTest { + assertNull(highlight("if (a").colorAt(15), "second attr") + assertEquals(testColors.propertyKey, highlight("return
;").colorAt(12), "after return") + assertEquals(testColors.propertyKey, highlight("").colorAt(5), "brace value") + } + + @Test + fun `html entity is colored as constant`() = runTest { + for (entity in listOf(" ", " ", " ")) { + assertEquals(testColors.constant, highlight(entity).colorAt(0), "'$entity' should be colored as constant") + } + } + + @Test fun `logical and is not mistaken for an entity`() = runTest { assertNull(highlight("a && b").colorAt(2)) } + + @Test + fun `less-than in an expression is not mistaken for a tag`() = runTest { + // "b" is at index 6 in "if (a" + assertEquals(testColors.comment, highlight("//
").colorAt(3)) + } + + @Test + fun `tag inside a string is not colored as a tag`() = runTest { + // "<" is at index 1 in "\"
\"" + assertEquals(testColors.string, highlight("\"
\"").colorAt(1)) + } + + @Test + fun `attribute rule wins over the javascript keyword rule`() = runTest { + // "for" is a JS keyword, but inside a tag it is an attribute name + assertEquals(testColors.propertyKey, highlight("