feat(user): define username format and mention syntax (#6134)
This commit is contained in:
parent
464e45b447
commit
8648b97b9e
47 changed files with 1160 additions and 420 deletions
260
docs/adr/0002-username-format-and-references.md
Normal file
260
docs/adr/0002-username-format-and-references.md
Normal file
|
|
@ -0,0 +1,260 @@
|
|||
# ADR 0002: Username Format and References
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Date: 2026-08-02
|
||||
|
||||
Domain glossary: [Memos domain glossary](../glossary.md)
|
||||
|
||||
## Context
|
||||
|
||||
A username is a user identifier used by account creation and updates, authentication, public user resource names, API lookup, SSO provisioning, and
|
||||
references to users. Those uses must share one format rather than defining separate username-like tokens.
|
||||
|
||||
Memos already restricts newly written usernames to ASCII letters, digits, and hyphens, with a 36-character limit and alphanumeric ends. Existing
|
||||
installations may also contain older usernames such as email addresses or values with underscores. Memos continues to read and resolve those legacy
|
||||
values, but they are not valid new usernames.
|
||||
|
||||
Markdown mentions currently drift from the writable username format: the backend accepts up to 63 Unicode letters, numbers, and hyphens; the frontend
|
||||
accepts up to 63 ASCII characters; both allow invalid username shapes; and extraction lowercases the result. Editor decoration also scans raw source in
|
||||
contexts where Markdown must be opaque.
|
||||
|
||||
This ADR defines:
|
||||
|
||||
- The format and case-preservation semantics for writable usernames.
|
||||
- How username references resolve.
|
||||
- Markdown mention as one source form of username reference.
|
||||
|
||||
Changing or migrating legacy username data is outside this ADR's scope.
|
||||
Username rename and reuse policy, and the durable binding of an already resolved mention across those lifecycle events, are also outside scope.
|
||||
Notification timing, deduplication, access policy, self-mention behavior, and other downstream effects are unchanged and outside scope.
|
||||
|
||||
## Decision drivers
|
||||
|
||||
- Give every username write path one small, stable validation rule.
|
||||
- Make a reference use the username format instead of inventing another identifier grammar.
|
||||
- Preserve the spelling and case selected by the user.
|
||||
- Keep username write validation, reference rendering, extraction, editor recognition, and notification resolution aligned.
|
||||
- Use parsed Markdown context rather than special-case regular expressions for code, links, email addresses, and other opaque syntax.
|
||||
- Keep recognition deterministic and linear in memo size.
|
||||
|
||||
## Terminology
|
||||
|
||||
**User**
|
||||
: A durable Memos account identified internally by an immutable user ID.
|
||||
|
||||
**Username**
|
||||
: A user-selected, case-sensitive public identifier used for account addressing. A username is distinct from a display name and from the user's internal
|
||||
ID.
|
||||
|
||||
**Writable username**
|
||||
: A username accepted when an account is created, renamed, or provisioned after this decision.
|
||||
|
||||
**Legacy username**
|
||||
: A stored username that does not satisfy the writable format but remains readable and addressable for compatibility.
|
||||
|
||||
**Username reference**
|
||||
: Source text that names a username so Memos can attempt to resolve it to a user ID. The source spelling is not itself a durable relationship.
|
||||
|
||||
**Mention candidate**
|
||||
: A Markdown source span with the form `@` followed by a complete writable username and satisfying the mention boundary and context rules.
|
||||
|
||||
**Resolved mention**
|
||||
: A mention candidate whose username resolves to a user under the consuming operation's existing account-status and visibility policy.
|
||||
|
||||
## Decision
|
||||
|
||||
### Writable username format
|
||||
|
||||
The normative grammar is:
|
||||
|
||||
```text
|
||||
Username := Alphanumeric
|
||||
| Alphanumeric UsernameCharacter{0,34} Alphanumeric
|
||||
UsernameCharacter := Alphanumeric | "-"
|
||||
Alphanumeric := ASCII letter | ASCII digit
|
||||
ASCII letter := "A".."Z" | "a".."z"
|
||||
ASCII digit := "0".."9"
|
||||
```
|
||||
|
||||
- A username is 1 through 36 ASCII characters.
|
||||
- ASCII letters, digits, and `-` are the only admitted characters.
|
||||
- The first and last characters are letters or digits.
|
||||
- Consecutive interior hyphens are valid; there is no additional rule limiting their number.
|
||||
- A digits-only value is valid.
|
||||
- Case is preserved.
|
||||
|
||||
Requiring alphanumeric ends also excludes values made only of hyphens and prevents a trailing hyphen from becoming an ambiguous-looking part of a
|
||||
Markdown mention.
|
||||
|
||||
The minimum is one because Username needs no product-level namespace-scarcity policy beyond being non-empty. The maximum is 36 so every UUID value has a
|
||||
canonical textual representation that fits while public URLs and reference tokens remain bounded.
|
||||
|
||||
The ASCII-only restriction is intentional: Username is a machine-facing public identifier used in URLs, authentication, and references. Internationalized
|
||||
human-readable naming belongs to Display Name rather than expanding Username normalization and comparison rules.
|
||||
|
||||
Examples:
|
||||
|
||||
| Value | Writable | Reason |
|
||||
| --- | --- | --- |
|
||||
| `alice` | yes | Letters only |
|
||||
| `Alice-2` | yes | Case is preserved; interior hyphen is valid |
|
||||
| `1alice` | yes | A digit may begin a username that also contains a letter |
|
||||
| `a--b` | yes | Consecutive interior hyphens are valid |
|
||||
| `a---b` | yes | Interior hyphens have no repetition limit |
|
||||
| `123` | yes | Numeric spelling has no special meaning |
|
||||
| `123-456` | yes | Digits and interior hyphens satisfy the grammar |
|
||||
| `00000000-0000-0000-0000-000000000000` | yes | A canonical UUID occupies 36 characters |
|
||||
| `-alice` | no | A hyphen cannot begin a username |
|
||||
| `alice-` | no | A hyphen cannot end a username |
|
||||
| `alice_smith` | no | `_` is outside the format |
|
||||
| `alice@example.com` | no | Email syntax is outside the format |
|
||||
| `álîçé` | no | The writable format is ASCII |
|
||||
| `张三` | no | Internationalized naming belongs to Display Name |
|
||||
|
||||
All account creation, username update, and automatic provisioning paths use this rule. A generic resource-name rule is not the username contract and
|
||||
must not be reused as a substitute merely because its current regular expression is similar.
|
||||
|
||||
### Username allocation
|
||||
|
||||
The writable format has no reserved spellings. Values such as `admin`, `root`, `system`, `api`, `memos`, and `support` are ordinary usernames and are
|
||||
available subject to the same exact-case uniqueness rule as every other value. A username does not grant authority or establish trust; those properties
|
||||
come from the resolved user ID, role, and explicit product presentation. Manual registration, rename, and automatic provisioning apply the same policy.
|
||||
|
||||
If Memos later needs a system-owned username, the corresponding account claims it through ordinary uniqueness rather than expanding a lexical reserved
|
||||
list.
|
||||
|
||||
SSO provisioning adopts the external identifier exactly when it is a valid, available writable username. If it is invalid or already belongs to another
|
||||
user, provisioning assigns a generated canonical UUID instead. A matching username never links or takes over an account; only the persisted identity
|
||||
binding formed by the identity-provider key and external subject identifies a returning SSO user.
|
||||
|
||||
### Username identity and resolution
|
||||
|
||||
Username spelling is case-preserving and username equality is exact ASCII byte equality. Memos does not lowercase, case-fold, or otherwise normalize a
|
||||
username before validation, uniqueness checks, authentication, resource lookup, or reference resolution. In particular, surrounding whitespace is not
|
||||
trimmed into a different username; a value such as ` alice ` is invalid. `Alice` and `alice` are distinct usernames and may identify different users.
|
||||
|
||||
All storage backends and application lookup paths must preserve this equality rule. A database collation that treats `Alice` and `alice` as equal, or a
|
||||
lookup that silently resolves one spelling to the other, does not conform to this decision.
|
||||
|
||||
The `username` column therefore uses an explicit case-sensitive binary collation on every supported storage backend: `utf8mb4_bin` on MySQL, `C` on
|
||||
PostgreSQL, and `BINARY` on SQLite. For the writable ASCII format, this makes exact-byte uniqueness and lookup a schema property rather than an assumption
|
||||
about each database's default collation.
|
||||
|
||||
A username-bearing field or resource-name segment always resolves as a username, including when its spelling contains only digits. Character shape must
|
||||
not select the internal user-ID namespace; an operation that addresses a user by internal ID must expose that choice separately.
|
||||
|
||||
The stable identity of a user is the internal user ID. A username reference is resolved as late as the consuming operation permits:
|
||||
|
||||
1. Preserve the username exactly as written.
|
||||
2. Look up that exact username in the operation's visibility and account-status scope.
|
||||
3. If it resolves, use the resulting user ID for durable effects.
|
||||
4. If it does not resolve, retain ordinary source text and produce no user-targeted effect.
|
||||
|
||||
Product intent permits username rename and later reuse while requiring an already resolved mention not to change users. A source spelling alone cannot
|
||||
satisfy all three properties: preserving the original target requires a durable binding to the user ID outside the `@username` text. This ADR does not
|
||||
define that binding, historical re-resolution behavior, or the username lifecycle needed to support it.
|
||||
|
||||
Legacy usernames remain valid exact lookup keys for existing API resource names and authentication flows. They are not accepted by write validation and
|
||||
are not added to new reference syntaxes. Changing legacy username values remains outside this decision, but every lookup preserves their stored spelling.
|
||||
|
||||
### Markdown mention syntax
|
||||
|
||||
A Markdown mention is the first defined username-reference syntax:
|
||||
|
||||
```text
|
||||
MentionCandidate := "@" Username
|
||||
```
|
||||
|
||||
The `Username` production is the complete writable username format above. Mention parsing has no independent character set or length limit.
|
||||
|
||||
Candidate recognition follows these rules:
|
||||
|
||||
1. The `@` is U+0040 COMMERCIAL AT in an eligible literal-source run.
|
||||
2. At the start of the document, the left boundary is satisfied. Otherwise the immediately preceding source character must not be a
|
||||
`UsernameCharacter`. The username format therefore defines the boundary: a preceding letter, digit, or hyphen blocks recognition.
|
||||
3. The lexer consumes the complete consecutive run of ASCII letters, digits, and hyphens after `@`, then validates that whole run as a writable username.
|
||||
It does not shorten an invalid run into a valid prefix.
|
||||
4. GFM email-address recognition takes precedence. No substring of a recognized email address is a mention.
|
||||
|
||||
Characters outside `UsernameCharacter`, including `_`, `@`, and non-ASCII characters, are ordinary boundaries and receive no special handling.
|
||||
|
||||
The rules produce:
|
||||
|
||||
| Source | Mention candidate |
|
||||
| --- | --- |
|
||||
| `@alice` | `alice` |
|
||||
| `@Alice-2` | `Alice-2` |
|
||||
| `hi, @alice.` | `alice` |
|
||||
| `中文@alice` | `alice` |
|
||||
| `hello@alice` | none |
|
||||
| `foo-@alice` | none |
|
||||
| `foo_@alice` | `alice` |
|
||||
| `@123` | `123` |
|
||||
| `@-alice` | none |
|
||||
| `@alice-` | none |
|
||||
| `@alice_smith` | `alice` |
|
||||
| `@alice@example` | `alice` |
|
||||
| `@alice@bob` | `alice` |
|
||||
| `@alice@example.com` | none |
|
||||
| `@` followed by 37 valid username characters | none |
|
||||
|
||||
### Markdown context
|
||||
|
||||
Mention recognition operates on original Markdown source after GFM and Memos block and inline structure is resolved. It uses the same literal-source-run
|
||||
and opaque-node model as other Memos inline extensions:
|
||||
|
||||
- Paragraphs, headings, block quotes, list items, table text, emphasis, strong emphasis, and strikethrough expose ordinary text.
|
||||
- Code spans, code blocks, resolved links and images, autolinks, recognized GFM email addresses, raw HTML, and math are opaque.
|
||||
- Escapes, character references, line endings, and syntax delimiters end a literal-source run. An escaped or character-reference-produced `@` is not an
|
||||
introducer.
|
||||
- Unknown future Markdown extension nodes are opaque unless their definition explicitly exposes ordinary text.
|
||||
|
||||
Parsing, read rendering, metadata extraction, editor decoration, and any future mention completion must use these same lexical and context rules.
|
||||
|
||||
This decision changes only which exact username spellings are extracted as mention candidates. Existing resolution, rendering, notification, and access
|
||||
policies consume those candidates without otherwise changing behavior. Autocomplete, suggestion ranking, and a dedicated structured mention payload are
|
||||
also outside this decision.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Username validation becomes a domain primitive shared by every write path.
|
||||
- Username uniqueness, authentication, resource lookup, and reference resolution use the same case-sensitive equality rule on every storage backend.
|
||||
- Mention syntax automatically follows future username-format decisions only when this ADR is superseded and both uses are updated together.
|
||||
- Existing valid mixed-case usernames can be mentioned without lowercasing.
|
||||
- Values accepted only by the old 63-character or Unicode mention lexers stop being mention candidates because they were never writable usernames.
|
||||
- Legacy email-like or underscore-containing usernames remain usable through compatibility lookup but cannot be authored as new Markdown mentions.
|
||||
- Markdown-aware recognition removes false mentions from code, links, email addresses, HTML, math, escapes, and character references.
|
||||
- Username rename, reuse, and historical mention stability remain an explicit follow-up because source text alone cannot represent the intended stable
|
||||
user binding.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
### Define mention syntax first
|
||||
|
||||
Rejected because it creates a second username-like grammar and allows account validation and references to drift again.
|
||||
|
||||
### Allow a broader mention token than writable usernames
|
||||
|
||||
Rejected because such candidates cannot reliably identify newly writable accounts and complicate boundaries, validation, and UI messaging.
|
||||
|
||||
### Store stable user IDs with a mention
|
||||
|
||||
Deferred. Stable IDs make renames and username reuse robust but require either a new source representation or persisted occurrence bindings, plus editing
|
||||
UX, import/export behavior, and compatibility policy. Exact username references are sufficient for candidate recognition in the current scope, but do not
|
||||
solve historical binding.
|
||||
|
||||
### Unicode usernames
|
||||
|
||||
Deferred. A safe Unicode identifier design needs a pinned Unicode version, normalization, script and confusable policy, and consistent database collation.
|
||||
ASCII keeps this decision small and matches the existing writable format.
|
||||
|
||||
## References
|
||||
|
||||
- [GitHub Flavored Markdown 0.29-gfm](https://github.github.com/gfm/)
|
||||
- [GitHub: Mentioning people and teams](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams)
|
||||
- [Slack message formatting: mentioning users](https://docs.slack.dev/messaging/formatting-message-text/#mentioning-users)
|
||||
- [Matrix client-server API: user and room mentions](https://spec.matrix.org/latest/client-server-api/#user-and-room-mentions)
|
||||
- [Activity Streams 2.0 Vocabulary: Mention](https://www.w3.org/TR/activitystreams-vocabulary/#dfn-mention)
|
||||
- [Unicode Standard Annex #31: Unicode Identifiers and Syntax](https://www.unicode.org/reports/tr31/)
|
||||
- [Unicode Technical Standard #39: Unicode Security Mechanisms](https://www.unicode.org/reports/tr39/)
|
||||
|
|
@ -8,6 +8,7 @@ implementation work is complete.
|
|||
| ADR | Title | Status | Date |
|
||||
| --- | --- | --- | --- |
|
||||
| [0001](0001-tag-syntax-and-recognition.md) | Tag Syntax and Recognition | Accepted | 2026-08-01 |
|
||||
| [0002](0002-username-format-and-references.md) | Username Format and References | Accepted | 2026-08-02 |
|
||||
|
||||
## Conventions
|
||||
|
||||
|
|
|
|||
|
|
@ -3,6 +3,47 @@
|
|||
This glossary defines product and domain language shared across Memos design documents. Protocol, Unicode, and parser-specific terms remain in the ADRs
|
||||
that use them.
|
||||
|
||||
## Users and usernames
|
||||
|
||||
### User
|
||||
|
||||
A durable Memos account. Its stable identity is an internal user ID, not its mutable username or display name.
|
||||
|
||||
### Username
|
||||
|
||||
A user-selected, case-sensitive public account identifier used for authentication, user resource names, lookup, and references. Username spelling is
|
||||
case-preserving, is not implicitly trimmed or normalized, and equality is exact ASCII byte equality, so `Alice` and `alice` are distinct usernames. A
|
||||
username is distinct from both a display name and the user's internal ID. Its spelling does not grant a role or establish that the user is trusted.
|
||||
|
||||
### Writable username
|
||||
|
||||
A username accepted for account creation, rename, or automatic provisioning. It is 1 through 36 ASCII letters, digits, or hyphens; begins and ends with an
|
||||
ASCII letter or digit; and may contain repeated interior hyphens without an additional restriction. Numeric spelling has no special meaning, so a value
|
||||
containing only digits is also a writable username. Internationalized human-readable naming belongs to the user's display name.
|
||||
|
||||
### Legacy username
|
||||
|
||||
A stored username that does not satisfy the writable format but remains readable and addressable for compatibility. Legacy usernames are not accepted by
|
||||
new write validation or added to new reference syntaxes.
|
||||
|
||||
### Username reference
|
||||
|
||||
Source text that names an exact, case-sensitive username so Memos can attempt to resolve it to a user ID. The source spelling alone is not a durable user
|
||||
binding. Stability across username rename and reuse requires additional persisted identity that the current username-reference specification does not
|
||||
define.
|
||||
|
||||
### Mention candidate
|
||||
|
||||
An eligible memo Markdown source span formed by an ASCII `@` followed by one complete writable username. Recognition also applies the boundary and opaque
|
||||
Markdown-context rules in the active username-reference specification. The same ASCII letters, digits, and hyphen admitted inside a username define the
|
||||
mention boundaries: they block a mention immediately to their right and are consumed as part of the complete candidate to their left. Other characters
|
||||
receive no special boundary handling.
|
||||
|
||||
### Resolved mention
|
||||
|
||||
A mention candidate whose exact username resolves to a user under the consuming operation's existing account-status and visibility policy. User-targeted
|
||||
effects apply to the resolved user ID, not to unresolved source text.
|
||||
|
||||
## Tags
|
||||
|
||||
### Tag
|
||||
|
|
|
|||
35
internal/base/username.go
Normal file
35
internal/base/username.go
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
package base
|
||||
|
||||
// MaxUsernameLength is the maximum number of ASCII characters in a writable username.
|
||||
const MaxUsernameLength = 36
|
||||
|
||||
// IsValidUsername reports whether username satisfies the writable username format.
|
||||
func IsValidUsername(username string) bool {
|
||||
if len(username) == 0 || len(username) > MaxUsernameLength || !isASCIIAlphanumeric(username[0]) || !isASCIIAlphanumeric(username[len(username)-1]) {
|
||||
return false
|
||||
}
|
||||
|
||||
for i := 0; i < len(username); i++ {
|
||||
if !IsUsernameCharacter(username[i]) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// IsUsernameCharacter reports whether an ASCII byte may occur in a writable username.
|
||||
func IsUsernameCharacter(char byte) bool {
|
||||
return isASCIIAlphanumeric(char) || char == '-'
|
||||
}
|
||||
|
||||
func isASCIIAlphanumeric(char byte) bool {
|
||||
return isASCIILetter(char) || isASCIIDigit(char)
|
||||
}
|
||||
|
||||
func isASCIILetter(char byte) bool {
|
||||
return char >= 'A' && char <= 'Z' || char >= 'a' && char <= 'z'
|
||||
}
|
||||
|
||||
func isASCIIDigit(char byte) bool {
|
||||
return char >= '0' && char <= '9'
|
||||
}
|
||||
43
internal/base/username_test.go
Normal file
43
internal/base/username_test.go
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
package base
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIsValidUsername(t *testing.T) {
|
||||
tests := []struct {
|
||||
username string
|
||||
want bool
|
||||
}{
|
||||
{username: "alice", want: true},
|
||||
{username: "Alice-2", want: true},
|
||||
{username: "1alice", want: true},
|
||||
{username: "a--b", want: true},
|
||||
{username: "a---b", want: true},
|
||||
{username: "a" + strings.Repeat("b", MaxUsernameLength-1), want: true},
|
||||
{username: ""},
|
||||
{username: strings.Repeat("a", MaxUsernameLength+1)},
|
||||
{username: "123", want: true},
|
||||
{username: "123-456", want: true},
|
||||
{username: "00000000-0000-0000-0000-000000000000", want: true},
|
||||
{username: "-"},
|
||||
{username: "---"},
|
||||
{username: "-alice"},
|
||||
{username: "alice-"},
|
||||
{username: "alice_smith"},
|
||||
{username: "alice@example.com"},
|
||||
{username: " alice "},
|
||||
{username: "alice smith"},
|
||||
{username: "álîçé"},
|
||||
{username: "张三"},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.username, func(t *testing.T) {
|
||||
if got := IsValidUsername(test.username); got != test.want {
|
||||
t.Fatalf("IsValidUsername(%q) = %t, want %t", test.username, got, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -10,6 +10,8 @@ type MentionNode struct {
|
|||
|
||||
// Username without the @ prefix.
|
||||
Username []byte
|
||||
// Source is the exact recognized source spelling, including the @ prefix.
|
||||
Source []byte
|
||||
}
|
||||
|
||||
// KindMention is the NodeKind for MentionNode.
|
||||
|
|
@ -24,5 +26,6 @@ func (*MentionNode) Kind() gast.NodeKind {
|
|||
func (n *MentionNode) Dump(source []byte, level int) {
|
||||
gast.DumpHelper(n, source, level, map[string]string{
|
||||
"Username": string(n.Username),
|
||||
"Source": string(n.Source),
|
||||
}, nil)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,23 +2,103 @@ package extensions
|
|||
|
||||
import (
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/ast"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
"github.com/yuin/goldmark/renderer"
|
||||
"github.com/yuin/goldmark/text"
|
||||
"github.com/yuin/goldmark/util"
|
||||
|
||||
"github.com/usememos/memos/internal/base"
|
||||
mast "github.com/usememos/memos/internal/markdown/ast"
|
||||
mparser "github.com/usememos/memos/internal/markdown/parser"
|
||||
)
|
||||
|
||||
type mentionExtension struct{}
|
||||
|
||||
// MentionExtension is a goldmark extension for @mention syntax.
|
||||
// MentionExtension is a goldmark extension for username references using @username syntax.
|
||||
var MentionExtension = &mentionExtension{}
|
||||
|
||||
// Extend extends the goldmark parser with mention support.
|
||||
// Extend extends the goldmark parser with username-reference support.
|
||||
func (*mentionExtension) Extend(m goldmark.Markdown) {
|
||||
m.Parser().AddOptions(
|
||||
parser.WithInlineParsers(
|
||||
// Priority 200 - run before standard link parser (500).
|
||||
util.Prioritized(mparser.NewMentionParser(), 200),
|
||||
parser.WithASTTransformers(
|
||||
// GFM email recognition runs at 950 and tag recognition at 1000.
|
||||
util.Prioritized(&mentionASTTransformer{}, 1050),
|
||||
),
|
||||
)
|
||||
m.Renderer().AddOptions(
|
||||
renderer.WithNodeRenderers(
|
||||
util.Prioritized(&mentionNodeRenderer{}, 500),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
type mentionASTTransformer struct{}
|
||||
|
||||
func (*mentionASTTransformer) Transform(document *ast.Document, reader text.Reader, _ parser.Context) {
|
||||
source := reader.Source()
|
||||
textNodes := eligibleLiteralTextNodes(document)
|
||||
mergeAdjacentLiteralText(textNodes, source)
|
||||
for _, textNode := range textNodes {
|
||||
if textNode.Parent() == nil {
|
||||
continue
|
||||
}
|
||||
replaceMentionsInText(textNode, source)
|
||||
}
|
||||
}
|
||||
|
||||
func replaceMentionsInText(textNode *ast.Text, source []byte) {
|
||||
segment := textNode.Segment
|
||||
runHasLeftBoundary := segment.Padding > 0 || segment.Start == 0 || !base.IsUsernameCharacter(source[segment.Start-1])
|
||||
matches := mparser.FindMentionMatches(segment.Value(source), runHasLeftBoundary)
|
||||
if len(matches) == 0 {
|
||||
return
|
||||
}
|
||||
|
||||
parent := textNode.Parent()
|
||||
cursor := segment.Start
|
||||
padding := segment.Padding
|
||||
for _, match := range matches {
|
||||
start := segment.Start + match.Start - segment.Padding
|
||||
end := segment.Start + match.End - segment.Padding
|
||||
if start > cursor || padding > 0 {
|
||||
insertSplitTextBefore(parent, textNode, textNode, text.NewSegmentPadding(cursor, start, padding), false)
|
||||
}
|
||||
|
||||
mentionNode := &mast.MentionNode{
|
||||
Username: append([]byte(nil), match.Username...),
|
||||
Source: append([]byte(nil), source[start:end]...),
|
||||
}
|
||||
mentionNode.SetPos(start)
|
||||
parent.InsertBefore(parent, textNode, mentionNode)
|
||||
cursor = end
|
||||
padding = 0
|
||||
}
|
||||
|
||||
if cursor < segment.Stop || textNode.SoftLineBreak() || textNode.HardLineBreak() {
|
||||
insertSplitTextBefore(parent, textNode, textNode, text.NewSegment(cursor, segment.Stop), true)
|
||||
}
|
||||
parent.RemoveChild(parent, textNode)
|
||||
}
|
||||
|
||||
type mentionNodeRenderer struct{}
|
||||
|
||||
func (*mentionNodeRenderer) RegisterFuncs(registerer renderer.NodeRendererFuncRegisterer) {
|
||||
registerer.Register(mast.KindMention, renderMentionNode)
|
||||
}
|
||||
|
||||
func renderMentionNode(writer util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) {
|
||||
if !entering {
|
||||
return ast.WalkContinue, nil
|
||||
}
|
||||
mentionNode, ok := node.(*mast.MentionNode)
|
||||
if !ok {
|
||||
return ast.WalkContinue, nil
|
||||
}
|
||||
spelling := mentionNode.Source
|
||||
if len(spelling) == 0 {
|
||||
spelling = append([]byte{'@'}, mentionNode.Username...)
|
||||
}
|
||||
_, _ = writer.Write(util.EscapeHTML(spelling))
|
||||
return ast.WalkContinue, nil
|
||||
}
|
||||
|
|
|
|||
25
internal/markdown/extensions/mention_test.go
Normal file
25
internal/markdown/extensions/mention_test.go
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
package extensions
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"github.com/yuin/goldmark/ast"
|
||||
"github.com/yuin/goldmark/text"
|
||||
|
||||
mast "github.com/usememos/memos/internal/markdown/ast"
|
||||
)
|
||||
|
||||
func TestReplaceMentionsInPaddedText(t *testing.T) {
|
||||
source := []byte("x@alice")
|
||||
paragraph := ast.NewParagraph()
|
||||
textNode := ast.NewTextSegment(text.NewSegmentPadding(1, len(source), 2))
|
||||
paragraph.AppendChild(paragraph, textNode)
|
||||
|
||||
replaceMentionsInText(textNode, source)
|
||||
|
||||
mention, ok := paragraph.LastChild().(*mast.MentionNode)
|
||||
require.True(t, ok)
|
||||
assert.Equal(t, "alice", string(mention.Username))
|
||||
}
|
||||
|
|
@ -418,7 +418,7 @@ func (s *service) ExtractAll(content []byte) (*ExtractedData, error) {
|
|||
data.Tags = appendTagHierarchy(data.Tags, string(tagNode.Tag))
|
||||
}
|
||||
if mentionNode, ok := n.(*mast.MentionNode); ok {
|
||||
data.Mentions = append(data.Mentions, strings.ToLower(string(mentionNode.Username)))
|
||||
data.Mentions = append(data.Mentions, string(mentionNode.Username))
|
||||
}
|
||||
|
||||
// Check if the first block-level child of the document is an H1 heading.
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@ import (
|
|||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"github.com/usememos/memos/internal/base"
|
||||
)
|
||||
|
||||
func TestNewService(t *testing.T) {
|
||||
|
|
@ -346,10 +348,10 @@ func TestExtractAllMentions(t *testing.T) {
|
|||
|
||||
data, err := svc.ExtractAll([]byte("Hi @Alice and @bob. Email support@example.com should stay plain. #tag"))
|
||||
require.NoError(t, err)
|
||||
assert.ElementsMatch(t, []string{"alice", "bob"}, data.Mentions)
|
||||
assert.ElementsMatch(t, []string{"Alice", "bob"}, data.Mentions)
|
||||
assert.ElementsMatch(t, []string{"tag"}, data.Tags)
|
||||
|
||||
maxLengthUsername := "a" + strings.Repeat("b", 62)
|
||||
maxLengthUsername := "a" + strings.Repeat("b", base.MaxUsernameLength-1)
|
||||
data, err = svc.ExtractAll([]byte("@" + maxLengthUsername))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, []string{maxLengthUsername}, data.Mentions)
|
||||
|
|
@ -359,6 +361,47 @@ func TestExtractAllMentions(t *testing.T) {
|
|||
assert.Empty(t, data.Mentions)
|
||||
}
|
||||
|
||||
func TestExtractAllMentionSyntaxAndContexts(t *testing.T) {
|
||||
svc := NewService(WithTagExtension(), WithMentionExtension())
|
||||
tests := []struct {
|
||||
name string
|
||||
content string
|
||||
expected []string
|
||||
}{
|
||||
{name: "writable usernames", content: "@alice @Alice-2 @1alice @a--b @123 @123-456", expected: []string{"alice", "Alice-2", "1alice", "a--b", "123", "123-456"}},
|
||||
{name: "invalid username shapes", content: "@-alice @alice- @álîçé"},
|
||||
{name: "left boundaries", content: "hello@alice foo-@bob foo_@carol 中文@dave (@erin)", expected: []string{"carol", "dave", "erin"}},
|
||||
{name: "non-username right boundaries", content: "@alice_smith @bob@carol", expected: []string{"alice", "bob"}},
|
||||
{name: "formatted text", content: "**@Alice** ~~@bob~~ _@carol_", expected: []string{"Alice", "bob", "carol"}},
|
||||
{name: "opaque Markdown", content: "`@code` [@link](/x)  https://example.com/@url $@math$ @ok", expected: []string{"ok"}},
|
||||
{name: "escapes and references", content: `\@escaped @entity @ok`, expected: []string{"ok"}},
|
||||
{name: "GFM email precedence", content: "@alice@example.com foo@bar.com@bob @ok", expected: []string{"ok"}},
|
||||
{name: "padded blockquote", content: ">\t@alice", expected: []string{"alice"}},
|
||||
{name: "padded list continuation", content: "- item\n \t@bob", expected: []string{"bob"}},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
data, err := svc.ExtractAll([]byte(test.content))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, test.expected, data.Mentions)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestMentionSourceSpelling(t *testing.T) {
|
||||
svc := NewService(WithTagExtension(), WithMentionExtension())
|
||||
content := "Hello @Alice-2"
|
||||
|
||||
rendered, err := svc.RenderMarkdown([]byte(content))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, content, rendered)
|
||||
|
||||
html, err := svc.RenderHTML([]byte(content))
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, html, "@Alice-2")
|
||||
}
|
||||
|
||||
func TestExtractAllSkipsTagsInsideLinks(t *testing.T) {
|
||||
svc := NewService(WithTagExtension())
|
||||
|
||||
|
|
|
|||
|
|
@ -1,92 +1,59 @@
|
|||
package parser
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
"github.com/yuin/goldmark/util"
|
||||
|
||||
gast "github.com/yuin/goldmark/ast"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
"github.com/yuin/goldmark/text"
|
||||
|
||||
mast "github.com/usememos/memos/internal/markdown/ast"
|
||||
"github.com/usememos/memos/internal/base"
|
||||
)
|
||||
|
||||
const (
|
||||
// MaxMentionLength matches the username token length accepted by the API.
|
||||
MaxMentionLength = 63
|
||||
)
|
||||
|
||||
type mentionParser struct{}
|
||||
|
||||
// NewMentionParser creates a new inline parser for @mention syntax.
|
||||
func NewMentionParser() parser.InlineParser {
|
||||
return &mentionParser{}
|
||||
// MentionMatch is one username reference in an eligible literal-source run.
|
||||
type MentionMatch struct {
|
||||
// Start is the byte offset of the @ introducer within the source run.
|
||||
Start int
|
||||
// End is the exclusive byte offset of the recognized source spelling.
|
||||
End int
|
||||
// Username is the exact username without the @ introducer.
|
||||
Username []byte
|
||||
}
|
||||
|
||||
// Trigger returns the characters that trigger this parser.
|
||||
func (*mentionParser) Trigger() []byte {
|
||||
return []byte{'@'}
|
||||
}
|
||||
|
||||
func isValidMentionRune(r rune) bool {
|
||||
return unicode.IsLetter(r) || unicode.IsNumber(r) || r == '-'
|
||||
}
|
||||
|
||||
func isMentionBoundary(r rune) bool {
|
||||
return unicode.IsSpace(r) || unicode.IsPunct(r) || unicode.IsSymbol(r)
|
||||
}
|
||||
|
||||
// Parse parses @mention syntax while avoiding email-address matches.
|
||||
func (*mentionParser) Parse(_ gast.Node, block text.Reader, _ parser.Context) gast.Node {
|
||||
line, segment := block.PeekLine()
|
||||
if len(line) == 0 || line[0] != '@' {
|
||||
return nil
|
||||
}
|
||||
lineStart := bytes.LastIndexByte(block.Source()[:segment.Start], '\n') + 1
|
||||
if _, _, ok := MatchGFMEmailAt(block.Source(), segment.Start, lineStart, segment.Start+len(line)); ok {
|
||||
return nil
|
||||
func hasMentionLeftBoundary(source []byte, pos int, runHasLeftBoundary bool) bool {
|
||||
if pos == 0 {
|
||||
return runHasLeftBoundary
|
||||
}
|
||||
return !base.IsUsernameCharacter(source[pos-1])
|
||||
}
|
||||
|
||||
prev := block.PrecendingCharacter()
|
||||
if prev != '\n' && !isMentionBoundary(prev) {
|
||||
return nil
|
||||
}
|
||||
|
||||
start := 1
|
||||
pos := start
|
||||
runeCount := 0
|
||||
hasLetterOrNumber := false
|
||||
|
||||
for pos < len(line) {
|
||||
r, size := utf8.DecodeRune(line[pos:])
|
||||
if r == utf8.RuneError && size == 1 {
|
||||
break
|
||||
// FindMentionMatches enumerates username references in one eligible literal-source run.
|
||||
// Markdown structure and GFM email precedence must be resolved before calling it.
|
||||
func FindMentionMatches(source []byte, runHasLeftBoundary bool) []MentionMatch {
|
||||
var matches []MentionMatch
|
||||
for pos := 0; pos < len(source); {
|
||||
if source[pos] == '\\' && pos+1 < len(source) && util.IsPunct(source[pos+1]) {
|
||||
pos += 2
|
||||
continue
|
||||
}
|
||||
if !isValidMentionRune(r) {
|
||||
break
|
||||
if source[pos] != '@' || !hasMentionLeftBoundary(source, pos, runHasLeftBoundary) {
|
||||
pos++
|
||||
continue
|
||||
}
|
||||
if unicode.IsLetter(r) || unicode.IsNumber(r) {
|
||||
hasLetterOrNumber = true
|
||||
|
||||
end := pos + 1
|
||||
for end < len(source) && base.IsUsernameCharacter(source[end]) {
|
||||
end++
|
||||
}
|
||||
runeCount++
|
||||
if runeCount > MaxMentionLength {
|
||||
return nil
|
||||
username := source[pos+1 : end]
|
||||
if base.IsValidUsername(string(username)) {
|
||||
matches = append(matches, MentionMatch{
|
||||
Start: pos,
|
||||
End: end,
|
||||
Username: append([]byte(nil), username...),
|
||||
})
|
||||
}
|
||||
if end == pos+1 {
|
||||
pos++
|
||||
} else {
|
||||
pos = end
|
||||
}
|
||||
pos += size
|
||||
}
|
||||
|
||||
if pos <= start || !hasLetterOrNumber {
|
||||
return nil
|
||||
}
|
||||
|
||||
username := line[start:pos]
|
||||
usernameCopy := make([]byte, len(username))
|
||||
copy(usernameCopy, username)
|
||||
|
||||
block.Advance(pos)
|
||||
|
||||
return &mast.MentionNode{
|
||||
Username: usernameCopy,
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
|
|
|||
42
internal/markdown/parser/mention_test.go
Normal file
42
internal/markdown/parser/mention_test.go
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
package parser
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
|
||||
"github.com/usememos/memos/internal/base"
|
||||
)
|
||||
|
||||
func TestFindMentionMatches(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
source string
|
||||
runHasLeftBoundary bool
|
||||
want []string
|
||||
}{
|
||||
{name: "valid", source: "@alice @Alice-2 @123 @123-456", runHasLeftBoundary: true, want: []string{"alice", "Alice-2", "123", "123-456"}},
|
||||
{name: "maximum length", source: "@a" + strings.Repeat("b", base.MaxUsernameLength-1), runHasLeftBoundary: true, want: []string{"a" + strings.Repeat("b", base.MaxUsernameLength-1)}},
|
||||
{name: "too long", source: "@a" + strings.Repeat("b", base.MaxUsernameLength), runHasLeftBoundary: true},
|
||||
{name: "invalid shapes", source: "@-alice @alice- @álîçé", runHasLeftBoundary: true},
|
||||
{name: "username character left boundary", source: "hello@alice foo-@bob foo_@carol", runHasLeftBoundary: true, want: []string{"carol"}},
|
||||
{name: "Unicode left boundary", source: "中文@alice", runHasLeftBoundary: true, want: []string{"alice"}},
|
||||
{name: "run starts after word", source: "@alice", runHasLeftBoundary: false},
|
||||
{name: "non-username right boundaries", source: "@alice_smith @bob@carol", runHasLeftBoundary: true, want: []string{"alice", "bob"}},
|
||||
{name: "punctuation", source: "(@alice), @bob.", runHasLeftBoundary: true, want: []string{"alice", "bob"}},
|
||||
{name: "escaped introducer", source: `\@alice @bob`, runHasLeftBoundary: true, want: []string{"bob"}},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
matches := FindMentionMatches([]byte(test.source), test.runHasLeftBoundary)
|
||||
var got []string
|
||||
for _, match := range matches {
|
||||
assert.Equal(t, "@"+string(match.Username), test.source[match.Start:match.End])
|
||||
got = append(got, string(match.Username))
|
||||
}
|
||||
assert.Equal(t, test.want, got)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
@ -195,8 +195,12 @@ func (r *MarkdownRenderer) renderNode(node gast.Node, source []byte, depth int)
|
|||
}
|
||||
|
||||
case *mast.MentionNode:
|
||||
r.buf.WriteByte('@')
|
||||
r.buf.Write(n.Username)
|
||||
if len(n.Source) > 0 {
|
||||
r.buf.Write(n.Source)
|
||||
} else {
|
||||
r.buf.WriteByte('@')
|
||||
r.buf.Write(n.Username)
|
||||
}
|
||||
|
||||
default:
|
||||
// For unknown nodes, try to render children
|
||||
|
|
|
|||
|
|
@ -312,7 +312,7 @@ message CreateUserRequest {
|
|||
];
|
||||
|
||||
// Optional. The resource ID to use for this user. If set, it must equal
|
||||
// user.username, follow the UID format, and not consist entirely of digits.
|
||||
// user.username and follow the username format.
|
||||
// Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
|
||||
string user_id = 2 [(google.api.field_behavior) = OPTIONAL];
|
||||
|
||||
|
|
|
|||
|
|
@ -651,7 +651,7 @@ type CreateUserRequest struct {
|
|||
// Required. The user to create.
|
||||
User *User `protobuf:"bytes,1,opt,name=user,proto3" json:"user,omitempty"`
|
||||
// Optional. The resource ID to use for this user. If set, it must equal
|
||||
// user.username, follow the UID format, and not consist entirely of digits.
|
||||
// user.username and follow the username format.
|
||||
// Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
|
||||
UserId string `protobuf:"bytes,2,opt,name=user_id,json=userId,proto3" json:"user_id,omitempty"`
|
||||
// Optional. If set, validate the request but don't actually create the user.
|
||||
|
|
|
|||
|
|
@ -1355,7 +1355,7 @@ paths:
|
|||
in: query
|
||||
description: |-
|
||||
Optional. The resource ID to use for this user. If set, it must equal
|
||||
user.username, follow the UID format, and not consist entirely of digits.
|
||||
user.username and follow the username format.
|
||||
Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
|
||||
schema:
|
||||
type: string
|
||||
|
|
|
|||
|
|
@ -66,11 +66,11 @@ func (s *APIV1Service) resolveSSOUser(ctx context.Context, currentUser *store.Us
|
|||
}
|
||||
|
||||
// createSSOUser prefers the mapped external identifier as the initial local
|
||||
// username when it satisfies the local username rules and is not a reserved
|
||||
// name. A database uniqueness conflict falls back to a generated UUID instead of
|
||||
// linking the SSO identity to the existing same-named account. User and identity
|
||||
// creation are committed atomically so a concurrent UUID fallback cannot win
|
||||
// after another request has claimed the preferred username.
|
||||
// username when it satisfies the local username rules. A database uniqueness
|
||||
// conflict falls back to a generated UUID instead of linking the SSO identity to
|
||||
// the existing same-named account. User and identity creation are committed
|
||||
// atomically so a concurrent UUID fallback cannot win after another request has
|
||||
// claimed the preferred username.
|
||||
//
|
||||
// tryUsername returns a non-nil user when the identity is resolved (either newly
|
||||
// created or reconciled to a concurrent winner) and (nil, nil) when the username
|
||||
|
|
@ -109,11 +109,8 @@ func (s *APIV1Service) createSSOUser(
|
|||
return s.getLinkedSSOUser(ctx, provider, externUID)
|
||||
}
|
||||
|
||||
// Only adopt the external identifier as the local username when it is a valid,
|
||||
// non-reserved name; otherwise an attacker-influenceable identifier could
|
||||
// squat a privileged or system handle. Reserved and invalid names fall back to
|
||||
// an opaque UUID.
|
||||
if err := validateWritableUsername(userInfo.Identifier); err == nil && !isReservedUsername(userInfo.Identifier) {
|
||||
// Adopt any valid external identifier. Invalid names fall back to an opaque UUID.
|
||||
if err := validateWritableUsername(userInfo.Identifier); err == nil {
|
||||
user, err := tryUsername(userInfo.Identifier)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
|
|
|||
|
|
@ -1,8 +1,6 @@
|
|||
package v1
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/pkg/errors"
|
||||
|
||||
"github.com/usememos/memos/internal/util"
|
||||
|
|
@ -10,27 +8,6 @@ import (
|
|||
|
||||
const ssoUsernameFallbackAttempts = 5
|
||||
|
||||
// reservedUsernames are names that must never be auto-assigned from an external
|
||||
// identity provider. A user-influenceable identifier (e.g. an OIDC
|
||||
// preferred_username) claiming one of these on first login would squat a
|
||||
// privileged or system-suggestive handle, so it falls back to a UUID instead.
|
||||
var reservedUsernames = map[string]struct{}{
|
||||
"admin": {},
|
||||
"administrator": {},
|
||||
"api": {},
|
||||
"memos": {},
|
||||
"root": {},
|
||||
"support": {},
|
||||
"system": {},
|
||||
}
|
||||
|
||||
// isReservedUsername reports whether username is reserved. The comparison is
|
||||
// case-insensitive because some backends fold case on the username unique index.
|
||||
func isReservedUsername(username string) bool {
|
||||
_, ok := reservedUsernames[strings.ToLower(strings.TrimSpace(username))]
|
||||
return ok
|
||||
}
|
||||
|
||||
// deriveSSOUsername produces the local username for a new SSO-created user.
|
||||
//
|
||||
// UUID usernames are the fallback when the IdP identifier cannot safely be used
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ func TestSSOSignInUsesValidIdentifierAsUsername(t *testing.T) {
|
|||
identifier string
|
||||
}{
|
||||
{name: "single character", identifier: "a"},
|
||||
{name: "numeric", identifier: "12345"},
|
||||
{name: "common username", identifier: "alice"},
|
||||
{name: "uppercase and hyphens", identifier: "Alice-01"},
|
||||
{name: "maximum length", identifier: strings.Repeat("a", 36)},
|
||||
|
|
@ -56,7 +57,6 @@ func TestSSOSignInFallsBackForInvalidIdentifier(t *testing.T) {
|
|||
name string
|
||||
identifier string
|
||||
}{
|
||||
{name: "numeric", identifier: "12345"},
|
||||
{name: "email", identifier: "alice@example.com"},
|
||||
{name: "underscore", identifier: "alice_example"},
|
||||
{name: "leading hyphen", identifier: "-alice"},
|
||||
|
|
@ -125,22 +125,20 @@ func TestSSOSignInDoesNotTakeOverExistingUsername(t *testing.T) {
|
|||
assertSingleSSOLink(ctx, t, ts, "username-collision", "alice", response.User.Username)
|
||||
}
|
||||
|
||||
func TestSSOSignInDoesNotAdoptReservedUsername(t *testing.T) {
|
||||
func TestSSOSignInAdoptsOrdinaryRoleLikeUsername(t *testing.T) {
|
||||
for _, identifier := range []string{"admin", "Admin", "support", "root"} {
|
||||
t.Run(identifier, func(t *testing.T) {
|
||||
ts := NewTestService(t)
|
||||
defer ts.Cleanup()
|
||||
|
||||
ctx := context.Background()
|
||||
mockIDP := newMockOAuthServer(t, "reserved-code", "reserved-token", map[string]any{"sub": identifier})
|
||||
mockIDP := newMockOAuthServer(t, "role-like-code", "role-like-token", map[string]any{"sub": identifier})
|
||||
defer mockIDP.Close()
|
||||
|
||||
idpName := createTestingOAuthIdentityProvider(ctx, t, ts, mockIDP.URL, "reserved-provider")
|
||||
response, err := signInWithTestingSSO(ctx, ts, idpName, "reserved-code")
|
||||
idpName := createTestingOAuthIdentityProvider(ctx, t, ts, mockIDP.URL, "role-like-provider")
|
||||
response, err := signInWithTestingSSO(ctx, ts, idpName, "role-like-code")
|
||||
require.NoError(t, err)
|
||||
require.NotEqual(t, identifier, response.User.Username)
|
||||
_, err = uuid.Parse(response.User.Username)
|
||||
require.NoError(t, err, "reserved identifier must fall back to a UUID")
|
||||
require.Equal(t, identifier, response.User.Username)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -88,22 +88,22 @@ func TestUserResourceName(t *testing.T) {
|
|||
require.Equal(t, "users/Gnammi", resp.Users[0].Name)
|
||||
})
|
||||
|
||||
t.Run("CreateUser rejects all-numeric usernames", func(t *testing.T) {
|
||||
t.Run("CreateUser accepts all-numeric usernames", func(t *testing.T) {
|
||||
ts := NewTestService(t)
|
||||
defer ts.Cleanup()
|
||||
|
||||
_, err := ts.Service.CreateUser(ctx, &apiv1.CreateUserRequest{
|
||||
created, err := ts.Service.CreateUser(ctx, &apiv1.CreateUserRequest{
|
||||
User: &apiv1.User{
|
||||
Username: "123",
|
||||
Email: "123@example.com",
|
||||
Password: "password123",
|
||||
},
|
||||
})
|
||||
require.Error(t, err)
|
||||
require.Contains(t, err.Error(), "invalid username")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "users/123", created.Name)
|
||||
})
|
||||
|
||||
t.Run("GetUser returns not found for numeric user resource names", func(t *testing.T) {
|
||||
t.Run("GetUser does not interpret a numeric username as an internal ID", func(t *testing.T) {
|
||||
ts := NewTestService(t)
|
||||
defer ts.Cleanup()
|
||||
|
||||
|
|
|
|||
|
|
@ -36,6 +36,21 @@ func TestBatchGetUsersReturnsExactUsernamesWithoutAuthentication(t *testing.T) {
|
|||
require.True(t, ok)
|
||||
}
|
||||
|
||||
func TestBatchGetUsersDoesNotNormalizeUsernames(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
ts := NewTestService(t)
|
||||
defer ts.Cleanup()
|
||||
|
||||
_, err := ts.CreateRegularUser(ctx, "batch-alpha")
|
||||
require.NoError(t, err)
|
||||
|
||||
resp, err := ts.Service.BatchGetUsers(ctx, &apiv1.BatchGetUsersRequest{
|
||||
Usernames: []string{" batch-alpha "},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
require.Empty(t, resp.Users)
|
||||
}
|
||||
|
||||
func TestBatchGetUsersRejectsTooManyUsernames(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
ts := NewTestService(t)
|
||||
|
|
|
|||
|
|
@ -222,7 +222,7 @@ func TestCreateUserRegistration(t *testing.T) {
|
|||
})
|
||||
|
||||
t.Run("CreateUser rejects invalid writable usernames", func(t *testing.T) {
|
||||
for _, username := range []string{"alice@example.com", "legacy_user", "123"} {
|
||||
for _, username := range []string{"alice@example.com", "legacy_user"} {
|
||||
t.Run(username, func(t *testing.T) {
|
||||
ts := NewTestService(t)
|
||||
defer ts.Cleanup()
|
||||
|
|
|
|||
|
|
@ -27,24 +27,12 @@ func parseUsernameFromName(name string) (string, error) {
|
|||
}
|
||||
|
||||
func validateWritableUsername(username string) error {
|
||||
if username == "" || isNumericUsername(username) || !base.UIDMatcher.MatchString(username) {
|
||||
if !base.IsValidUsername(username) {
|
||||
return errors.Errorf("invalid username %q", username)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func isNumericUsername(username string) bool {
|
||||
if username == "" {
|
||||
return false
|
||||
}
|
||||
for _, char := range username {
|
||||
if char < '0' || char > '9' {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ResolveUserByName resolves a username-based user resource name to a store user.
|
||||
func ResolveUserByName(ctx context.Context, stores *store.Store, name string) (*store.User, error) {
|
||||
username, err := parseUsernameFromName(name)
|
||||
|
|
|
|||
|
|
@ -51,9 +51,8 @@ func TestValidateWritableUsername(t *testing.T) {
|
|||
wantError: true,
|
||||
},
|
||||
{
|
||||
name: "numeric",
|
||||
username: "123",
|
||||
wantError: true,
|
||||
name: "numeric",
|
||||
username: "123",
|
||||
},
|
||||
{
|
||||
name: "email",
|
||||
|
|
|
|||
|
|
@ -116,12 +116,11 @@ func (s *APIV1Service) ListUsers(ctx context.Context, request *v1pb.ListUsersReq
|
|||
return response, nil
|
||||
}
|
||||
|
||||
func normalizeBatchUsernames(usernames []string) ([]string, int) {
|
||||
func uniqueBatchUsernames(usernames []string) ([]string, int) {
|
||||
uniqueUsernames := make([]string, 0, len(usernames))
|
||||
seen := make(map[string]struct{}, len(usernames))
|
||||
nonEmptyCount := 0
|
||||
for _, username := range usernames {
|
||||
username = strings.TrimSpace(username)
|
||||
if username == "" {
|
||||
continue
|
||||
}
|
||||
|
|
@ -140,7 +139,7 @@ func (s *APIV1Service) BatchGetUsers(ctx context.Context, request *v1pb.BatchGet
|
|||
return &v1pb.BatchGetUsersResponse{Users: []*v1pb.User{}}, nil
|
||||
}
|
||||
|
||||
uniqueUsernames, nonEmptyUsernameCount := normalizeBatchUsernames(request.Usernames)
|
||||
uniqueUsernames, nonEmptyUsernameCount := uniqueBatchUsernames(request.Usernames)
|
||||
if nonEmptyUsernameCount > maxBatchGetUsers {
|
||||
return nil, status.Errorf(codes.InvalidArgument, "too many usernames (max %d)", maxBatchGetUsers)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
ALTER TABLE `user`
|
||||
MODIFY `username` VARCHAR(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL;
|
||||
|
|
@ -11,7 +11,7 @@ CREATE TABLE `user` (
|
|||
`created_ts` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`updated_ts` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
`row_status` VARCHAR(256) NOT NULL DEFAULT 'NORMAL',
|
||||
`username` VARCHAR(256) NOT NULL UNIQUE,
|
||||
`username` VARCHAR(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL UNIQUE,
|
||||
`role` VARCHAR(256) NOT NULL DEFAULT 'USER',
|
||||
`email` VARCHAR(256) NOT NULL DEFAULT '',
|
||||
`nickname` VARCHAR(256) NOT NULL DEFAULT '',
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
ALTER TABLE "user"
|
||||
ALTER COLUMN username TYPE TEXT COLLATE "C";
|
||||
|
|
@ -11,7 +11,7 @@ CREATE TABLE "user" (
|
|||
created_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
|
||||
updated_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
|
||||
row_status TEXT NOT NULL DEFAULT 'NORMAL',
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
username TEXT COLLATE "C" NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'USER',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
nickname TEXT NOT NULL DEFAULT '',
|
||||
|
|
|
|||
24
store/migration/sqlite/0.30/01__case_sensitive_username.sql
Normal file
24
store/migration/sqlite/0.30/01__case_sensitive_username.sql
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
ALTER TABLE user RENAME TO user_old;
|
||||
|
||||
CREATE TABLE user (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
|
||||
username TEXT COLLATE BINARY NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'USER',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
nickname TEXT NOT NULL DEFAULT '',
|
||||
password_hash TEXT NOT NULL,
|
||||
avatar_url TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
INSERT INTO user (
|
||||
id, created_ts, updated_ts, row_status, username, role, email, nickname, password_hash, avatar_url, description
|
||||
)
|
||||
SELECT
|
||||
id, created_ts, updated_ts, row_status, username, role, email, nickname, password_hash, avatar_url, description
|
||||
FROM user_old;
|
||||
|
||||
DROP TABLE user_old;
|
||||
|
|
@ -12,7 +12,7 @@ CREATE TABLE user (
|
|||
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
username TEXT COLLATE BINARY NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'USER',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
nickname TEXT NOT NULL DEFAULT '',
|
||||
|
|
|
|||
|
|
@ -135,8 +135,16 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {
|
|||
);
|
||||
CREATE TABLE user (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'USER'
|
||||
role TEXT NOT NULL DEFAULT 'USER',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
nickname TEXT NOT NULL DEFAULT '',
|
||||
password_hash TEXT NOT NULL,
|
||||
avatar_url TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE TABLE user_setting (
|
||||
user_id INTEGER NOT NULL,
|
||||
|
|
@ -174,7 +182,10 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {
|
|||
|
||||
_, err = db.ExecContext(ctx, "INSERT INTO system_setting (name, value) VALUES ('BASIC', ?), ('TAGS', ?)", string(basicSettingBytes), string(tagsSettingBytes))
|
||||
require.NoError(t, err)
|
||||
_, err = db.ExecContext(ctx, "INSERT INTO user (id, username, role) VALUES (1, 'tag-owner', 'USER'), (2, 'keeps-existing', 'USER')")
|
||||
_, err = db.ExecContext(ctx, `
|
||||
INSERT INTO user (id, username, role, password_hash, avatar_url)
|
||||
VALUES (1, 'tag-owner', 'USER', 'legacy-hash', ''), (2, 'keeps-existing', 'USER', 'legacy-hash', '')
|
||||
`)
|
||||
require.NoError(t, err)
|
||||
_, err = db.ExecContext(ctx, "INSERT INTO user_setting (user_id, key, value) VALUES (2, 'TAGS', ?)", string(existingUserTagsBytes))
|
||||
require.NoError(t, err)
|
||||
|
|
@ -208,6 +219,133 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {
|
|||
require.NotContains(t, existing.GetTags().GetTags(), "bug")
|
||||
}
|
||||
|
||||
func TestCaseSensitiveUsernameMigration(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
driver := getDriverFromEnv()
|
||||
var dsn string
|
||||
switch driver {
|
||||
case "sqlite":
|
||||
dsn = fmt.Sprintf("%s/memos_username_migration.db", t.TempDir())
|
||||
case "mysql":
|
||||
dsn = GetMySQLDSN(t)
|
||||
case "postgres":
|
||||
dsn = GetPostgresDSN(t)
|
||||
default:
|
||||
t.Fatalf("unsupported driver: %s", driver)
|
||||
}
|
||||
|
||||
db, err := sql.Open(driver, dsn)
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = db.ExecContext(ctx, caseSensitiveUsernameMigrationFixture(driver))
|
||||
require.NoError(t, err)
|
||||
|
||||
basicSettingBytes, err := protojson.Marshal(&storepb.InstanceBasicSetting{SchemaVersion: "0.30.1"})
|
||||
require.NoError(t, err)
|
||||
insertBasicSetting := "INSERT INTO system_setting (name, value, description) VALUES ('BASIC', ?, '')"
|
||||
if driver == "postgres" {
|
||||
insertBasicSetting = "INSERT INTO system_setting (name, value, description) VALUES ('BASIC', $1, '')"
|
||||
}
|
||||
_, err = db.ExecContext(ctx, insertBasicSetting, string(basicSettingBytes))
|
||||
require.NoError(t, err)
|
||||
insertUser := "INSERT INTO user (username, role, password_hash, avatar_url) VALUES ('Alice', 'USER', 'legacy-hash', '')"
|
||||
if driver == "mysql" {
|
||||
insertUser = "INSERT INTO `user` (username, role, password_hash, avatar_url) VALUES ('Alice', 'USER', 'legacy-hash', '')"
|
||||
} else if driver == "postgres" {
|
||||
insertUser = `INSERT INTO "user" (username, role, password_hash, avatar_url) VALUES ('Alice', 'USER', 'legacy-hash', '')`
|
||||
}
|
||||
_, err = db.ExecContext(ctx, insertUser)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, db.Close())
|
||||
|
||||
ts := NewTestingStoreWithDSN(ctx, t, driver, dsn)
|
||||
require.NoError(t, ts.Migrate(ctx))
|
||||
defer ts.Close()
|
||||
|
||||
lower, err := createTestingUserWithRole(ctx, ts, "alice", store.RoleUser)
|
||||
require.NoError(t, err)
|
||||
|
||||
upperUsername := "Alice"
|
||||
upper, err := ts.GetUser(ctx, &store.FindUser{Username: &upperUsername})
|
||||
require.NoError(t, err)
|
||||
require.NotEqual(t, upper.ID, lower.ID)
|
||||
require.Equal(t, "Alice", upper.Username)
|
||||
}
|
||||
|
||||
func caseSensitiveUsernameMigrationFixture(driver string) string {
|
||||
switch driver {
|
||||
case "mysql":
|
||||
return `
|
||||
CREATE TABLE system_setting (
|
||||
name VARCHAR(256) NOT NULL PRIMARY KEY,
|
||||
value LONGTEXT NOT NULL,
|
||||
description TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE ` + "`user`" + ` (
|
||||
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||
created_ts TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_ts TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
row_status VARCHAR(256) NOT NULL DEFAULT 'NORMAL',
|
||||
username VARCHAR(256) NOT NULL UNIQUE,
|
||||
role VARCHAR(256) NOT NULL DEFAULT 'USER',
|
||||
email VARCHAR(256) NOT NULL DEFAULT '',
|
||||
nickname VARCHAR(256) NOT NULL DEFAULT '',
|
||||
password_hash VARCHAR(256) NOT NULL,
|
||||
avatar_url LONGTEXT NOT NULL,
|
||||
description VARCHAR(256) NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE TABLE memo (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY);
|
||||
`
|
||||
case "postgres":
|
||||
return `
|
||||
CREATE TABLE system_setting (
|
||||
name TEXT NOT NULL PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
description TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE "user" (
|
||||
id SERIAL PRIMARY KEY,
|
||||
created_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
|
||||
updated_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
|
||||
row_status TEXT NOT NULL DEFAULT 'NORMAL',
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'USER',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
nickname TEXT NOT NULL DEFAULT '',
|
||||
password_hash TEXT NOT NULL,
|
||||
avatar_url TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE TABLE memo (id SERIAL PRIMARY KEY);
|
||||
`
|
||||
case "sqlite":
|
||||
return `
|
||||
CREATE TABLE system_setting (
|
||||
name TEXT NOT NULL,
|
||||
value TEXT NOT NULL,
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
UNIQUE(name)
|
||||
);
|
||||
CREATE TABLE user (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
|
||||
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
role TEXT NOT NULL DEFAULT 'USER',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
nickname TEXT NOT NULL DEFAULT '',
|
||||
password_hash TEXT NOT NULL,
|
||||
avatar_url TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE TABLE memo (id INTEGER PRIMARY KEY AUTOINCREMENT);
|
||||
`
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
// TestMigrationFromStableVersion verifies that upgrading from a stable Memos version
|
||||
// to the current version works correctly. This is the critical upgrade path test.
|
||||
//
|
||||
|
|
|
|||
|
|
@ -117,6 +117,33 @@ func TestUserGetByUsername(t *testing.T) {
|
|||
ts.Close()
|
||||
}
|
||||
|
||||
func TestUsernameEqualityIsCaseSensitive(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx := context.Background()
|
||||
ts := NewTestingStore(ctx, t)
|
||||
defer ts.Close()
|
||||
|
||||
upper, err := createTestingUserWithRole(ctx, ts, "Alice", store.RoleUser)
|
||||
require.NoError(t, err)
|
||||
lower, err := createTestingUserWithRole(ctx, ts, "alice", store.RoleUser)
|
||||
require.NoError(t, err)
|
||||
require.NotEqual(t, upper.ID, lower.ID)
|
||||
|
||||
for username, wantID := range map[string]int32{
|
||||
"Alice": upper.ID,
|
||||
"alice": lower.ID,
|
||||
} {
|
||||
found, err := ts.GetUser(ctx, &store.FindUser{Username: &username})
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, wantID, found.ID)
|
||||
|
||||
listed, err := ts.ListUsers(ctx, &store.FindUser{UsernameList: []string{username}})
|
||||
require.NoError(t, err)
|
||||
require.Len(t, listed, 1)
|
||||
require.Equal(t, wantID, listed[0].ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUserListByRole(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx := context.Background()
|
||||
|
|
|
|||
|
|
@ -10,10 +10,9 @@ import { isMentionElement, isTagElement, isTaskListItemElement } from "@/types/m
|
|||
import { lazyWithReload } from "@/utils/lazy";
|
||||
import { rehypeHeadingId } from "@/utils/rehype-plugins/rehype-heading-id";
|
||||
import { remarkDisableSetext } from "@/utils/remark-plugins/remark-disable-setext";
|
||||
import { remarkMention } from "@/utils/remark-plugins/remark-mention";
|
||||
import { remarkPreserveType } from "@/utils/remark-plugins/remark-preserve-type";
|
||||
import { remarkSplitMixedTaskLists } from "@/utils/remark-plugins/remark-split-mixed-task-lists";
|
||||
import { remarkTag } from "@/utils/remark-plugins/remark-tag";
|
||||
import { remarkMemoSyntax } from "@/utils/remark-plugins/remark-tag";
|
||||
import { CodeBlock } from "./CodeBlock";
|
||||
import { SANITIZE_SCHEMA } from "./constants";
|
||||
import { MarkdownRenderContext, rootMarkdownRenderContext } from "./MarkdownRenderContext";
|
||||
|
|
@ -58,7 +57,7 @@ function getMentionUsername(node: Element, children?: ReactNode): string {
|
|||
|
||||
const text = Array.isArray(children) ? children.join("") : children;
|
||||
if (typeof text === "string" && text.startsWith("@")) {
|
||||
return text.slice(1).toLowerCase();
|
||||
return text.slice(1);
|
||||
}
|
||||
|
||||
return "";
|
||||
|
|
@ -165,8 +164,7 @@ export const MemoMarkdownRendererCore = ({
|
|||
...mathRemarkPlugins,
|
||||
remarkGfm,
|
||||
remarkSplitMixedTaskLists,
|
||||
remarkTag,
|
||||
remarkMention,
|
||||
remarkMemoSyntax,
|
||||
remarkBreaks,
|
||||
remarkPreserveType,
|
||||
]}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ import { createContext, type ReactNode, useContext, useMemo } from "react";
|
|||
import { userDetailQueryOptions, useUsersByUsernames } from "@/hooks/useUserQueries";
|
||||
import { extractUsernameFromName } from "@/lib/resource-names";
|
||||
import type { User } from "@/types/proto/api/v1/user_service_pb";
|
||||
import { extractMentionUsernames } from "@/utils/remark-plugins/remark-mention";
|
||||
import { extractMentionUsernames } from "@/utils/mention-extraction";
|
||||
|
||||
interface UserResolutionContextValue {
|
||||
mentionUsernamesByContent: ReadonlyMap<string, string[]>;
|
||||
|
|
|
|||
|
|
@ -13,7 +13,9 @@ import {
|
|||
} from "@/utils/markdown-link";
|
||||
import { findDecodedMarkdownSourceBoundaries } from "@/utils/markdown-source-boundaries";
|
||||
import { memoMarkdownExtensions } from "@/utils/memo-markdown-extension";
|
||||
import { findMentionMatches, type MentionMatch } from "@/utils/mention-grammar";
|
||||
import { findTagMatches, type TagMatch } from "@/utils/tag-grammar";
|
||||
import { isUsernameCharacter } from "@/utils/username";
|
||||
|
||||
interface SourceRange {
|
||||
from: number;
|
||||
|
|
@ -290,11 +292,7 @@ function mergeSourceRanges(ranges: SourceRange[]): SourceRange[] {
|
|||
return merged;
|
||||
}
|
||||
|
||||
export function markdownGFMEmailSourceRanges(state: EditorState, to = state.doc.length): GFMEmailSourceRange[] {
|
||||
return parsedMarkdownContext(state, to).emailRanges;
|
||||
}
|
||||
|
||||
export function literalTagSourceRanges(state: EditorState, from: number, to: number): SourceRange[] {
|
||||
export function literalMarkdownSourceRanges(state: EditorState, from: number, to: number): SourceRange[] {
|
||||
const ranges: SourceRange[] = [];
|
||||
const context = parsedMarkdownContext(state, to);
|
||||
collectLiteralRanges(
|
||||
|
|
@ -316,7 +314,7 @@ export function literalTagSourceRanges(state: EditorState, from: number, to: num
|
|||
/** Find tags only in literal Markdown source exposed by the editor syntax tree. */
|
||||
export function findMarkdownTagMatches(state: EditorState, from: number, to: number): TagMatch[] {
|
||||
const matches: TagMatch[] = [];
|
||||
for (const range of literalTagSourceRanges(state, from, to)) {
|
||||
for (const range of literalMarkdownSourceRanges(state, from, to)) {
|
||||
const source = state.doc.sliceString(range.from, range.to);
|
||||
for (const match of findTagMatches(source)) {
|
||||
matches.push({ ...match, from: range.from + match.from, to: range.from + match.to });
|
||||
|
|
@ -325,6 +323,19 @@ export function findMarkdownTagMatches(state: EditorState, from: number, to: num
|
|||
return matches;
|
||||
}
|
||||
|
||||
/** Find username references only in literal Markdown source exposed by the editor syntax tree. */
|
||||
export function findMarkdownMentionMatches(state: EditorState, from: number, to: number): MentionMatch[] {
|
||||
const matches: MentionMatch[] = [];
|
||||
for (const range of literalMarkdownSourceRanges(state, from, to)) {
|
||||
const source = state.doc.sliceString(range.from, range.to);
|
||||
const runHasLeftBoundary = range.from === 0 || !isUsernameCharacter(state.doc.sliceString(range.from - 1, range.from));
|
||||
for (const match of findMentionMatches(source, runHasLeftBoundary)) {
|
||||
matches.push({ ...match, from: range.from + match.from, to: range.from + match.to });
|
||||
}
|
||||
}
|
||||
return matches;
|
||||
}
|
||||
|
||||
/** Return the active tag whose recognized source span ends at the cursor. */
|
||||
export function tagMatchBefore(state: EditorState, position: number): TagMatch | undefined {
|
||||
const line = state.doc.lineAt(position);
|
||||
|
|
@ -335,5 +346,5 @@ export function tagMatchBefore(state: EditorState, position: number): TagMatch |
|
|||
/** Whether an offset is in a literal-source run (used for explicit `#` completion). */
|
||||
export function isLiteralTagPosition(state: EditorState, position: number): boolean {
|
||||
const line = state.doc.lineAt(position);
|
||||
return literalTagSourceRanges(state, line.from, position).some((range) => range.to === position);
|
||||
return literalMarkdownSourceRanges(state, line.from, position).some((range) => range.to === position);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
import { RangeSetBuilder } from "@codemirror/state";
|
||||
import { Decoration, type DecorationSet, EditorView } from "@codemirror/view";
|
||||
import { MENTION_RUN } from "@/utils/mention-grammar";
|
||||
import { findMarkdownTagMatches, markdownGFMEmailSourceRanges } from "./markdownTagRanges";
|
||||
import { findMarkdownMentionMatches, findMarkdownTagMatches } from "./markdownTagRanges";
|
||||
import { viewportDecorations } from "./viewportDecorations";
|
||||
|
||||
const MENTION_RE = new RegExp(`(^|[^A-Za-z0-9])@(${MENTION_RUN})`, "gu");
|
||||
const tagMark = Decoration.mark({ class: "cm-memo-tag" });
|
||||
const mentionMark = Decoration.mark({ class: "cm-memo-mention" });
|
||||
|
||||
|
|
@ -19,20 +17,13 @@ function build(view: EditorView): DecorationSet {
|
|||
if (previous && range.from <= previous.to) previous.to = Math.max(previous.to, range.to);
|
||||
else scanRanges.push({ ...range });
|
||||
}
|
||||
const emailRanges = markdownGFMEmailSourceRanges(view.state, scanRanges.at(-1)?.to ?? view.viewport.to);
|
||||
|
||||
for (const { from, to } of scanRanges) {
|
||||
for (const match of findMarkdownTagMatches(view.state, from, to)) {
|
||||
ranges.push({ from: match.from, to: match.to, deco: tagMark });
|
||||
}
|
||||
|
||||
const text = view.state.doc.sliceString(from, to);
|
||||
MENTION_RE.lastIndex = 0;
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = MENTION_RE.exec(text)) !== null) {
|
||||
const start = from + match.index + match[1].length; // skip the boundary char
|
||||
if (emailRanges.some((range) => range.from <= start && start < range.to)) continue;
|
||||
ranges.push({ from: start, to: start + 1 + match[2].length, deco: mentionMark });
|
||||
for (const match of findMarkdownMentionMatches(view.state, from, to)) {
|
||||
ranges.push({ from: match.from, to: match.to, deco: mentionMark });
|
||||
}
|
||||
}
|
||||
ranges.sort((a, b) => a.from - b.from || a.to - b.to);
|
||||
|
|
|
|||
|
|
@ -293,7 +293,7 @@ export type CreateUserRequest = Message<"memos.api.v1.CreateUserRequest"> & {
|
|||
|
||||
/**
|
||||
* Optional. The resource ID to use for this user. If set, it must equal
|
||||
* user.username, follow the UID format, and not consist entirely of digits.
|
||||
* user.username and follow the username format.
|
||||
* Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
|
||||
*
|
||||
* @generated from field: string user_id = 2;
|
||||
|
|
|
|||
1
web/src/utils/mention-extraction.ts
Normal file
1
web/src/utils/mention-extraction.ts
Normal file
|
|
@ -0,0 +1 @@
|
|||
export { extractMentionUsernames } from "@/utils/remark-plugins/remark-tag";
|
||||
|
|
@ -1,28 +1,32 @@
|
|||
/**
|
||||
* The single source of truth for memos' `@mention` lexing grammar, shared by the
|
||||
* editor and the read-only renderer (utils/remark-plugins/remark-mention.ts) so
|
||||
* they can't drift.
|
||||
*
|
||||
* A mention character is an ASCII letter, digit, or hyphen. A username is capped
|
||||
* at MAX_MENTION_LENGTH characters and must contain at least one letter or digit
|
||||
* (a run of only hyphens is not a mention). This matches the Go backend parser
|
||||
* (internal/markdown/parser/mention.go).
|
||||
*/
|
||||
export const MENTION_CHAR_CLASS = "[A-Za-z0-9-]";
|
||||
import { isUsernameCharacter, isValidUsername } from "@/utils/username";
|
||||
|
||||
export const MAX_MENTION_LENGTH = 63;
|
||||
|
||||
/**
|
||||
* Regex source for a username run: 1..MAX_MENTION_LENGTH mention characters that
|
||||
* include at least one letter or digit. An over-long run is left as plain text.
|
||||
* The leading lookahead enforces the "must contain an alphanumeric" rule so
|
||||
* `@---` is left as plain text. Embed after an `@`, e.g. `^@(${MENTION_RUN})`.
|
||||
*/
|
||||
export const MENTION_RUN = `(?=${MENTION_CHAR_CLASS}{0,${MAX_MENTION_LENGTH - 1}}[A-Za-z0-9])${MENTION_CHAR_CLASS}{1,${MAX_MENTION_LENGTH}}(?!${MENTION_CHAR_CLASS})`;
|
||||
|
||||
const SINGLE_MENTION_CHAR = /^[A-Za-z0-9-]$/;
|
||||
|
||||
/** Whether a single character is allowed inside a username. */
|
||||
export function isMentionChar(char: string): boolean {
|
||||
return SINGLE_MENTION_CHAR.test(char);
|
||||
export interface MentionMatch {
|
||||
from: number;
|
||||
to: number;
|
||||
username: string;
|
||||
}
|
||||
|
||||
function hasLeftBoundary(source: string, position: number, runHasLeftBoundary: boolean): boolean {
|
||||
if (position === 0) return runHasLeftBoundary;
|
||||
return !isUsernameCharacter(source[position - 1]);
|
||||
}
|
||||
|
||||
/** Find username references in one eligible literal Markdown source run. */
|
||||
export function findMentionMatches(source: string, runHasLeftBoundary = true): MentionMatch[] {
|
||||
const matches: MentionMatch[] = [];
|
||||
for (let position = 0; position < source.length; ) {
|
||||
if (source[position] !== "@" || !hasLeftBoundary(source, position, runHasLeftBoundary)) {
|
||||
position++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let end = position + 1;
|
||||
while (end < source.length && isUsernameCharacter(source[end])) end++;
|
||||
const username = source.slice(position + 1, end);
|
||||
if (isValidUsername(username)) {
|
||||
matches.push({ from: position, to: end, username });
|
||||
}
|
||||
position = end === position + 1 ? position + 1 : end;
|
||||
}
|
||||
return matches;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,155 +0,0 @@
|
|||
import type { Root, Text } from "mdast";
|
||||
import type { Node as UnistNode } from "unist";
|
||||
import { visit } from "unist-util-visit";
|
||||
import type { MentionNode, MentionNodeData } from "@/types/markdown";
|
||||
import { isMentionChar, MAX_MENTION_LENGTH } from "@/utils/mention-grammar";
|
||||
|
||||
function isMentionBoundary(char: string): boolean {
|
||||
if (!char) return true;
|
||||
return !isMentionChar(char);
|
||||
}
|
||||
|
||||
type Segment = { type: "text"; value: string } | { type: "mention"; value: string };
|
||||
|
||||
export function parseMentionsFromText(text: string, leadingChar = ""): Segment[] {
|
||||
const segments: Segment[] = [];
|
||||
const chars = [...text];
|
||||
let i = 0;
|
||||
|
||||
const appendText = (value: string) => {
|
||||
const previous = segments.at(-1);
|
||||
if (previous?.type === "text") {
|
||||
previous.value += value;
|
||||
} else {
|
||||
segments.push({ type: "text", value });
|
||||
}
|
||||
};
|
||||
|
||||
while (i < chars.length) {
|
||||
const prevChar = i > 0 ? chars[i - 1] : leadingChar;
|
||||
if (chars[i] === "@" && isMentionBoundary(prevChar) && i + 1 < chars.length && isMentionChar(chars[i + 1])) {
|
||||
let j = i + 1;
|
||||
while (j < chars.length && isMentionChar(chars[j]) && j - i - 1 < MAX_MENTION_LENGTH) {
|
||||
j++;
|
||||
}
|
||||
|
||||
const username = chars.slice(i + 1, j).join("");
|
||||
const isOverlong = j < chars.length && isMentionChar(chars[j]);
|
||||
const hasLetterOrNumber = [...username].some((char) => /[A-Za-z0-9]/.test(char));
|
||||
if (username && !isOverlong && hasLetterOrNumber) {
|
||||
segments.push({ type: "mention", value: username.toLowerCase() });
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
let j = i + 1;
|
||||
while (j < chars.length && chars[j] !== "@") {
|
||||
j++;
|
||||
}
|
||||
appendText(chars.slice(i, j).join(""));
|
||||
i = j;
|
||||
}
|
||||
|
||||
return segments;
|
||||
}
|
||||
|
||||
export function extractMentionUsernames(text: string): string[] {
|
||||
const usernames = parseMentionsFromText(text)
|
||||
.filter((segment): segment is { type: "mention"; value: string } => segment.type === "mention")
|
||||
.map((segment) => segment.value);
|
||||
return Array.from(new Set(usernames));
|
||||
}
|
||||
|
||||
function createMentionNode(username: string): MentionNode {
|
||||
const data: MentionNodeData = {
|
||||
hName: "span",
|
||||
hProperties: {
|
||||
className: "mention",
|
||||
"data-mention": username,
|
||||
},
|
||||
hChildren: [{ type: "text", value: `@${username}` }],
|
||||
};
|
||||
|
||||
return {
|
||||
type: "mentionNode",
|
||||
value: username,
|
||||
data,
|
||||
} as MentionNode;
|
||||
}
|
||||
|
||||
function trailingText(node: UnistNode | undefined): string {
|
||||
if (!node) return "";
|
||||
if (node.type === "text") return (node as Text).value;
|
||||
const nodeChildren = (node as { children?: UnistNode[] }).children;
|
||||
if (nodeChildren?.length) return trailingText(nodeChildren.at(-1));
|
||||
const children = (node.data as { hChildren?: Array<{ type?: string; value?: string }> } | undefined)?.hChildren;
|
||||
const last = children?.at(-1);
|
||||
return last?.type === "text" ? (last.value ?? "") : "";
|
||||
}
|
||||
|
||||
function previousSourceCharacter(source: string, offset: number): string {
|
||||
return [...source.slice(Math.max(0, offset - 2), offset)].at(-1) ?? "";
|
||||
}
|
||||
|
||||
function synthesizedLiteralTrailingText(node: UnistNode | undefined, source: string): string {
|
||||
if (!node) return "";
|
||||
if (node.type === "tagNode" || node.type === "mentionNode") return trailingText(node);
|
||||
if (node.type !== "link" || !(node as { url?: string }).url?.startsWith("mailto:")) return "";
|
||||
|
||||
const rendered = trailingText(node);
|
||||
const from = node.position?.start.offset;
|
||||
const to = node.position?.end.offset;
|
||||
if (from !== undefined && to !== undefined && source.slice(from, to) !== rendered) {
|
||||
// Explicit links and autolinks end in Markdown syntax (`)` or `>`), which
|
||||
// is the mention boundary. Only a literal GFM email contributes its final
|
||||
// rendered character across an mdast child boundary.
|
||||
return "";
|
||||
}
|
||||
return rendered;
|
||||
}
|
||||
|
||||
function leadingCharacter(parent: UnistNode, index: number, node: Text, source: string): string {
|
||||
const previous = (parent as { children?: UnistNode[] }).children?.[index - 1];
|
||||
if (!previous) return "";
|
||||
|
||||
const currentFrom = node.position?.start.offset;
|
||||
const previousTo = previous.position?.end.offset;
|
||||
if (currentFrom !== undefined && previousTo === currentFrom) {
|
||||
// Use the actual source boundary instead of text rendered by an arbitrary
|
||||
// preceding node. Formatting and link delimiters are meaningful mention
|
||||
// boundaries even though they are not present in rendered text.
|
||||
return previousSourceCharacter(source, currentFrom);
|
||||
}
|
||||
|
||||
return [...synthesizedLiteralTrailingText(previous, source)].at(-1) ?? "";
|
||||
}
|
||||
|
||||
type VFileLike = { value?: string | Uint8Array };
|
||||
|
||||
export const remarkMention = () => {
|
||||
return (tree: Root, file: VFileLike) => {
|
||||
const source = typeof file.value === "string" ? file.value : "";
|
||||
visit(tree, (node, index, parent) => {
|
||||
if (node.type !== "text" || !parent || parent.type === "link" || typeof index !== "number") return;
|
||||
|
||||
const textNode = node as Text;
|
||||
const segments = parseMentionsFromText(textNode.value, leadingCharacter(parent, index, textNode, source));
|
||||
if (segments.every((segment) => segment.type === "text")) {
|
||||
return;
|
||||
}
|
||||
|
||||
const newNodes = segments.map((segment) => {
|
||||
if (segment.type === "mention") {
|
||||
return createMentionNode(segment.value);
|
||||
}
|
||||
return {
|
||||
type: "text",
|
||||
value: segment.value,
|
||||
} as Text;
|
||||
});
|
||||
|
||||
(parent.children as UnistNode[]).splice(index, 1, ...(newNodes as UnistNode[]));
|
||||
});
|
||||
};
|
||||
};
|
||||
|
|
@ -1,16 +1,24 @@
|
|||
import { parser as markdownParser } from "@lezer/markdown";
|
||||
import type { Link, Root, Text } from "mdast";
|
||||
import { fromMarkdown } from "mdast-util-from-markdown";
|
||||
import { gfmFromMarkdown } from "mdast-util-gfm";
|
||||
import { gfm } from "micromark-extension-gfm";
|
||||
import { decodeString } from "micromark-util-decode-string";
|
||||
import type { Node as UnistNode } from "unist";
|
||||
import type { TagNode, TagNodeData } from "@/types/markdown";
|
||||
import type { MentionNode, MentionNodeData, TagNode, TagNodeData } from "@/types/markdown";
|
||||
import { findMarkdownGFMEmailRanges, type GFMEmailSourceRange, type MarkdownSourceNode } from "@/utils/gfm-email";
|
||||
import { findMarkdownGFMURLRanges } from "@/utils/gfm-url";
|
||||
import { hasExactMarkdownRange, resolvedMarkdownLinkRanges } from "@/utils/markdown-link";
|
||||
import { decodedMarkdownCharacterReferenceAt, findDecodedMarkdownSourceBoundaries } from "@/utils/markdown-source-boundaries";
|
||||
import { memoMarkdownExtensions } from "@/utils/memo-markdown-extension";
|
||||
import { findMentionMatches } from "@/utils/mention-grammar";
|
||||
import { findTagMatches } from "@/utils/tag-grammar";
|
||||
import { isUsernameCharacter } from "@/utils/username";
|
||||
|
||||
type Segment = { type: "text"; value: string } | { type: "tag"; source: string; value: string };
|
||||
type Segment =
|
||||
| { type: "text"; value: string }
|
||||
| { type: "tag"; source: string; value: string }
|
||||
| { type: "mention"; source: string; value: string };
|
||||
|
||||
interface SourceRange {
|
||||
from: number;
|
||||
|
|
@ -336,22 +344,33 @@ function segmentsForTextNode(
|
|||
const runs = literalRunsForTextNode(value, span, source, boundaries);
|
||||
if (!runs) return undefined;
|
||||
|
||||
const tags = runs.flatMap((run) =>
|
||||
findTagMatches(run.source).map((match) => ({
|
||||
from: run.valueFrom + match.from,
|
||||
to: run.valueFrom + match.to,
|
||||
source: run.source.slice(match.from, match.to),
|
||||
value: match.value,
|
||||
})),
|
||||
);
|
||||
if (tags.length === 0) return [{ type: "text", value }];
|
||||
const matches = runs
|
||||
.flatMap((run) => [
|
||||
...findTagMatches(run.source).map((match) => ({
|
||||
type: "tag" as const,
|
||||
from: run.valueFrom + match.from,
|
||||
to: run.valueFrom + match.to,
|
||||
source: run.source.slice(match.from, match.to),
|
||||
value: match.value,
|
||||
})),
|
||||
...findMentionMatches(run.source, run.sourceFrom === 0 || !isUsernameCharacter(source[run.sourceFrom - 1])).map((match) => ({
|
||||
type: "mention" as const,
|
||||
from: run.valueFrom + match.from,
|
||||
to: run.valueFrom + match.to,
|
||||
source: run.source.slice(match.from, match.to),
|
||||
value: match.username,
|
||||
})),
|
||||
])
|
||||
.sort((left, right) => left.from - right.from || left.to - right.to);
|
||||
if (matches.length === 0) return [{ type: "text", value }];
|
||||
|
||||
const segments: Segment[] = [];
|
||||
let cursor = 0;
|
||||
for (const tag of tags) {
|
||||
if (cursor < tag.from) segments.push({ type: "text", value: value.slice(cursor, tag.from) });
|
||||
segments.push({ type: "tag", source: tag.source, value: tag.value });
|
||||
cursor = tag.to;
|
||||
for (const match of matches) {
|
||||
if (match.from < cursor) continue;
|
||||
if (cursor < match.from) segments.push({ type: "text", value: value.slice(cursor, match.from) });
|
||||
segments.push({ type: match.type, source: match.source, value: match.value });
|
||||
cursor = match.to;
|
||||
}
|
||||
if (cursor < value.length) segments.push({ type: "text", value: value.slice(cursor) });
|
||||
return segments;
|
||||
|
|
@ -374,6 +393,23 @@ function createTagNode(tagValue: string, source: string): TagNode {
|
|||
} as TagNode;
|
||||
}
|
||||
|
||||
function createMentionNode(username: string, source: string): MentionNode {
|
||||
const data: MentionNodeData = {
|
||||
hName: "span",
|
||||
hProperties: {
|
||||
className: "mention",
|
||||
"data-mention": username,
|
||||
},
|
||||
hChildren: [{ type: "text", value: source }],
|
||||
};
|
||||
|
||||
return {
|
||||
type: "mentionNode",
|
||||
value: username,
|
||||
data,
|
||||
} as MentionNode;
|
||||
}
|
||||
|
||||
type ParentNode = UnistNode & { children: UnistNode[] };
|
||||
|
||||
function isParentNode(node: UnistNode): node is ParentNode {
|
||||
|
|
@ -680,7 +716,7 @@ function reconcileTextNodes(
|
|||
reconcileUnderscoreEmailEmphasis(parent, source, context, recoveredSpans);
|
||||
}
|
||||
|
||||
function transformTagTextNodes(
|
||||
function transformMemoTextNodes(
|
||||
parent: ParentNode,
|
||||
source: string,
|
||||
context: MarkdownSourceContext,
|
||||
|
|
@ -694,29 +730,52 @@ function transformTagTextNodes(
|
|||
const segments = segmentsForTextNode(textNode.value, nodeSpan(textNode, recoveredSpans), source, context.boundaries);
|
||||
if (!segments || segments.every((segment) => segment.type === "text")) continue;
|
||||
|
||||
const newNodes = segments.map((segment) =>
|
||||
segment.type === "tag" ? createTagNode(segment.value, segment.source) : ({ type: "text", value: segment.value } as Text),
|
||||
);
|
||||
const newNodes = segments.map((segment) => {
|
||||
if (segment.type === "tag") return createTagNode(segment.value, segment.source);
|
||||
if (segment.type === "mention") return createMentionNode(segment.value, segment.source);
|
||||
return { type: "text", value: segment.value } as Text;
|
||||
});
|
||||
parent.children.splice(index, 1, ...(newNodes as UnistNode[]));
|
||||
index += newNodes.length - 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (isParentNode(child) && TRANSPARENT_PARENT_TYPES.has(child.type)) {
|
||||
transformTagTextNodes(child, source, context, recoveredSpans);
|
||||
transformMemoTextNodes(child, source, context, recoveredSpans);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type VFileLike = { value?: string | Uint8Array };
|
||||
|
||||
export const remarkTag = () => {
|
||||
function transformMemoSyntax(tree: Root, rawSource: string): void {
|
||||
const source = rawSource.startsWith("\uFEFF") ? rawSource.slice(1) : rawSource;
|
||||
const context = markdownSourceContext(source);
|
||||
const recoveredSpans = new WeakMap<UnistNode, SourceSpan>();
|
||||
reconcileTextNodes(tree as ParentNode, source, context, recoveredSpans);
|
||||
transformMemoTextNodes(tree as ParentNode, source, context, recoveredSpans);
|
||||
}
|
||||
|
||||
/** Transform Memos inline source syntaxes after GFM and math parsing. */
|
||||
export const remarkMemoSyntax = () => {
|
||||
return (tree: Root, file: VFileLike) => {
|
||||
const rawSource = typeof file.value === "string" ? file.value : "";
|
||||
const source = rawSource.startsWith("\uFEFF") ? rawSource.slice(1) : rawSource;
|
||||
const context = markdownSourceContext(source);
|
||||
const recoveredSpans = new WeakMap<UnistNode, SourceSpan>();
|
||||
reconcileTextNodes(tree as ParentNode, source, context, recoveredSpans);
|
||||
transformTagTextNodes(tree as ParentNode, source, context, recoveredSpans);
|
||||
transformMemoSyntax(tree, typeof file.value === "string" ? file.value : "");
|
||||
};
|
||||
};
|
||||
|
||||
/** Extract exact mention candidates with the same Markdown rules used for rendering. */
|
||||
export function extractMentionUsernames(source: string): string[] {
|
||||
const tree = fromMarkdown(source, {
|
||||
extensions: [gfm()],
|
||||
mdastExtensions: [gfmFromMarkdown()],
|
||||
});
|
||||
transformMemoSyntax(tree, source);
|
||||
|
||||
const usernames: string[] = [];
|
||||
const collect = (node: UnistNode): void => {
|
||||
if (node.type === "mentionNode") usernames.push((node as MentionNode).value);
|
||||
if (isParentNode(node)) node.children.forEach(collect);
|
||||
};
|
||||
collect(tree);
|
||||
return Array.from(new Set(usernames));
|
||||
}
|
||||
|
|
|
|||
20
web/src/utils/username.ts
Normal file
20
web/src/utils/username.ts
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
/** Maximum number of ASCII characters in a writable username. */
|
||||
export const MAX_USERNAME_LENGTH = 36;
|
||||
|
||||
const ASCII_ALPHANUMERIC = /^[A-Za-z0-9]$/;
|
||||
|
||||
/** Whether a character may occur anywhere in a writable username. */
|
||||
export function isUsernameCharacter(char: string): boolean {
|
||||
return ASCII_ALPHANUMERIC.test(char) || char === "-";
|
||||
}
|
||||
|
||||
/** Whether a value satisfies the writable username format. */
|
||||
export function isValidUsername(username: string): boolean {
|
||||
if (username.length === 0 || username.length > MAX_USERNAME_LENGTH) return false;
|
||||
if (!ASCII_ALPHANUMERIC.test(username[0]) || !ASCII_ALPHANUMERIC.test(username.at(-1) ?? "")) return false;
|
||||
|
||||
for (const char of username) {
|
||||
if (!isUsernameCharacter(char)) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
|
@ -3,7 +3,7 @@ import { forceParsing } from "@codemirror/language";
|
|||
import { EditorState } from "@codemirror/state";
|
||||
import { EditorView } from "@codemirror/view";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { findMarkdownTagMatches } from "@/components/MemoEditor/Editor/markdownTagRanges";
|
||||
import { findMarkdownMentionMatches, findMarkdownTagMatches } from "@/components/MemoEditor/Editor/markdownTagRanges";
|
||||
import { tagMentionDecorations } from "@/components/MemoEditor/Editor/tagMentionDecorations";
|
||||
import { memoMarkdownExtensions } from "@/utils/memo-markdown-extension";
|
||||
|
||||
|
|
@ -214,5 +214,22 @@ describe("tag/mention decorations", () => {
|
|||
it("allows flow math to interrupt a paragraph", () => expect(countClass("text\n$$\n#math", "cm-memo-tag")).toBe(0));
|
||||
it("does not let same-line math hide following text", () => expect(countClass("$$#inline$$\n#ok", "cm-memo-tag")).toBe(1));
|
||||
it("allows ordinary text between inline HTML tags", () => expect(countClass("<span>#inside</span>", "cm-memo-tag")).toBe(1));
|
||||
it("decorates @mentions", () => expect(countClass("hi @alice", "cm-memo-mention")).toBe(1));
|
||||
it("decorates writable username references", () =>
|
||||
expect(countClass("@alice @Alice-2 @1alice @a--b @123 @123-456", "cm-memo-mention")).toBe(6));
|
||||
it("rejects invalid username shapes", () => expect(countClass("@-alice @alice- @álîçé", "cm-memo-mention")).toBe(0));
|
||||
it("applies mention boundaries to source text", () =>
|
||||
expect(countClass("hello@alice foo-@bob foo_@carol 中文@dave (@erin) @frank@grace", "cm-memo-mention")).toBe(4));
|
||||
it("decorates mentions in transparent formatting", () => expect(countClass("**@Alice** ~~@bob~~ _@carol_", "cm-memo-mention")).toBe(3));
|
||||
it("keeps mentions in opaque Markdown source undecorated", () =>
|
||||
expect(
|
||||
countClass("`@code` [@link](/x)  https://example.com/@url $@math$ \\@escaped @entity @ok", "cm-memo-mention"),
|
||||
).toBe(1));
|
||||
it("keeps GFM emails opaque and does not cross their source boundary", () =>
|
||||
expect(countClass("@alice@example.com foo@bar.com@bob @ok", "cm-memo-mention")).toBe(1));
|
||||
it("returns the exact mention source range and username", () => {
|
||||
const source = "x @Alice-2.";
|
||||
const state = EditorState.create({ doc: source, extensions: [markdown({ extensions: memoMarkdownExtensions })] });
|
||||
|
||||
expect(findMarkdownMentionMatches(state, 0, source.length)).toEqual([{ from: 2, to: 10, username: "Alice-2" }]);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,21 +1,56 @@
|
|||
import { describe, expect, it } from "vitest";
|
||||
import { parseMentionsFromText } from "@/utils/remark-plugins/remark-mention";
|
||||
import { extractMentionUsernames } from "@/utils/mention-extraction";
|
||||
import { findMentionMatches } from "@/utils/mention-grammar";
|
||||
import { MAX_USERNAME_LENGTH } from "@/utils/username";
|
||||
|
||||
describe("parseMentionsFromText", () => {
|
||||
it("accepts a 63-character username", () => {
|
||||
const username = `a${"b".repeat(62)}`;
|
||||
expect(parseMentionsFromText(`@${username}`)).toEqual([{ type: "mention", value: username }]);
|
||||
});
|
||||
|
||||
it("leaves a 64-character username as text", () => {
|
||||
const username = `a${"b".repeat(63)}`;
|
||||
expect(parseMentionsFromText(`@${username}`)).toEqual([{ type: "text", value: `@${username}` }]);
|
||||
});
|
||||
|
||||
it("keeps rejected adjacent mentions in the same literal text segment", () => {
|
||||
expect(parseMentionsFromText("@example.com@alice", "_")).toEqual([
|
||||
{ type: "mention", value: "example" },
|
||||
{ type: "text", value: ".com@alice" },
|
||||
describe("findMentionMatches", () => {
|
||||
it("recognizes complete writable usernames without changing case", () => {
|
||||
expect(findMentionMatches("@alice @Alice-2 @1alice @a--b @123 @123-456").map((match) => match.username)).toEqual([
|
||||
"alice",
|
||||
"Alice-2",
|
||||
"1alice",
|
||||
"a--b",
|
||||
"123",
|
||||
"123-456",
|
||||
]);
|
||||
});
|
||||
|
||||
it("accepts 36 characters and rejects the complete 37-character run", () => {
|
||||
const maximum = `a${"b".repeat(MAX_USERNAME_LENGTH - 1)}`;
|
||||
expect(findMentionMatches(`@${maximum}`).map((match) => match.username)).toEqual([maximum]);
|
||||
expect(findMentionMatches(`@${maximum}c`)).toEqual([]);
|
||||
});
|
||||
|
||||
it.each(["@-alice", "@alice-", "@álîçé"])('rejects the invalid username shape "%s"', (source) => {
|
||||
expect(findMentionMatches(source)).toEqual([]);
|
||||
});
|
||||
|
||||
it("uses username characters as the source boundary", () => {
|
||||
expect(findMentionMatches("hello@alice foo-@bob foo_@carol 中文@dave (@erin)").map((match) => match.username)).toEqual([
|
||||
"carol",
|
||||
"dave",
|
||||
"erin",
|
||||
]);
|
||||
expect(findMentionMatches("@alice", false)).toEqual([]);
|
||||
});
|
||||
|
||||
it("treats other characters as ordinary right boundaries", () => {
|
||||
expect(findMentionMatches("@alice_smith @bob@carol").map((match) => match.username)).toEqual(["alice", "bob"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("extractMentionUsernames", () => {
|
||||
it("uses the same Markdown contexts and exact spelling as rendering", () => {
|
||||
expect(
|
||||
extractMentionUsernames("**@Alice** ~~@bob~~ `@code` [@link](/x)  https://example.com/@url $@math$ @Alice"),
|
||||
).toEqual(["Alice", "bob"]);
|
||||
});
|
||||
|
||||
it("keeps escapes, character references, and GFM emails opaque", () => {
|
||||
expect(extractMentionUsernames("\\@escaped @entity @alice@example.com foo@bar.com@bob @ok")).toEqual(["ok"]);
|
||||
});
|
||||
|
||||
it("keeps case-distinct username references distinct", () => {
|
||||
expect(extractMentionUsernames("@Alice @alice @Alice")).toEqual(["Alice", "alice"]);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -4,18 +4,15 @@ import remarkBreaks from "remark-breaks";
|
|||
import remarkGfm from "remark-gfm";
|
||||
import remarkMath from "remark-math";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { remarkMention } from "@/utils/remark-plugins/remark-mention";
|
||||
import { remarkTag } from "@/utils/remark-plugins/remark-tag";
|
||||
import { remarkMemoSyntax } from "@/utils/remark-plugins/remark-tag";
|
||||
|
||||
const renderMarkdown = (content: string): string =>
|
||||
renderToStaticMarkup(
|
||||
<ReactMarkdown remarkPlugins={[remarkMath, remarkGfm, remarkTag, remarkMention, remarkBreaks]}>{content}</ReactMarkdown>,
|
||||
);
|
||||
renderToStaticMarkup(<ReactMarkdown remarkPlugins={[remarkMath, remarkGfm, remarkMemoSyntax, remarkBreaks]}>{content}</ReactMarkdown>);
|
||||
|
||||
const renderMarkdownWithoutMath = (content: string): string =>
|
||||
renderToStaticMarkup(<ReactMarkdown remarkPlugins={[remarkGfm, remarkTag, remarkMention, remarkBreaks]}>{content}</ReactMarkdown>);
|
||||
renderToStaticMarkup(<ReactMarkdown remarkPlugins={[remarkGfm, remarkMemoSyntax, remarkBreaks]}>{content}</ReactMarkdown>);
|
||||
|
||||
describe("remarkTag", () => {
|
||||
describe("remarkMemoSyntax", () => {
|
||||
it("does not turn URL fragments inside autolinks into tags", () => {
|
||||
const html = renderMarkdown("https://github.com/dmtrKovalenko/fff#pi-agent-extension\n\nProject #memo-tag");
|
||||
|
||||
|
|
@ -266,6 +263,28 @@ describe("remarkTag", () => {
|
|||
expect(html).toContain('<em><a href="mailto:foo@example.com">foo@example.com</a> <span class="tag" data-tag="tag">#tag</span></em>');
|
||||
});
|
||||
|
||||
it("renders only complete writable usernames and preserves their case", () => {
|
||||
const maximum = `A${"b".repeat(35)}`;
|
||||
const html = renderMarkdown(`@Alice-2 @1alice @123 @-alice @alice- @alice_smith @${maximum} @${maximum}c`);
|
||||
|
||||
expect(html).toContain('data-mention="Alice-2"');
|
||||
expect(html).toContain('data-mention="1alice"');
|
||||
expect(html).toContain(`data-mention="${maximum}"`);
|
||||
expect(html).toContain('data-mention="123"');
|
||||
expect(html).not.toContain('data-mention="-alice"');
|
||||
expect(html).not.toContain('data-mention="alice-"');
|
||||
expect(html).toContain('data-mention="alice"');
|
||||
expect(html).not.toContain(`data-mention="${maximum}c"`);
|
||||
});
|
||||
|
||||
it("uses Markdown structure for mention eligibility", () => {
|
||||
const html = renderMarkdown("**@Alice** `@code` [@link](/x) https://example.com/@url $@math$ \\@escaped @entity @ok");
|
||||
|
||||
expect(html).toContain('data-mention="Alice"');
|
||||
expect(html).toContain('data-mention="ok"');
|
||||
expect(html.match(/data-mention=/g)).toHaveLength(2);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"__foo@example.com #tag__ @carol",
|
||||
|
|
@ -331,7 +350,7 @@ describe("remarkTag", () => {
|
|||
expect(strongHTML).toContain('data-mention="alice"');
|
||||
expect(emphasisHTML).toContain('data-mention="alice"');
|
||||
expect(tagHTML).not.toContain('data-mention="alice"');
|
||||
expect(adjacentMentionsHTML.match(/data-mention=/g)).toHaveLength(1);
|
||||
expect(adjacentMentionsHTML).toContain('data-mention="alice"');
|
||||
expect(adjacentMentionsHTML).not.toContain('data-mention="bob"');
|
||||
});
|
||||
|
||||
|
|
|
|||
30
web/tests/username.test.ts
Normal file
30
web/tests/username.test.ts
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
import { describe, expect, test } from "vitest";
|
||||
import { isValidUsername, MAX_USERNAME_LENGTH } from "@/utils/username";
|
||||
|
||||
describe("isValidUsername", () => {
|
||||
test.each([
|
||||
["alice", true],
|
||||
["Alice-2", true],
|
||||
["1alice", true],
|
||||
["a--b", true],
|
||||
["a---b", true],
|
||||
[`a${"b".repeat(MAX_USERNAME_LENGTH - 1)}`, true],
|
||||
["", false],
|
||||
["a".repeat(MAX_USERNAME_LENGTH + 1), false],
|
||||
["123", true],
|
||||
["123-456", true],
|
||||
["00000000-0000-0000-0000-000000000000", true],
|
||||
["-", false],
|
||||
["---", false],
|
||||
["-alice", false],
|
||||
["alice-", false],
|
||||
["alice_smith", false],
|
||||
["alice@example.com", false],
|
||||
[" alice ", false],
|
||||
["alice smith", false],
|
||||
["álîçé", false],
|
||||
["张三", false],
|
||||
])("validates %j", (username, expected) => {
|
||||
expect(isValidUsername(username)).toBe(expected);
|
||||
});
|
||||
});
|
||||
Loading…
Reference in a new issue