feat(user): define username format and mention syntax (#6134)

This commit is contained in:
Johnny 2026-08-02 19:57:16 +08:00 committed by GitHub
parent 464e45b447
commit 8648b97b9e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
47 changed files with 1160 additions and 420 deletions

View file

@ -0,0 +1,260 @@
# ADR 0002: Username Format and References
Status: Accepted
Date: 2026-08-02
Domain glossary: [Memos domain glossary](../glossary.md)
## Context
A username is a user identifier used by account creation and updates, authentication, public user resource names, API lookup, SSO provisioning, and
references to users. Those uses must share one format rather than defining separate username-like tokens.
Memos already restricts newly written usernames to ASCII letters, digits, and hyphens, with a 36-character limit and alphanumeric ends. Existing
installations may also contain older usernames such as email addresses or values with underscores. Memos continues to read and resolve those legacy
values, but they are not valid new usernames.
Markdown mentions currently drift from the writable username format: the backend accepts up to 63 Unicode letters, numbers, and hyphens; the frontend
accepts up to 63 ASCII characters; both allow invalid username shapes; and extraction lowercases the result. Editor decoration also scans raw source in
contexts where Markdown must be opaque.
This ADR defines:
- The format and case-preservation semantics for writable usernames.
- How username references resolve.
- Markdown mention as one source form of username reference.
Changing or migrating legacy username data is outside this ADR's scope.
Username rename and reuse policy, and the durable binding of an already resolved mention across those lifecycle events, are also outside scope.
Notification timing, deduplication, access policy, self-mention behavior, and other downstream effects are unchanged and outside scope.
## Decision drivers
- Give every username write path one small, stable validation rule.
- Make a reference use the username format instead of inventing another identifier grammar.
- Preserve the spelling and case selected by the user.
- Keep username write validation, reference rendering, extraction, editor recognition, and notification resolution aligned.
- Use parsed Markdown context rather than special-case regular expressions for code, links, email addresses, and other opaque syntax.
- Keep recognition deterministic and linear in memo size.
## Terminology
**User**
: A durable Memos account identified internally by an immutable user ID.
**Username**
: A user-selected, case-sensitive public identifier used for account addressing. A username is distinct from a display name and from the user's internal
ID.
**Writable username**
: A username accepted when an account is created, renamed, or provisioned after this decision.
**Legacy username**
: A stored username that does not satisfy the writable format but remains readable and addressable for compatibility.
**Username reference**
: Source text that names a username so Memos can attempt to resolve it to a user ID. The source spelling is not itself a durable relationship.
**Mention candidate**
: A Markdown source span with the form `@` followed by a complete writable username and satisfying the mention boundary and context rules.
**Resolved mention**
: A mention candidate whose username resolves to a user under the consuming operation's existing account-status and visibility policy.
## Decision
### Writable username format
The normative grammar is:
```text
Username := Alphanumeric
| Alphanumeric UsernameCharacter{0,34} Alphanumeric
UsernameCharacter := Alphanumeric | "-"
Alphanumeric := ASCII letter | ASCII digit
ASCII letter := "A".."Z" | "a".."z"
ASCII digit := "0".."9"
```
- A username is 1 through 36 ASCII characters.
- ASCII letters, digits, and `-` are the only admitted characters.
- The first and last characters are letters or digits.
- Consecutive interior hyphens are valid; there is no additional rule limiting their number.
- A digits-only value is valid.
- Case is preserved.
Requiring alphanumeric ends also excludes values made only of hyphens and prevents a trailing hyphen from becoming an ambiguous-looking part of a
Markdown mention.
The minimum is one because Username needs no product-level namespace-scarcity policy beyond being non-empty. The maximum is 36 so every UUID value has a
canonical textual representation that fits while public URLs and reference tokens remain bounded.
The ASCII-only restriction is intentional: Username is a machine-facing public identifier used in URLs, authentication, and references. Internationalized
human-readable naming belongs to Display Name rather than expanding Username normalization and comparison rules.
Examples:
| Value | Writable | Reason |
| --- | --- | --- |
| `alice` | yes | Letters only |
| `Alice-2` | yes | Case is preserved; interior hyphen is valid |
| `1alice` | yes | A digit may begin a username that also contains a letter |
| `a--b` | yes | Consecutive interior hyphens are valid |
| `a---b` | yes | Interior hyphens have no repetition limit |
| `123` | yes | Numeric spelling has no special meaning |
| `123-456` | yes | Digits and interior hyphens satisfy the grammar |
| `00000000-0000-0000-0000-000000000000` | yes | A canonical UUID occupies 36 characters |
| `-alice` | no | A hyphen cannot begin a username |
| `alice-` | no | A hyphen cannot end a username |
| `alice_smith` | no | `_` is outside the format |
| `alice@example.com` | no | Email syntax is outside the format |
| `álîçé` | no | The writable format is ASCII |
| `张三` | no | Internationalized naming belongs to Display Name |
All account creation, username update, and automatic provisioning paths use this rule. A generic resource-name rule is not the username contract and
must not be reused as a substitute merely because its current regular expression is similar.
### Username allocation
The writable format has no reserved spellings. Values such as `admin`, `root`, `system`, `api`, `memos`, and `support` are ordinary usernames and are
available subject to the same exact-case uniqueness rule as every other value. A username does not grant authority or establish trust; those properties
come from the resolved user ID, role, and explicit product presentation. Manual registration, rename, and automatic provisioning apply the same policy.
If Memos later needs a system-owned username, the corresponding account claims it through ordinary uniqueness rather than expanding a lexical reserved
list.
SSO provisioning adopts the external identifier exactly when it is a valid, available writable username. If it is invalid or already belongs to another
user, provisioning assigns a generated canonical UUID instead. A matching username never links or takes over an account; only the persisted identity
binding formed by the identity-provider key and external subject identifies a returning SSO user.
### Username identity and resolution
Username spelling is case-preserving and username equality is exact ASCII byte equality. Memos does not lowercase, case-fold, or otherwise normalize a
username before validation, uniqueness checks, authentication, resource lookup, or reference resolution. In particular, surrounding whitespace is not
trimmed into a different username; a value such as ` alice ` is invalid. `Alice` and `alice` are distinct usernames and may identify different users.
All storage backends and application lookup paths must preserve this equality rule. A database collation that treats `Alice` and `alice` as equal, or a
lookup that silently resolves one spelling to the other, does not conform to this decision.
The `username` column therefore uses an explicit case-sensitive binary collation on every supported storage backend: `utf8mb4_bin` on MySQL, `C` on
PostgreSQL, and `BINARY` on SQLite. For the writable ASCII format, this makes exact-byte uniqueness and lookup a schema property rather than an assumption
about each database's default collation.
A username-bearing field or resource-name segment always resolves as a username, including when its spelling contains only digits. Character shape must
not select the internal user-ID namespace; an operation that addresses a user by internal ID must expose that choice separately.
The stable identity of a user is the internal user ID. A username reference is resolved as late as the consuming operation permits:
1. Preserve the username exactly as written.
2. Look up that exact username in the operation's visibility and account-status scope.
3. If it resolves, use the resulting user ID for durable effects.
4. If it does not resolve, retain ordinary source text and produce no user-targeted effect.
Product intent permits username rename and later reuse while requiring an already resolved mention not to change users. A source spelling alone cannot
satisfy all three properties: preserving the original target requires a durable binding to the user ID outside the `@username` text. This ADR does not
define that binding, historical re-resolution behavior, or the username lifecycle needed to support it.
Legacy usernames remain valid exact lookup keys for existing API resource names and authentication flows. They are not accepted by write validation and
are not added to new reference syntaxes. Changing legacy username values remains outside this decision, but every lookup preserves their stored spelling.
### Markdown mention syntax
A Markdown mention is the first defined username-reference syntax:
```text
MentionCandidate := "@" Username
```
The `Username` production is the complete writable username format above. Mention parsing has no independent character set or length limit.
Candidate recognition follows these rules:
1. The `@` is U+0040 COMMERCIAL AT in an eligible literal-source run.
2. At the start of the document, the left boundary is satisfied. Otherwise the immediately preceding source character must not be a
`UsernameCharacter`. The username format therefore defines the boundary: a preceding letter, digit, or hyphen blocks recognition.
3. The lexer consumes the complete consecutive run of ASCII letters, digits, and hyphens after `@`, then validates that whole run as a writable username.
It does not shorten an invalid run into a valid prefix.
4. GFM email-address recognition takes precedence. No substring of a recognized email address is a mention.
Characters outside `UsernameCharacter`, including `_`, `@`, and non-ASCII characters, are ordinary boundaries and receive no special handling.
The rules produce:
| Source | Mention candidate |
| --- | --- |
| `@alice` | `alice` |
| `@Alice-2` | `Alice-2` |
| `hi, @alice.` | `alice` |
| `中文@alice` | `alice` |
| `hello@alice` | none |
| `foo-@alice` | none |
| `foo_@alice` | `alice` |
| `@123` | `123` |
| `@-alice` | none |
| `@alice-` | none |
| `@alice_smith` | `alice` |
| `@alice@example` | `alice` |
| `@alice@bob` | `alice` |
| `@alice@example.com` | none |
| `@` followed by 37 valid username characters | none |
### Markdown context
Mention recognition operates on original Markdown source after GFM and Memos block and inline structure is resolved. It uses the same literal-source-run
and opaque-node model as other Memos inline extensions:
- Paragraphs, headings, block quotes, list items, table text, emphasis, strong emphasis, and strikethrough expose ordinary text.
- Code spans, code blocks, resolved links and images, autolinks, recognized GFM email addresses, raw HTML, and math are opaque.
- Escapes, character references, line endings, and syntax delimiters end a literal-source run. An escaped or character-reference-produced `@` is not an
introducer.
- Unknown future Markdown extension nodes are opaque unless their definition explicitly exposes ordinary text.
Parsing, read rendering, metadata extraction, editor decoration, and any future mention completion must use these same lexical and context rules.
This decision changes only which exact username spellings are extracted as mention candidates. Existing resolution, rendering, notification, and access
policies consume those candidates without otherwise changing behavior. Autocomplete, suggestion ranking, and a dedicated structured mention payload are
also outside this decision.
## Consequences
- Username validation becomes a domain primitive shared by every write path.
- Username uniqueness, authentication, resource lookup, and reference resolution use the same case-sensitive equality rule on every storage backend.
- Mention syntax automatically follows future username-format decisions only when this ADR is superseded and both uses are updated together.
- Existing valid mixed-case usernames can be mentioned without lowercasing.
- Values accepted only by the old 63-character or Unicode mention lexers stop being mention candidates because they were never writable usernames.
- Legacy email-like or underscore-containing usernames remain usable through compatibility lookup but cannot be authored as new Markdown mentions.
- Markdown-aware recognition removes false mentions from code, links, email addresses, HTML, math, escapes, and character references.
- Username rename, reuse, and historical mention stability remain an explicit follow-up because source text alone cannot represent the intended stable
user binding.
## Alternatives considered
### Define mention syntax first
Rejected because it creates a second username-like grammar and allows account validation and references to drift again.
### Allow a broader mention token than writable usernames
Rejected because such candidates cannot reliably identify newly writable accounts and complicate boundaries, validation, and UI messaging.
### Store stable user IDs with a mention
Deferred. Stable IDs make renames and username reuse robust but require either a new source representation or persisted occurrence bindings, plus editing
UX, import/export behavior, and compatibility policy. Exact username references are sufficient for candidate recognition in the current scope, but do not
solve historical binding.
### Unicode usernames
Deferred. A safe Unicode identifier design needs a pinned Unicode version, normalization, script and confusable policy, and consistent database collation.
ASCII keeps this decision small and matches the existing writable format.
## References
- [GitHub Flavored Markdown 0.29-gfm](https://github.github.com/gfm/)
- [GitHub: Mentioning people and teams](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams)
- [Slack message formatting: mentioning users](https://docs.slack.dev/messaging/formatting-message-text/#mentioning-users)
- [Matrix client-server API: user and room mentions](https://spec.matrix.org/latest/client-server-api/#user-and-room-mentions)
- [Activity Streams 2.0 Vocabulary: Mention](https://www.w3.org/TR/activitystreams-vocabulary/#dfn-mention)
- [Unicode Standard Annex #31: Unicode Identifiers and Syntax](https://www.unicode.org/reports/tr31/)
- [Unicode Technical Standard #39: Unicode Security Mechanisms](https://www.unicode.org/reports/tr39/)

View file

@ -8,6 +8,7 @@ implementation work is complete.
| ADR | Title | Status | Date |
| --- | --- | --- | --- |
| [0001](0001-tag-syntax-and-recognition.md) | Tag Syntax and Recognition | Accepted | 2026-08-01 |
| [0002](0002-username-format-and-references.md) | Username Format and References | Accepted | 2026-08-02 |
## Conventions

View file

@ -3,6 +3,47 @@
This glossary defines product and domain language shared across Memos design documents. Protocol, Unicode, and parser-specific terms remain in the ADRs
that use them.
## Users and usernames
### User
A durable Memos account. Its stable identity is an internal user ID, not its mutable username or display name.
### Username
A user-selected, case-sensitive public account identifier used for authentication, user resource names, lookup, and references. Username spelling is
case-preserving, is not implicitly trimmed or normalized, and equality is exact ASCII byte equality, so `Alice` and `alice` are distinct usernames. A
username is distinct from both a display name and the user's internal ID. Its spelling does not grant a role or establish that the user is trusted.
### Writable username
A username accepted for account creation, rename, or automatic provisioning. It is 1 through 36 ASCII letters, digits, or hyphens; begins and ends with an
ASCII letter or digit; and may contain repeated interior hyphens without an additional restriction. Numeric spelling has no special meaning, so a value
containing only digits is also a writable username. Internationalized human-readable naming belongs to the user's display name.
### Legacy username
A stored username that does not satisfy the writable format but remains readable and addressable for compatibility. Legacy usernames are not accepted by
new write validation or added to new reference syntaxes.
### Username reference
Source text that names an exact, case-sensitive username so Memos can attempt to resolve it to a user ID. The source spelling alone is not a durable user
binding. Stability across username rename and reuse requires additional persisted identity that the current username-reference specification does not
define.
### Mention candidate
An eligible memo Markdown source span formed by an ASCII `@` followed by one complete writable username. Recognition also applies the boundary and opaque
Markdown-context rules in the active username-reference specification. The same ASCII letters, digits, and hyphen admitted inside a username define the
mention boundaries: they block a mention immediately to their right and are consumed as part of the complete candidate to their left. Other characters
receive no special boundary handling.
### Resolved mention
A mention candidate whose exact username resolves to a user under the consuming operation's existing account-status and visibility policy. User-targeted
effects apply to the resolved user ID, not to unresolved source text.
## Tags
### Tag

35
internal/base/username.go Normal file
View file

@ -0,0 +1,35 @@
package base
// MaxUsernameLength is the maximum number of ASCII characters in a writable username.
const MaxUsernameLength = 36
// IsValidUsername reports whether username satisfies the writable username format.
func IsValidUsername(username string) bool {
if len(username) == 0 || len(username) > MaxUsernameLength || !isASCIIAlphanumeric(username[0]) || !isASCIIAlphanumeric(username[len(username)-1]) {
return false
}
for i := 0; i < len(username); i++ {
if !IsUsernameCharacter(username[i]) {
return false
}
}
return true
}
// IsUsernameCharacter reports whether an ASCII byte may occur in a writable username.
func IsUsernameCharacter(char byte) bool {
return isASCIIAlphanumeric(char) || char == '-'
}
func isASCIIAlphanumeric(char byte) bool {
return isASCIILetter(char) || isASCIIDigit(char)
}
func isASCIILetter(char byte) bool {
return char >= 'A' && char <= 'Z' || char >= 'a' && char <= 'z'
}
func isASCIIDigit(char byte) bool {
return char >= '0' && char <= '9'
}

View file

@ -0,0 +1,43 @@
package base
import (
"strings"
"testing"
)
func TestIsValidUsername(t *testing.T) {
tests := []struct {
username string
want bool
}{
{username: "alice", want: true},
{username: "Alice-2", want: true},
{username: "1alice", want: true},
{username: "a--b", want: true},
{username: "a---b", want: true},
{username: "a" + strings.Repeat("b", MaxUsernameLength-1), want: true},
{username: ""},
{username: strings.Repeat("a", MaxUsernameLength+1)},
{username: "123", want: true},
{username: "123-456", want: true},
{username: "00000000-0000-0000-0000-000000000000", want: true},
{username: "-"},
{username: "---"},
{username: "-alice"},
{username: "alice-"},
{username: "alice_smith"},
{username: "alice@example.com"},
{username: " alice "},
{username: "alice smith"},
{username: "álîçé"},
{username: "张三"},
}
for _, test := range tests {
t.Run(test.username, func(t *testing.T) {
if got := IsValidUsername(test.username); got != test.want {
t.Fatalf("IsValidUsername(%q) = %t, want %t", test.username, got, test.want)
}
})
}
}

View file

@ -10,6 +10,8 @@ type MentionNode struct {
// Username without the @ prefix.
Username []byte
// Source is the exact recognized source spelling, including the @ prefix.
Source []byte
}
// KindMention is the NodeKind for MentionNode.
@ -24,5 +26,6 @@ func (*MentionNode) Kind() gast.NodeKind {
func (n *MentionNode) Dump(source []byte, level int) {
gast.DumpHelper(n, source, level, map[string]string{
"Username": string(n.Username),
"Source": string(n.Source),
}, nil)
}

View file

@ -2,23 +2,103 @@ package extensions
import (
"github.com/yuin/goldmark"
"github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/parser"
"github.com/yuin/goldmark/renderer"
"github.com/yuin/goldmark/text"
"github.com/yuin/goldmark/util"
"github.com/usememos/memos/internal/base"
mast "github.com/usememos/memos/internal/markdown/ast"
mparser "github.com/usememos/memos/internal/markdown/parser"
)
type mentionExtension struct{}
// MentionExtension is a goldmark extension for @mention syntax.
// MentionExtension is a goldmark extension for username references using @username syntax.
var MentionExtension = &mentionExtension{}
// Extend extends the goldmark parser with mention support.
// Extend extends the goldmark parser with username-reference support.
func (*mentionExtension) Extend(m goldmark.Markdown) {
m.Parser().AddOptions(
parser.WithInlineParsers(
// Priority 200 - run before standard link parser (500).
util.Prioritized(mparser.NewMentionParser(), 200),
parser.WithASTTransformers(
// GFM email recognition runs at 950 and tag recognition at 1000.
util.Prioritized(&mentionASTTransformer{}, 1050),
),
)
m.Renderer().AddOptions(
renderer.WithNodeRenderers(
util.Prioritized(&mentionNodeRenderer{}, 500),
),
)
}
type mentionASTTransformer struct{}
func (*mentionASTTransformer) Transform(document *ast.Document, reader text.Reader, _ parser.Context) {
source := reader.Source()
textNodes := eligibleLiteralTextNodes(document)
mergeAdjacentLiteralText(textNodes, source)
for _, textNode := range textNodes {
if textNode.Parent() == nil {
continue
}
replaceMentionsInText(textNode, source)
}
}
func replaceMentionsInText(textNode *ast.Text, source []byte) {
segment := textNode.Segment
runHasLeftBoundary := segment.Padding > 0 || segment.Start == 0 || !base.IsUsernameCharacter(source[segment.Start-1])
matches := mparser.FindMentionMatches(segment.Value(source), runHasLeftBoundary)
if len(matches) == 0 {
return
}
parent := textNode.Parent()
cursor := segment.Start
padding := segment.Padding
for _, match := range matches {
start := segment.Start + match.Start - segment.Padding
end := segment.Start + match.End - segment.Padding
if start > cursor || padding > 0 {
insertSplitTextBefore(parent, textNode, textNode, text.NewSegmentPadding(cursor, start, padding), false)
}
mentionNode := &mast.MentionNode{
Username: append([]byte(nil), match.Username...),
Source: append([]byte(nil), source[start:end]...),
}
mentionNode.SetPos(start)
parent.InsertBefore(parent, textNode, mentionNode)
cursor = end
padding = 0
}
if cursor < segment.Stop || textNode.SoftLineBreak() || textNode.HardLineBreak() {
insertSplitTextBefore(parent, textNode, textNode, text.NewSegment(cursor, segment.Stop), true)
}
parent.RemoveChild(parent, textNode)
}
type mentionNodeRenderer struct{}
func (*mentionNodeRenderer) RegisterFuncs(registerer renderer.NodeRendererFuncRegisterer) {
registerer.Register(mast.KindMention, renderMentionNode)
}
func renderMentionNode(writer util.BufWriter, _ []byte, node ast.Node, entering bool) (ast.WalkStatus, error) {
if !entering {
return ast.WalkContinue, nil
}
mentionNode, ok := node.(*mast.MentionNode)
if !ok {
return ast.WalkContinue, nil
}
spelling := mentionNode.Source
if len(spelling) == 0 {
spelling = append([]byte{'@'}, mentionNode.Username...)
}
_, _ = writer.Write(util.EscapeHTML(spelling))
return ast.WalkContinue, nil
}

View file

@ -0,0 +1,25 @@
package extensions
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/text"
mast "github.com/usememos/memos/internal/markdown/ast"
)
func TestReplaceMentionsInPaddedText(t *testing.T) {
source := []byte("x@alice")
paragraph := ast.NewParagraph()
textNode := ast.NewTextSegment(text.NewSegmentPadding(1, len(source), 2))
paragraph.AppendChild(paragraph, textNode)
replaceMentionsInText(textNode, source)
mention, ok := paragraph.LastChild().(*mast.MentionNode)
require.True(t, ok)
assert.Equal(t, "alice", string(mention.Username))
}

View file

@ -418,7 +418,7 @@ func (s *service) ExtractAll(content []byte) (*ExtractedData, error) {
data.Tags = appendTagHierarchy(data.Tags, string(tagNode.Tag))
}
if mentionNode, ok := n.(*mast.MentionNode); ok {
data.Mentions = append(data.Mentions, strings.ToLower(string(mentionNode.Username)))
data.Mentions = append(data.Mentions, string(mentionNode.Username))
}
// Check if the first block-level child of the document is an H1 heading.

View file

@ -6,6 +6,8 @@ import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/usememos/memos/internal/base"
)
func TestNewService(t *testing.T) {
@ -346,10 +348,10 @@ func TestExtractAllMentions(t *testing.T) {
data, err := svc.ExtractAll([]byte("Hi @Alice and @bob. Email support@example.com should stay plain. #tag"))
require.NoError(t, err)
assert.ElementsMatch(t, []string{"alice", "bob"}, data.Mentions)
assert.ElementsMatch(t, []string{"Alice", "bob"}, data.Mentions)
assert.ElementsMatch(t, []string{"tag"}, data.Tags)
maxLengthUsername := "a" + strings.Repeat("b", 62)
maxLengthUsername := "a" + strings.Repeat("b", base.MaxUsernameLength-1)
data, err = svc.ExtractAll([]byte("@" + maxLengthUsername))
require.NoError(t, err)
assert.Equal(t, []string{maxLengthUsername}, data.Mentions)
@ -359,6 +361,47 @@ func TestExtractAllMentions(t *testing.T) {
assert.Empty(t, data.Mentions)
}
func TestExtractAllMentionSyntaxAndContexts(t *testing.T) {
svc := NewService(WithTagExtension(), WithMentionExtension())
tests := []struct {
name string
content string
expected []string
}{
{name: "writable usernames", content: "@alice @Alice-2 @1alice @a--b @123 @123-456", expected: []string{"alice", "Alice-2", "1alice", "a--b", "123", "123-456"}},
{name: "invalid username shapes", content: "@-alice @alice- @álîçé"},
{name: "left boundaries", content: "hello@alice foo-@bob foo_@carol 中文@dave (@erin)", expected: []string{"carol", "dave", "erin"}},
{name: "non-username right boundaries", content: "@alice_smith @bob@carol", expected: []string{"alice", "bob"}},
{name: "formatted text", content: "**@Alice** ~~@bob~~ _@carol_", expected: []string{"Alice", "bob", "carol"}},
{name: "opaque Markdown", content: "`@code` [@link](/x) ![@image](/x) https://example.com/@url $@math$ @ok", expected: []string{"ok"}},
{name: "escapes and references", content: `\@escaped &#64;entity @ok`, expected: []string{"ok"}},
{name: "GFM email precedence", content: "@alice@example.com foo@bar.com@bob @ok", expected: []string{"ok"}},
{name: "padded blockquote", content: ">\t@alice", expected: []string{"alice"}},
{name: "padded list continuation", content: "- item\n \t@bob", expected: []string{"bob"}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
data, err := svc.ExtractAll([]byte(test.content))
require.NoError(t, err)
assert.Equal(t, test.expected, data.Mentions)
})
}
}
func TestMentionSourceSpelling(t *testing.T) {
svc := NewService(WithTagExtension(), WithMentionExtension())
content := "Hello @Alice-2"
rendered, err := svc.RenderMarkdown([]byte(content))
require.NoError(t, err)
assert.Equal(t, content, rendered)
html, err := svc.RenderHTML([]byte(content))
require.NoError(t, err)
assert.Contains(t, html, "@Alice-2")
}
func TestExtractAllSkipsTagsInsideLinks(t *testing.T) {
svc := NewService(WithTagExtension())

View file

@ -1,92 +1,59 @@
package parser
import (
"bytes"
"unicode"
"unicode/utf8"
"github.com/yuin/goldmark/util"
gast "github.com/yuin/goldmark/ast"
"github.com/yuin/goldmark/parser"
"github.com/yuin/goldmark/text"
mast "github.com/usememos/memos/internal/markdown/ast"
"github.com/usememos/memos/internal/base"
)
const (
// MaxMentionLength matches the username token length accepted by the API.
MaxMentionLength = 63
)
type mentionParser struct{}
// NewMentionParser creates a new inline parser for @mention syntax.
func NewMentionParser() parser.InlineParser {
return &mentionParser{}
// MentionMatch is one username reference in an eligible literal-source run.
type MentionMatch struct {
// Start is the byte offset of the @ introducer within the source run.
Start int
// End is the exclusive byte offset of the recognized source spelling.
End int
// Username is the exact username without the @ introducer.
Username []byte
}
// Trigger returns the characters that trigger this parser.
func (*mentionParser) Trigger() []byte {
return []byte{'@'}
}
func isValidMentionRune(r rune) bool {
return unicode.IsLetter(r) || unicode.IsNumber(r) || r == '-'
}
func isMentionBoundary(r rune) bool {
return unicode.IsSpace(r) || unicode.IsPunct(r) || unicode.IsSymbol(r)
}
// Parse parses @mention syntax while avoiding email-address matches.
func (*mentionParser) Parse(_ gast.Node, block text.Reader, _ parser.Context) gast.Node {
line, segment := block.PeekLine()
if len(line) == 0 || line[0] != '@' {
return nil
}
lineStart := bytes.LastIndexByte(block.Source()[:segment.Start], '\n') + 1
if _, _, ok := MatchGFMEmailAt(block.Source(), segment.Start, lineStart, segment.Start+len(line)); ok {
return nil
func hasMentionLeftBoundary(source []byte, pos int, runHasLeftBoundary bool) bool {
if pos == 0 {
return runHasLeftBoundary
}
return !base.IsUsernameCharacter(source[pos-1])
}
prev := block.PrecendingCharacter()
if prev != '\n' && !isMentionBoundary(prev) {
return nil
}
start := 1
pos := start
runeCount := 0
hasLetterOrNumber := false
for pos < len(line) {
r, size := utf8.DecodeRune(line[pos:])
if r == utf8.RuneError && size == 1 {
break
// FindMentionMatches enumerates username references in one eligible literal-source run.
// Markdown structure and GFM email precedence must be resolved before calling it.
func FindMentionMatches(source []byte, runHasLeftBoundary bool) []MentionMatch {
var matches []MentionMatch
for pos := 0; pos < len(source); {
if source[pos] == '\\' && pos+1 < len(source) && util.IsPunct(source[pos+1]) {
pos += 2
continue
}
if !isValidMentionRune(r) {
break
if source[pos] != '@' || !hasMentionLeftBoundary(source, pos, runHasLeftBoundary) {
pos++
continue
}
if unicode.IsLetter(r) || unicode.IsNumber(r) {
hasLetterOrNumber = true
end := pos + 1
for end < len(source) && base.IsUsernameCharacter(source[end]) {
end++
}
runeCount++
if runeCount > MaxMentionLength {
return nil
username := source[pos+1 : end]
if base.IsValidUsername(string(username)) {
matches = append(matches, MentionMatch{
Start: pos,
End: end,
Username: append([]byte(nil), username...),
})
}
if end == pos+1 {
pos++
} else {
pos = end
}
pos += size
}
if pos <= start || !hasLetterOrNumber {
return nil
}
username := line[start:pos]
usernameCopy := make([]byte, len(username))
copy(usernameCopy, username)
block.Advance(pos)
return &mast.MentionNode{
Username: usernameCopy,
}
return matches
}

View file

@ -0,0 +1,42 @@
package parser
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/usememos/memos/internal/base"
)
func TestFindMentionMatches(t *testing.T) {
tests := []struct {
name string
source string
runHasLeftBoundary bool
want []string
}{
{name: "valid", source: "@alice @Alice-2 @123 @123-456", runHasLeftBoundary: true, want: []string{"alice", "Alice-2", "123", "123-456"}},
{name: "maximum length", source: "@a" + strings.Repeat("b", base.MaxUsernameLength-1), runHasLeftBoundary: true, want: []string{"a" + strings.Repeat("b", base.MaxUsernameLength-1)}},
{name: "too long", source: "@a" + strings.Repeat("b", base.MaxUsernameLength), runHasLeftBoundary: true},
{name: "invalid shapes", source: "@-alice @alice- @álîçé", runHasLeftBoundary: true},
{name: "username character left boundary", source: "hello@alice foo-@bob foo_@carol", runHasLeftBoundary: true, want: []string{"carol"}},
{name: "Unicode left boundary", source: "中文@alice", runHasLeftBoundary: true, want: []string{"alice"}},
{name: "run starts after word", source: "@alice", runHasLeftBoundary: false},
{name: "non-username right boundaries", source: "@alice_smith @bob@carol", runHasLeftBoundary: true, want: []string{"alice", "bob"}},
{name: "punctuation", source: "(@alice), @bob.", runHasLeftBoundary: true, want: []string{"alice", "bob"}},
{name: "escaped introducer", source: `\@alice @bob`, runHasLeftBoundary: true, want: []string{"bob"}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
matches := FindMentionMatches([]byte(test.source), test.runHasLeftBoundary)
var got []string
for _, match := range matches {
assert.Equal(t, "@"+string(match.Username), test.source[match.Start:match.End])
got = append(got, string(match.Username))
}
assert.Equal(t, test.want, got)
})
}
}

View file

@ -195,8 +195,12 @@ func (r *MarkdownRenderer) renderNode(node gast.Node, source []byte, depth int)
}
case *mast.MentionNode:
r.buf.WriteByte('@')
r.buf.Write(n.Username)
if len(n.Source) > 0 {
r.buf.Write(n.Source)
} else {
r.buf.WriteByte('@')
r.buf.Write(n.Username)
}
default:
// For unknown nodes, try to render children

View file

@ -312,7 +312,7 @@ message CreateUserRequest {
];
// Optional. The resource ID to use for this user. If set, it must equal
// user.username, follow the UID format, and not consist entirely of digits.
// user.username and follow the username format.
// Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
string user_id = 2 [(google.api.field_behavior) = OPTIONAL];

View file

@ -651,7 +651,7 @@ type CreateUserRequest struct {
// Required. The user to create.
User *User `protobuf:"bytes,1,opt,name=user,proto3" json:"user,omitempty"`
// Optional. The resource ID to use for this user. If set, it must equal
// user.username, follow the UID format, and not consist entirely of digits.
// user.username and follow the username format.
// Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
UserId string `protobuf:"bytes,2,opt,name=user_id,json=userId,proto3" json:"user_id,omitempty"`
// Optional. If set, validate the request but don't actually create the user.

View file

@ -1355,7 +1355,7 @@ paths:
in: query
description: |-
Optional. The resource ID to use for this user. If set, it must equal
user.username, follow the UID format, and not consist entirely of digits.
user.username and follow the username format.
Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
schema:
type: string

View file

@ -66,11 +66,11 @@ func (s *APIV1Service) resolveSSOUser(ctx context.Context, currentUser *store.Us
}
// createSSOUser prefers the mapped external identifier as the initial local
// username when it satisfies the local username rules and is not a reserved
// name. A database uniqueness conflict falls back to a generated UUID instead of
// linking the SSO identity to the existing same-named account. User and identity
// creation are committed atomically so a concurrent UUID fallback cannot win
// after another request has claimed the preferred username.
// username when it satisfies the local username rules. A database uniqueness
// conflict falls back to a generated UUID instead of linking the SSO identity to
// the existing same-named account. User and identity creation are committed
// atomically so a concurrent UUID fallback cannot win after another request has
// claimed the preferred username.
//
// tryUsername returns a non-nil user when the identity is resolved (either newly
// created or reconciled to a concurrent winner) and (nil, nil) when the username
@ -109,11 +109,8 @@ func (s *APIV1Service) createSSOUser(
return s.getLinkedSSOUser(ctx, provider, externUID)
}
// Only adopt the external identifier as the local username when it is a valid,
// non-reserved name; otherwise an attacker-influenceable identifier could
// squat a privileged or system handle. Reserved and invalid names fall back to
// an opaque UUID.
if err := validateWritableUsername(userInfo.Identifier); err == nil && !isReservedUsername(userInfo.Identifier) {
// Adopt any valid external identifier. Invalid names fall back to an opaque UUID.
if err := validateWritableUsername(userInfo.Identifier); err == nil {
user, err := tryUsername(userInfo.Identifier)
if err != nil {
return nil, err

View file

@ -1,8 +1,6 @@
package v1
import (
"strings"
"github.com/pkg/errors"
"github.com/usememos/memos/internal/util"
@ -10,27 +8,6 @@ import (
const ssoUsernameFallbackAttempts = 5
// reservedUsernames are names that must never be auto-assigned from an external
// identity provider. A user-influenceable identifier (e.g. an OIDC
// preferred_username) claiming one of these on first login would squat a
// privileged or system-suggestive handle, so it falls back to a UUID instead.
var reservedUsernames = map[string]struct{}{
"admin": {},
"administrator": {},
"api": {},
"memos": {},
"root": {},
"support": {},
"system": {},
}
// isReservedUsername reports whether username is reserved. The comparison is
// case-insensitive because some backends fold case on the username unique index.
func isReservedUsername(username string) bool {
_, ok := reservedUsernames[strings.ToLower(strings.TrimSpace(username))]
return ok
}
// deriveSSOUsername produces the local username for a new SSO-created user.
//
// UUID usernames are the fallback when the IdP identifier cannot safely be used

View file

@ -23,6 +23,7 @@ func TestSSOSignInUsesValidIdentifierAsUsername(t *testing.T) {
identifier string
}{
{name: "single character", identifier: "a"},
{name: "numeric", identifier: "12345"},
{name: "common username", identifier: "alice"},
{name: "uppercase and hyphens", identifier: "Alice-01"},
{name: "maximum length", identifier: strings.Repeat("a", 36)},
@ -56,7 +57,6 @@ func TestSSOSignInFallsBackForInvalidIdentifier(t *testing.T) {
name string
identifier string
}{
{name: "numeric", identifier: "12345"},
{name: "email", identifier: "alice@example.com"},
{name: "underscore", identifier: "alice_example"},
{name: "leading hyphen", identifier: "-alice"},
@ -125,22 +125,20 @@ func TestSSOSignInDoesNotTakeOverExistingUsername(t *testing.T) {
assertSingleSSOLink(ctx, t, ts, "username-collision", "alice", response.User.Username)
}
func TestSSOSignInDoesNotAdoptReservedUsername(t *testing.T) {
func TestSSOSignInAdoptsOrdinaryRoleLikeUsername(t *testing.T) {
for _, identifier := range []string{"admin", "Admin", "support", "root"} {
t.Run(identifier, func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()
ctx := context.Background()
mockIDP := newMockOAuthServer(t, "reserved-code", "reserved-token", map[string]any{"sub": identifier})
mockIDP := newMockOAuthServer(t, "role-like-code", "role-like-token", map[string]any{"sub": identifier})
defer mockIDP.Close()
idpName := createTestingOAuthIdentityProvider(ctx, t, ts, mockIDP.URL, "reserved-provider")
response, err := signInWithTestingSSO(ctx, ts, idpName, "reserved-code")
idpName := createTestingOAuthIdentityProvider(ctx, t, ts, mockIDP.URL, "role-like-provider")
response, err := signInWithTestingSSO(ctx, ts, idpName, "role-like-code")
require.NoError(t, err)
require.NotEqual(t, identifier, response.User.Username)
_, err = uuid.Parse(response.User.Username)
require.NoError(t, err, "reserved identifier must fall back to a UUID")
require.Equal(t, identifier, response.User.Username)
})
}
}

View file

@ -88,22 +88,22 @@ func TestUserResourceName(t *testing.T) {
require.Equal(t, "users/Gnammi", resp.Users[0].Name)
})
t.Run("CreateUser rejects all-numeric usernames", func(t *testing.T) {
t.Run("CreateUser accepts all-numeric usernames", func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()
_, err := ts.Service.CreateUser(ctx, &apiv1.CreateUserRequest{
created, err := ts.Service.CreateUser(ctx, &apiv1.CreateUserRequest{
User: &apiv1.User{
Username: "123",
Email: "123@example.com",
Password: "password123",
},
})
require.Error(t, err)
require.Contains(t, err.Error(), "invalid username")
require.NoError(t, err)
require.Equal(t, "users/123", created.Name)
})
t.Run("GetUser returns not found for numeric user resource names", func(t *testing.T) {
t.Run("GetUser does not interpret a numeric username as an internal ID", func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()

View file

@ -36,6 +36,21 @@ func TestBatchGetUsersReturnsExactUsernamesWithoutAuthentication(t *testing.T) {
require.True(t, ok)
}
func TestBatchGetUsersDoesNotNormalizeUsernames(t *testing.T) {
ctx := context.Background()
ts := NewTestService(t)
defer ts.Cleanup()
_, err := ts.CreateRegularUser(ctx, "batch-alpha")
require.NoError(t, err)
resp, err := ts.Service.BatchGetUsers(ctx, &apiv1.BatchGetUsersRequest{
Usernames: []string{" batch-alpha "},
})
require.NoError(t, err)
require.Empty(t, resp.Users)
}
func TestBatchGetUsersRejectsTooManyUsernames(t *testing.T) {
ctx := context.Background()
ts := NewTestService(t)

View file

@ -222,7 +222,7 @@ func TestCreateUserRegistration(t *testing.T) {
})
t.Run("CreateUser rejects invalid writable usernames", func(t *testing.T) {
for _, username := range []string{"alice@example.com", "legacy_user", "123"} {
for _, username := range []string{"alice@example.com", "legacy_user"} {
t.Run(username, func(t *testing.T) {
ts := NewTestService(t)
defer ts.Cleanup()

View file

@ -27,24 +27,12 @@ func parseUsernameFromName(name string) (string, error) {
}
func validateWritableUsername(username string) error {
if username == "" || isNumericUsername(username) || !base.UIDMatcher.MatchString(username) {
if !base.IsValidUsername(username) {
return errors.Errorf("invalid username %q", username)
}
return nil
}
func isNumericUsername(username string) bool {
if username == "" {
return false
}
for _, char := range username {
if char < '0' || char > '9' {
return false
}
}
return true
}
// ResolveUserByName resolves a username-based user resource name to a store user.
func ResolveUserByName(ctx context.Context, stores *store.Store, name string) (*store.User, error) {
username, err := parseUsernameFromName(name)

View file

@ -51,9 +51,8 @@ func TestValidateWritableUsername(t *testing.T) {
wantError: true,
},
{
name: "numeric",
username: "123",
wantError: true,
name: "numeric",
username: "123",
},
{
name: "email",

View file

@ -116,12 +116,11 @@ func (s *APIV1Service) ListUsers(ctx context.Context, request *v1pb.ListUsersReq
return response, nil
}
func normalizeBatchUsernames(usernames []string) ([]string, int) {
func uniqueBatchUsernames(usernames []string) ([]string, int) {
uniqueUsernames := make([]string, 0, len(usernames))
seen := make(map[string]struct{}, len(usernames))
nonEmptyCount := 0
for _, username := range usernames {
username = strings.TrimSpace(username)
if username == "" {
continue
}
@ -140,7 +139,7 @@ func (s *APIV1Service) BatchGetUsers(ctx context.Context, request *v1pb.BatchGet
return &v1pb.BatchGetUsersResponse{Users: []*v1pb.User{}}, nil
}
uniqueUsernames, nonEmptyUsernameCount := normalizeBatchUsernames(request.Usernames)
uniqueUsernames, nonEmptyUsernameCount := uniqueBatchUsernames(request.Usernames)
if nonEmptyUsernameCount > maxBatchGetUsers {
return nil, status.Errorf(codes.InvalidArgument, "too many usernames (max %d)", maxBatchGetUsers)
}

View file

@ -0,0 +1,2 @@
ALTER TABLE `user`
MODIFY `username` VARCHAR(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL;

View file

@ -11,7 +11,7 @@ CREATE TABLE `user` (
`created_ts` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_ts` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
`row_status` VARCHAR(256) NOT NULL DEFAULT 'NORMAL',
`username` VARCHAR(256) NOT NULL UNIQUE,
`username` VARCHAR(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL UNIQUE,
`role` VARCHAR(256) NOT NULL DEFAULT 'USER',
`email` VARCHAR(256) NOT NULL DEFAULT '',
`nickname` VARCHAR(256) NOT NULL DEFAULT '',

View file

@ -0,0 +1,2 @@
ALTER TABLE "user"
ALTER COLUMN username TYPE TEXT COLLATE "C";

View file

@ -11,7 +11,7 @@ CREATE TABLE "user" (
created_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
updated_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
row_status TEXT NOT NULL DEFAULT 'NORMAL',
username TEXT NOT NULL UNIQUE,
username TEXT COLLATE "C" NOT NULL UNIQUE,
role TEXT NOT NULL DEFAULT 'USER',
email TEXT NOT NULL DEFAULT '',
nickname TEXT NOT NULL DEFAULT '',

View file

@ -0,0 +1,24 @@
ALTER TABLE user RENAME TO user_old;
CREATE TABLE user (
id INTEGER PRIMARY KEY AUTOINCREMENT,
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
username TEXT COLLATE BINARY NOT NULL UNIQUE,
role TEXT NOT NULL DEFAULT 'USER',
email TEXT NOT NULL DEFAULT '',
nickname TEXT NOT NULL DEFAULT '',
password_hash TEXT NOT NULL,
avatar_url TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT ''
);
INSERT INTO user (
id, created_ts, updated_ts, row_status, username, role, email, nickname, password_hash, avatar_url, description
)
SELECT
id, created_ts, updated_ts, row_status, username, role, email, nickname, password_hash, avatar_url, description
FROM user_old;
DROP TABLE user_old;

View file

@ -12,7 +12,7 @@ CREATE TABLE user (
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
username TEXT NOT NULL UNIQUE,
username TEXT COLLATE BINARY NOT NULL UNIQUE,
role TEXT NOT NULL DEFAULT 'USER',
email TEXT NOT NULL DEFAULT '',
nickname TEXT NOT NULL DEFAULT '',

View file

@ -135,8 +135,16 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {
);
CREATE TABLE user (
id INTEGER PRIMARY KEY AUTOINCREMENT,
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
username TEXT NOT NULL UNIQUE,
role TEXT NOT NULL DEFAULT 'USER'
role TEXT NOT NULL DEFAULT 'USER',
email TEXT NOT NULL DEFAULT '',
nickname TEXT NOT NULL DEFAULT '',
password_hash TEXT NOT NULL,
avatar_url TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT ''
);
CREATE TABLE user_setting (
user_id INTEGER NOT NULL,
@ -174,7 +182,10 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {
_, err = db.ExecContext(ctx, "INSERT INTO system_setting (name, value) VALUES ('BASIC', ?), ('TAGS', ?)", string(basicSettingBytes), string(tagsSettingBytes))
require.NoError(t, err)
_, err = db.ExecContext(ctx, "INSERT INTO user (id, username, role) VALUES (1, 'tag-owner', 'USER'), (2, 'keeps-existing', 'USER')")
_, err = db.ExecContext(ctx, `
INSERT INTO user (id, username, role, password_hash, avatar_url)
VALUES (1, 'tag-owner', 'USER', 'legacy-hash', ''), (2, 'keeps-existing', 'USER', 'legacy-hash', '')
`)
require.NoError(t, err)
_, err = db.ExecContext(ctx, "INSERT INTO user_setting (user_id, key, value) VALUES (2, 'TAGS', ?)", string(existingUserTagsBytes))
require.NoError(t, err)
@ -208,6 +219,133 @@ func TestMigrationCopiesInstanceTagsToUserSettings(t *testing.T) {
require.NotContains(t, existing.GetTags().GetTags(), "bug")
}
func TestCaseSensitiveUsernameMigration(t *testing.T) {
ctx := context.Background()
driver := getDriverFromEnv()
var dsn string
switch driver {
case "sqlite":
dsn = fmt.Sprintf("%s/memos_username_migration.db", t.TempDir())
case "mysql":
dsn = GetMySQLDSN(t)
case "postgres":
dsn = GetPostgresDSN(t)
default:
t.Fatalf("unsupported driver: %s", driver)
}
db, err := sql.Open(driver, dsn)
require.NoError(t, err)
_, err = db.ExecContext(ctx, caseSensitiveUsernameMigrationFixture(driver))
require.NoError(t, err)
basicSettingBytes, err := protojson.Marshal(&storepb.InstanceBasicSetting{SchemaVersion: "0.30.1"})
require.NoError(t, err)
insertBasicSetting := "INSERT INTO system_setting (name, value, description) VALUES ('BASIC', ?, '')"
if driver == "postgres" {
insertBasicSetting = "INSERT INTO system_setting (name, value, description) VALUES ('BASIC', $1, '')"
}
_, err = db.ExecContext(ctx, insertBasicSetting, string(basicSettingBytes))
require.NoError(t, err)
insertUser := "INSERT INTO user (username, role, password_hash, avatar_url) VALUES ('Alice', 'USER', 'legacy-hash', '')"
if driver == "mysql" {
insertUser = "INSERT INTO `user` (username, role, password_hash, avatar_url) VALUES ('Alice', 'USER', 'legacy-hash', '')"
} else if driver == "postgres" {
insertUser = `INSERT INTO "user" (username, role, password_hash, avatar_url) VALUES ('Alice', 'USER', 'legacy-hash', '')`
}
_, err = db.ExecContext(ctx, insertUser)
require.NoError(t, err)
require.NoError(t, db.Close())
ts := NewTestingStoreWithDSN(ctx, t, driver, dsn)
require.NoError(t, ts.Migrate(ctx))
defer ts.Close()
lower, err := createTestingUserWithRole(ctx, ts, "alice", store.RoleUser)
require.NoError(t, err)
upperUsername := "Alice"
upper, err := ts.GetUser(ctx, &store.FindUser{Username: &upperUsername})
require.NoError(t, err)
require.NotEqual(t, upper.ID, lower.ID)
require.Equal(t, "Alice", upper.Username)
}
func caseSensitiveUsernameMigrationFixture(driver string) string {
switch driver {
case "mysql":
return `
CREATE TABLE system_setting (
name VARCHAR(256) NOT NULL PRIMARY KEY,
value LONGTEXT NOT NULL,
description TEXT NOT NULL
);
CREATE TABLE ` + "`user`" + ` (
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
created_ts TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_ts TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
row_status VARCHAR(256) NOT NULL DEFAULT 'NORMAL',
username VARCHAR(256) NOT NULL UNIQUE,
role VARCHAR(256) NOT NULL DEFAULT 'USER',
email VARCHAR(256) NOT NULL DEFAULT '',
nickname VARCHAR(256) NOT NULL DEFAULT '',
password_hash VARCHAR(256) NOT NULL,
avatar_url LONGTEXT NOT NULL,
description VARCHAR(256) NOT NULL DEFAULT ''
);
CREATE TABLE memo (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY);
`
case "postgres":
return `
CREATE TABLE system_setting (
name TEXT NOT NULL PRIMARY KEY,
value TEXT NOT NULL,
description TEXT NOT NULL
);
CREATE TABLE "user" (
id SERIAL PRIMARY KEY,
created_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
updated_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW()),
row_status TEXT NOT NULL DEFAULT 'NORMAL',
username TEXT NOT NULL UNIQUE,
role TEXT NOT NULL DEFAULT 'USER',
email TEXT NOT NULL DEFAULT '',
nickname TEXT NOT NULL DEFAULT '',
password_hash TEXT NOT NULL,
avatar_url TEXT NOT NULL,
description TEXT NOT NULL DEFAULT ''
);
CREATE TABLE memo (id SERIAL PRIMARY KEY);
`
case "sqlite":
return `
CREATE TABLE system_setting (
name TEXT NOT NULL,
value TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
UNIQUE(name)
);
CREATE TABLE user (
id INTEGER PRIMARY KEY AUTOINCREMENT,
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
row_status TEXT NOT NULL CHECK (row_status IN ('NORMAL', 'ARCHIVED')) DEFAULT 'NORMAL',
username TEXT NOT NULL UNIQUE,
role TEXT NOT NULL DEFAULT 'USER',
email TEXT NOT NULL DEFAULT '',
nickname TEXT NOT NULL DEFAULT '',
password_hash TEXT NOT NULL,
avatar_url TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT ''
);
CREATE TABLE memo (id INTEGER PRIMARY KEY AUTOINCREMENT);
`
default:
return ""
}
}
// TestMigrationFromStableVersion verifies that upgrading from a stable Memos version
// to the current version works correctly. This is the critical upgrade path test.
//

View file

@ -117,6 +117,33 @@ func TestUserGetByUsername(t *testing.T) {
ts.Close()
}
func TestUsernameEqualityIsCaseSensitive(t *testing.T) {
t.Parallel()
ctx := context.Background()
ts := NewTestingStore(ctx, t)
defer ts.Close()
upper, err := createTestingUserWithRole(ctx, ts, "Alice", store.RoleUser)
require.NoError(t, err)
lower, err := createTestingUserWithRole(ctx, ts, "alice", store.RoleUser)
require.NoError(t, err)
require.NotEqual(t, upper.ID, lower.ID)
for username, wantID := range map[string]int32{
"Alice": upper.ID,
"alice": lower.ID,
} {
found, err := ts.GetUser(ctx, &store.FindUser{Username: &username})
require.NoError(t, err)
require.Equal(t, wantID, found.ID)
listed, err := ts.ListUsers(ctx, &store.FindUser{UsernameList: []string{username}})
require.NoError(t, err)
require.Len(t, listed, 1)
require.Equal(t, wantID, listed[0].ID)
}
}
func TestUserListByRole(t *testing.T) {
t.Parallel()
ctx := context.Background()

View file

@ -10,10 +10,9 @@ import { isMentionElement, isTagElement, isTaskListItemElement } from "@/types/m
import { lazyWithReload } from "@/utils/lazy";
import { rehypeHeadingId } from "@/utils/rehype-plugins/rehype-heading-id";
import { remarkDisableSetext } from "@/utils/remark-plugins/remark-disable-setext";
import { remarkMention } from "@/utils/remark-plugins/remark-mention";
import { remarkPreserveType } from "@/utils/remark-plugins/remark-preserve-type";
import { remarkSplitMixedTaskLists } from "@/utils/remark-plugins/remark-split-mixed-task-lists";
import { remarkTag } from "@/utils/remark-plugins/remark-tag";
import { remarkMemoSyntax } from "@/utils/remark-plugins/remark-tag";
import { CodeBlock } from "./CodeBlock";
import { SANITIZE_SCHEMA } from "./constants";
import { MarkdownRenderContext, rootMarkdownRenderContext } from "./MarkdownRenderContext";
@ -58,7 +57,7 @@ function getMentionUsername(node: Element, children?: ReactNode): string {
const text = Array.isArray(children) ? children.join("") : children;
if (typeof text === "string" && text.startsWith("@")) {
return text.slice(1).toLowerCase();
return text.slice(1);
}
return "";
@ -165,8 +164,7 @@ export const MemoMarkdownRendererCore = ({
...mathRemarkPlugins,
remarkGfm,
remarkSplitMixedTaskLists,
remarkTag,
remarkMention,
remarkMemoSyntax,
remarkBreaks,
remarkPreserveType,
]}

View file

@ -3,7 +3,7 @@ import { createContext, type ReactNode, useContext, useMemo } from "react";
import { userDetailQueryOptions, useUsersByUsernames } from "@/hooks/useUserQueries";
import { extractUsernameFromName } from "@/lib/resource-names";
import type { User } from "@/types/proto/api/v1/user_service_pb";
import { extractMentionUsernames } from "@/utils/remark-plugins/remark-mention";
import { extractMentionUsernames } from "@/utils/mention-extraction";
interface UserResolutionContextValue {
mentionUsernamesByContent: ReadonlyMap<string, string[]>;

View file

@ -13,7 +13,9 @@ import {
} from "@/utils/markdown-link";
import { findDecodedMarkdownSourceBoundaries } from "@/utils/markdown-source-boundaries";
import { memoMarkdownExtensions } from "@/utils/memo-markdown-extension";
import { findMentionMatches, type MentionMatch } from "@/utils/mention-grammar";
import { findTagMatches, type TagMatch } from "@/utils/tag-grammar";
import { isUsernameCharacter } from "@/utils/username";
interface SourceRange {
from: number;
@ -290,11 +292,7 @@ function mergeSourceRanges(ranges: SourceRange[]): SourceRange[] {
return merged;
}
export function markdownGFMEmailSourceRanges(state: EditorState, to = state.doc.length): GFMEmailSourceRange[] {
return parsedMarkdownContext(state, to).emailRanges;
}
export function literalTagSourceRanges(state: EditorState, from: number, to: number): SourceRange[] {
export function literalMarkdownSourceRanges(state: EditorState, from: number, to: number): SourceRange[] {
const ranges: SourceRange[] = [];
const context = parsedMarkdownContext(state, to);
collectLiteralRanges(
@ -316,7 +314,7 @@ export function literalTagSourceRanges(state: EditorState, from: number, to: num
/** Find tags only in literal Markdown source exposed by the editor syntax tree. */
export function findMarkdownTagMatches(state: EditorState, from: number, to: number): TagMatch[] {
const matches: TagMatch[] = [];
for (const range of literalTagSourceRanges(state, from, to)) {
for (const range of literalMarkdownSourceRanges(state, from, to)) {
const source = state.doc.sliceString(range.from, range.to);
for (const match of findTagMatches(source)) {
matches.push({ ...match, from: range.from + match.from, to: range.from + match.to });
@ -325,6 +323,19 @@ export function findMarkdownTagMatches(state: EditorState, from: number, to: num
return matches;
}
/** Find username references only in literal Markdown source exposed by the editor syntax tree. */
export function findMarkdownMentionMatches(state: EditorState, from: number, to: number): MentionMatch[] {
const matches: MentionMatch[] = [];
for (const range of literalMarkdownSourceRanges(state, from, to)) {
const source = state.doc.sliceString(range.from, range.to);
const runHasLeftBoundary = range.from === 0 || !isUsernameCharacter(state.doc.sliceString(range.from - 1, range.from));
for (const match of findMentionMatches(source, runHasLeftBoundary)) {
matches.push({ ...match, from: range.from + match.from, to: range.from + match.to });
}
}
return matches;
}
/** Return the active tag whose recognized source span ends at the cursor. */
export function tagMatchBefore(state: EditorState, position: number): TagMatch | undefined {
const line = state.doc.lineAt(position);
@ -335,5 +346,5 @@ export function tagMatchBefore(state: EditorState, position: number): TagMatch |
/** Whether an offset is in a literal-source run (used for explicit `#` completion). */
export function isLiteralTagPosition(state: EditorState, position: number): boolean {
const line = state.doc.lineAt(position);
return literalTagSourceRanges(state, line.from, position).some((range) => range.to === position);
return literalMarkdownSourceRanges(state, line.from, position).some((range) => range.to === position);
}

View file

@ -1,10 +1,8 @@
import { RangeSetBuilder } from "@codemirror/state";
import { Decoration, type DecorationSet, EditorView } from "@codemirror/view";
import { MENTION_RUN } from "@/utils/mention-grammar";
import { findMarkdownTagMatches, markdownGFMEmailSourceRanges } from "./markdownTagRanges";
import { findMarkdownMentionMatches, findMarkdownTagMatches } from "./markdownTagRanges";
import { viewportDecorations } from "./viewportDecorations";
const MENTION_RE = new RegExp(`(^|[^A-Za-z0-9])@(${MENTION_RUN})`, "gu");
const tagMark = Decoration.mark({ class: "cm-memo-tag" });
const mentionMark = Decoration.mark({ class: "cm-memo-mention" });
@ -19,20 +17,13 @@ function build(view: EditorView): DecorationSet {
if (previous && range.from <= previous.to) previous.to = Math.max(previous.to, range.to);
else scanRanges.push({ ...range });
}
const emailRanges = markdownGFMEmailSourceRanges(view.state, scanRanges.at(-1)?.to ?? view.viewport.to);
for (const { from, to } of scanRanges) {
for (const match of findMarkdownTagMatches(view.state, from, to)) {
ranges.push({ from: match.from, to: match.to, deco: tagMark });
}
const text = view.state.doc.sliceString(from, to);
MENTION_RE.lastIndex = 0;
let match: RegExpExecArray | null;
while ((match = MENTION_RE.exec(text)) !== null) {
const start = from + match.index + match[1].length; // skip the boundary char
if (emailRanges.some((range) => range.from <= start && start < range.to)) continue;
ranges.push({ from: start, to: start + 1 + match[2].length, deco: mentionMark });
for (const match of findMarkdownMentionMatches(view.state, from, to)) {
ranges.push({ from: match.from, to: match.to, deco: mentionMark });
}
}
ranges.sort((a, b) => a.from - b.from || a.to - b.to);

View file

@ -293,7 +293,7 @@ export type CreateUserRequest = Message<"memos.api.v1.CreateUserRequest"> & {
/**
* Optional. The resource ID to use for this user. If set, it must equal
* user.username, follow the UID format, and not consist entirely of digits.
* user.username and follow the username format.
* Format: ^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$
*
* @generated from field: string user_id = 2;

View file

@ -0,0 +1 @@
export { extractMentionUsernames } from "@/utils/remark-plugins/remark-tag";

View file

@ -1,28 +1,32 @@
/**
* The single source of truth for memos' `@mention` lexing grammar, shared by the
* editor and the read-only renderer (utils/remark-plugins/remark-mention.ts) so
* they can't drift.
*
* A mention character is an ASCII letter, digit, or hyphen. A username is capped
* at MAX_MENTION_LENGTH characters and must contain at least one letter or digit
* (a run of only hyphens is not a mention). This matches the Go backend parser
* (internal/markdown/parser/mention.go).
*/
export const MENTION_CHAR_CLASS = "[A-Za-z0-9-]";
import { isUsernameCharacter, isValidUsername } from "@/utils/username";
export const MAX_MENTION_LENGTH = 63;
/**
* Regex source for a username run: 1..MAX_MENTION_LENGTH mention characters that
* include at least one letter or digit. An over-long run is left as plain text.
* The leading lookahead enforces the "must contain an alphanumeric" rule so
* `@---` is left as plain text. Embed after an `@`, e.g. `^@(${MENTION_RUN})`.
*/
export const MENTION_RUN = `(?=${MENTION_CHAR_CLASS}{0,${MAX_MENTION_LENGTH - 1}}[A-Za-z0-9])${MENTION_CHAR_CLASS}{1,${MAX_MENTION_LENGTH}}(?!${MENTION_CHAR_CLASS})`;
const SINGLE_MENTION_CHAR = /^[A-Za-z0-9-]$/;
/** Whether a single character is allowed inside a username. */
export function isMentionChar(char: string): boolean {
return SINGLE_MENTION_CHAR.test(char);
export interface MentionMatch {
from: number;
to: number;
username: string;
}
function hasLeftBoundary(source: string, position: number, runHasLeftBoundary: boolean): boolean {
if (position === 0) return runHasLeftBoundary;
return !isUsernameCharacter(source[position - 1]);
}
/** Find username references in one eligible literal Markdown source run. */
export function findMentionMatches(source: string, runHasLeftBoundary = true): MentionMatch[] {
const matches: MentionMatch[] = [];
for (let position = 0; position < source.length; ) {
if (source[position] !== "@" || !hasLeftBoundary(source, position, runHasLeftBoundary)) {
position++;
continue;
}
let end = position + 1;
while (end < source.length && isUsernameCharacter(source[end])) end++;
const username = source.slice(position + 1, end);
if (isValidUsername(username)) {
matches.push({ from: position, to: end, username });
}
position = end === position + 1 ? position + 1 : end;
}
return matches;
}

View file

@ -1,155 +0,0 @@
import type { Root, Text } from "mdast";
import type { Node as UnistNode } from "unist";
import { visit } from "unist-util-visit";
import type { MentionNode, MentionNodeData } from "@/types/markdown";
import { isMentionChar, MAX_MENTION_LENGTH } from "@/utils/mention-grammar";
function isMentionBoundary(char: string): boolean {
if (!char) return true;
return !isMentionChar(char);
}
type Segment = { type: "text"; value: string } | { type: "mention"; value: string };
export function parseMentionsFromText(text: string, leadingChar = ""): Segment[] {
const segments: Segment[] = [];
const chars = [...text];
let i = 0;
const appendText = (value: string) => {
const previous = segments.at(-1);
if (previous?.type === "text") {
previous.value += value;
} else {
segments.push({ type: "text", value });
}
};
while (i < chars.length) {
const prevChar = i > 0 ? chars[i - 1] : leadingChar;
if (chars[i] === "@" && isMentionBoundary(prevChar) && i + 1 < chars.length && isMentionChar(chars[i + 1])) {
let j = i + 1;
while (j < chars.length && isMentionChar(chars[j]) && j - i - 1 < MAX_MENTION_LENGTH) {
j++;
}
const username = chars.slice(i + 1, j).join("");
const isOverlong = j < chars.length && isMentionChar(chars[j]);
const hasLetterOrNumber = [...username].some((char) => /[A-Za-z0-9]/.test(char));
if (username && !isOverlong && hasLetterOrNumber) {
segments.push({ type: "mention", value: username.toLowerCase() });
i = j;
continue;
}
}
let j = i + 1;
while (j < chars.length && chars[j] !== "@") {
j++;
}
appendText(chars.slice(i, j).join(""));
i = j;
}
return segments;
}
export function extractMentionUsernames(text: string): string[] {
const usernames = parseMentionsFromText(text)
.filter((segment): segment is { type: "mention"; value: string } => segment.type === "mention")
.map((segment) => segment.value);
return Array.from(new Set(usernames));
}
function createMentionNode(username: string): MentionNode {
const data: MentionNodeData = {
hName: "span",
hProperties: {
className: "mention",
"data-mention": username,
},
hChildren: [{ type: "text", value: `@${username}` }],
};
return {
type: "mentionNode",
value: username,
data,
} as MentionNode;
}
function trailingText(node: UnistNode | undefined): string {
if (!node) return "";
if (node.type === "text") return (node as Text).value;
const nodeChildren = (node as { children?: UnistNode[] }).children;
if (nodeChildren?.length) return trailingText(nodeChildren.at(-1));
const children = (node.data as { hChildren?: Array<{ type?: string; value?: string }> } | undefined)?.hChildren;
const last = children?.at(-1);
return last?.type === "text" ? (last.value ?? "") : "";
}
function previousSourceCharacter(source: string, offset: number): string {
return [...source.slice(Math.max(0, offset - 2), offset)].at(-1) ?? "";
}
function synthesizedLiteralTrailingText(node: UnistNode | undefined, source: string): string {
if (!node) return "";
if (node.type === "tagNode" || node.type === "mentionNode") return trailingText(node);
if (node.type !== "link" || !(node as { url?: string }).url?.startsWith("mailto:")) return "";
const rendered = trailingText(node);
const from = node.position?.start.offset;
const to = node.position?.end.offset;
if (from !== undefined && to !== undefined && source.slice(from, to) !== rendered) {
// Explicit links and autolinks end in Markdown syntax (`)` or `>`), which
// is the mention boundary. Only a literal GFM email contributes its final
// rendered character across an mdast child boundary.
return "";
}
return rendered;
}
function leadingCharacter(parent: UnistNode, index: number, node: Text, source: string): string {
const previous = (parent as { children?: UnistNode[] }).children?.[index - 1];
if (!previous) return "";
const currentFrom = node.position?.start.offset;
const previousTo = previous.position?.end.offset;
if (currentFrom !== undefined && previousTo === currentFrom) {
// Use the actual source boundary instead of text rendered by an arbitrary
// preceding node. Formatting and link delimiters are meaningful mention
// boundaries even though they are not present in rendered text.
return previousSourceCharacter(source, currentFrom);
}
return [...synthesizedLiteralTrailingText(previous, source)].at(-1) ?? "";
}
type VFileLike = { value?: string | Uint8Array };
export const remarkMention = () => {
return (tree: Root, file: VFileLike) => {
const source = typeof file.value === "string" ? file.value : "";
visit(tree, (node, index, parent) => {
if (node.type !== "text" || !parent || parent.type === "link" || typeof index !== "number") return;
const textNode = node as Text;
const segments = parseMentionsFromText(textNode.value, leadingCharacter(parent, index, textNode, source));
if (segments.every((segment) => segment.type === "text")) {
return;
}
const newNodes = segments.map((segment) => {
if (segment.type === "mention") {
return createMentionNode(segment.value);
}
return {
type: "text",
value: segment.value,
} as Text;
});
(parent.children as UnistNode[]).splice(index, 1, ...(newNodes as UnistNode[]));
});
};
};

View file

@ -1,16 +1,24 @@
import { parser as markdownParser } from "@lezer/markdown";
import type { Link, Root, Text } from "mdast";
import { fromMarkdown } from "mdast-util-from-markdown";
import { gfmFromMarkdown } from "mdast-util-gfm";
import { gfm } from "micromark-extension-gfm";
import { decodeString } from "micromark-util-decode-string";
import type { Node as UnistNode } from "unist";
import type { TagNode, TagNodeData } from "@/types/markdown";
import type { MentionNode, MentionNodeData, TagNode, TagNodeData } from "@/types/markdown";
import { findMarkdownGFMEmailRanges, type GFMEmailSourceRange, type MarkdownSourceNode } from "@/utils/gfm-email";
import { findMarkdownGFMURLRanges } from "@/utils/gfm-url";
import { hasExactMarkdownRange, resolvedMarkdownLinkRanges } from "@/utils/markdown-link";
import { decodedMarkdownCharacterReferenceAt, findDecodedMarkdownSourceBoundaries } from "@/utils/markdown-source-boundaries";
import { memoMarkdownExtensions } from "@/utils/memo-markdown-extension";
import { findMentionMatches } from "@/utils/mention-grammar";
import { findTagMatches } from "@/utils/tag-grammar";
import { isUsernameCharacter } from "@/utils/username";
type Segment = { type: "text"; value: string } | { type: "tag"; source: string; value: string };
type Segment =
| { type: "text"; value: string }
| { type: "tag"; source: string; value: string }
| { type: "mention"; source: string; value: string };
interface SourceRange {
from: number;
@ -336,22 +344,33 @@ function segmentsForTextNode(
const runs = literalRunsForTextNode(value, span, source, boundaries);
if (!runs) return undefined;
const tags = runs.flatMap((run) =>
findTagMatches(run.source).map((match) => ({
from: run.valueFrom + match.from,
to: run.valueFrom + match.to,
source: run.source.slice(match.from, match.to),
value: match.value,
})),
);
if (tags.length === 0) return [{ type: "text", value }];
const matches = runs
.flatMap((run) => [
...findTagMatches(run.source).map((match) => ({
type: "tag" as const,
from: run.valueFrom + match.from,
to: run.valueFrom + match.to,
source: run.source.slice(match.from, match.to),
value: match.value,
})),
...findMentionMatches(run.source, run.sourceFrom === 0 || !isUsernameCharacter(source[run.sourceFrom - 1])).map((match) => ({
type: "mention" as const,
from: run.valueFrom + match.from,
to: run.valueFrom + match.to,
source: run.source.slice(match.from, match.to),
value: match.username,
})),
])
.sort((left, right) => left.from - right.from || left.to - right.to);
if (matches.length === 0) return [{ type: "text", value }];
const segments: Segment[] = [];
let cursor = 0;
for (const tag of tags) {
if (cursor < tag.from) segments.push({ type: "text", value: value.slice(cursor, tag.from) });
segments.push({ type: "tag", source: tag.source, value: tag.value });
cursor = tag.to;
for (const match of matches) {
if (match.from < cursor) continue;
if (cursor < match.from) segments.push({ type: "text", value: value.slice(cursor, match.from) });
segments.push({ type: match.type, source: match.source, value: match.value });
cursor = match.to;
}
if (cursor < value.length) segments.push({ type: "text", value: value.slice(cursor) });
return segments;
@ -374,6 +393,23 @@ function createTagNode(tagValue: string, source: string): TagNode {
} as TagNode;
}
function createMentionNode(username: string, source: string): MentionNode {
const data: MentionNodeData = {
hName: "span",
hProperties: {
className: "mention",
"data-mention": username,
},
hChildren: [{ type: "text", value: source }],
};
return {
type: "mentionNode",
value: username,
data,
} as MentionNode;
}
type ParentNode = UnistNode & { children: UnistNode[] };
function isParentNode(node: UnistNode): node is ParentNode {
@ -680,7 +716,7 @@ function reconcileTextNodes(
reconcileUnderscoreEmailEmphasis(parent, source, context, recoveredSpans);
}
function transformTagTextNodes(
function transformMemoTextNodes(
parent: ParentNode,
source: string,
context: MarkdownSourceContext,
@ -694,29 +730,52 @@ function transformTagTextNodes(
const segments = segmentsForTextNode(textNode.value, nodeSpan(textNode, recoveredSpans), source, context.boundaries);
if (!segments || segments.every((segment) => segment.type === "text")) continue;
const newNodes = segments.map((segment) =>
segment.type === "tag" ? createTagNode(segment.value, segment.source) : ({ type: "text", value: segment.value } as Text),
);
const newNodes = segments.map((segment) => {
if (segment.type === "tag") return createTagNode(segment.value, segment.source);
if (segment.type === "mention") return createMentionNode(segment.value, segment.source);
return { type: "text", value: segment.value } as Text;
});
parent.children.splice(index, 1, ...(newNodes as UnistNode[]));
index += newNodes.length - 1;
continue;
}
if (isParentNode(child) && TRANSPARENT_PARENT_TYPES.has(child.type)) {
transformTagTextNodes(child, source, context, recoveredSpans);
transformMemoTextNodes(child, source, context, recoveredSpans);
}
}
}
type VFileLike = { value?: string | Uint8Array };
export const remarkTag = () => {
function transformMemoSyntax(tree: Root, rawSource: string): void {
const source = rawSource.startsWith("\uFEFF") ? rawSource.slice(1) : rawSource;
const context = markdownSourceContext(source);
const recoveredSpans = new WeakMap<UnistNode, SourceSpan>();
reconcileTextNodes(tree as ParentNode, source, context, recoveredSpans);
transformMemoTextNodes(tree as ParentNode, source, context, recoveredSpans);
}
/** Transform Memos inline source syntaxes after GFM and math parsing. */
export const remarkMemoSyntax = () => {
return (tree: Root, file: VFileLike) => {
const rawSource = typeof file.value === "string" ? file.value : "";
const source = rawSource.startsWith("\uFEFF") ? rawSource.slice(1) : rawSource;
const context = markdownSourceContext(source);
const recoveredSpans = new WeakMap<UnistNode, SourceSpan>();
reconcileTextNodes(tree as ParentNode, source, context, recoveredSpans);
transformTagTextNodes(tree as ParentNode, source, context, recoveredSpans);
transformMemoSyntax(tree, typeof file.value === "string" ? file.value : "");
};
};
/** Extract exact mention candidates with the same Markdown rules used for rendering. */
export function extractMentionUsernames(source: string): string[] {
const tree = fromMarkdown(source, {
extensions: [gfm()],
mdastExtensions: [gfmFromMarkdown()],
});
transformMemoSyntax(tree, source);
const usernames: string[] = [];
const collect = (node: UnistNode): void => {
if (node.type === "mentionNode") usernames.push((node as MentionNode).value);
if (isParentNode(node)) node.children.forEach(collect);
};
collect(tree);
return Array.from(new Set(usernames));
}

20
web/src/utils/username.ts Normal file
View file

@ -0,0 +1,20 @@
/** Maximum number of ASCII characters in a writable username. */
export const MAX_USERNAME_LENGTH = 36;
const ASCII_ALPHANUMERIC = /^[A-Za-z0-9]$/;
/** Whether a character may occur anywhere in a writable username. */
export function isUsernameCharacter(char: string): boolean {
return ASCII_ALPHANUMERIC.test(char) || char === "-";
}
/** Whether a value satisfies the writable username format. */
export function isValidUsername(username: string): boolean {
if (username.length === 0 || username.length > MAX_USERNAME_LENGTH) return false;
if (!ASCII_ALPHANUMERIC.test(username[0]) || !ASCII_ALPHANUMERIC.test(username.at(-1) ?? "")) return false;
for (const char of username) {
if (!isUsernameCharacter(char)) return false;
}
return true;
}

View file

@ -3,7 +3,7 @@ import { forceParsing } from "@codemirror/language";
import { EditorState } from "@codemirror/state";
import { EditorView } from "@codemirror/view";
import { describe, expect, it } from "vitest";
import { findMarkdownTagMatches } from "@/components/MemoEditor/Editor/markdownTagRanges";
import { findMarkdownMentionMatches, findMarkdownTagMatches } from "@/components/MemoEditor/Editor/markdownTagRanges";
import { tagMentionDecorations } from "@/components/MemoEditor/Editor/tagMentionDecorations";
import { memoMarkdownExtensions } from "@/utils/memo-markdown-extension";
@ -214,5 +214,22 @@ describe("tag/mention decorations", () => {
it("allows flow math to interrupt a paragraph", () => expect(countClass("text\n$$\n#math", "cm-memo-tag")).toBe(0));
it("does not let same-line math hide following text", () => expect(countClass("$$#inline$$\n#ok", "cm-memo-tag")).toBe(1));
it("allows ordinary text between inline HTML tags", () => expect(countClass("<span>#inside</span>", "cm-memo-tag")).toBe(1));
it("decorates @mentions", () => expect(countClass("hi @alice", "cm-memo-mention")).toBe(1));
it("decorates writable username references", () =>
expect(countClass("@alice @Alice-2 @1alice @a--b @123 @123-456", "cm-memo-mention")).toBe(6));
it("rejects invalid username shapes", () => expect(countClass("@-alice @alice- @álîçé", "cm-memo-mention")).toBe(0));
it("applies mention boundaries to source text", () =>
expect(countClass("hello@alice foo-@bob foo_@carol 中文@dave (@erin) @frank@grace", "cm-memo-mention")).toBe(4));
it("decorates mentions in transparent formatting", () => expect(countClass("**@Alice** ~~@bob~~ _@carol_", "cm-memo-mention")).toBe(3));
it("keeps mentions in opaque Markdown source undecorated", () =>
expect(
countClass("`@code` [@link](/x) ![@image](/x) https://example.com/@url $@math$ \\@escaped &#64;entity @ok", "cm-memo-mention"),
).toBe(1));
it("keeps GFM emails opaque and does not cross their source boundary", () =>
expect(countClass("@alice@example.com foo@bar.com@bob @ok", "cm-memo-mention")).toBe(1));
it("returns the exact mention source range and username", () => {
const source = "x @Alice-2.";
const state = EditorState.create({ doc: source, extensions: [markdown({ extensions: memoMarkdownExtensions })] });
expect(findMarkdownMentionMatches(state, 0, source.length)).toEqual([{ from: 2, to: 10, username: "Alice-2" }]);
});
});

View file

@ -1,21 +1,56 @@
import { describe, expect, it } from "vitest";
import { parseMentionsFromText } from "@/utils/remark-plugins/remark-mention";
import { extractMentionUsernames } from "@/utils/mention-extraction";
import { findMentionMatches } from "@/utils/mention-grammar";
import { MAX_USERNAME_LENGTH } from "@/utils/username";
describe("parseMentionsFromText", () => {
it("accepts a 63-character username", () => {
const username = `a${"b".repeat(62)}`;
expect(parseMentionsFromText(`@${username}`)).toEqual([{ type: "mention", value: username }]);
});
it("leaves a 64-character username as text", () => {
const username = `a${"b".repeat(63)}`;
expect(parseMentionsFromText(`@${username}`)).toEqual([{ type: "text", value: `@${username}` }]);
});
it("keeps rejected adjacent mentions in the same literal text segment", () => {
expect(parseMentionsFromText("@example.com@alice", "_")).toEqual([
{ type: "mention", value: "example" },
{ type: "text", value: ".com@alice" },
describe("findMentionMatches", () => {
it("recognizes complete writable usernames without changing case", () => {
expect(findMentionMatches("@alice @Alice-2 @1alice @a--b @123 @123-456").map((match) => match.username)).toEqual([
"alice",
"Alice-2",
"1alice",
"a--b",
"123",
"123-456",
]);
});
it("accepts 36 characters and rejects the complete 37-character run", () => {
const maximum = `a${"b".repeat(MAX_USERNAME_LENGTH - 1)}`;
expect(findMentionMatches(`@${maximum}`).map((match) => match.username)).toEqual([maximum]);
expect(findMentionMatches(`@${maximum}c`)).toEqual([]);
});
it.each(["@-alice", "@alice-", "@álîçé"])('rejects the invalid username shape "%s"', (source) => {
expect(findMentionMatches(source)).toEqual([]);
});
it("uses username characters as the source boundary", () => {
expect(findMentionMatches("hello@alice foo-@bob foo_@carol 中文@dave (@erin)").map((match) => match.username)).toEqual([
"carol",
"dave",
"erin",
]);
expect(findMentionMatches("@alice", false)).toEqual([]);
});
it("treats other characters as ordinary right boundaries", () => {
expect(findMentionMatches("@alice_smith @bob@carol").map((match) => match.username)).toEqual(["alice", "bob"]);
});
});
describe("extractMentionUsernames", () => {
it("uses the same Markdown contexts and exact spelling as rendering", () => {
expect(
extractMentionUsernames("**@Alice** ~~@bob~~ `@code` [@link](/x) ![@image](/x) https://example.com/@url $@math$ @Alice"),
).toEqual(["Alice", "bob"]);
});
it("keeps escapes, character references, and GFM emails opaque", () => {
expect(extractMentionUsernames("\\@escaped &#64;entity @alice@example.com foo@bar.com@bob @ok")).toEqual(["ok"]);
});
it("keeps case-distinct username references distinct", () => {
expect(extractMentionUsernames("@Alice @alice @Alice")).toEqual(["Alice", "alice"]);
});
});

View file

@ -4,18 +4,15 @@ import remarkBreaks from "remark-breaks";
import remarkGfm from "remark-gfm";
import remarkMath from "remark-math";
import { describe, expect, it } from "vitest";
import { remarkMention } from "@/utils/remark-plugins/remark-mention";
import { remarkTag } from "@/utils/remark-plugins/remark-tag";
import { remarkMemoSyntax } from "@/utils/remark-plugins/remark-tag";
const renderMarkdown = (content: string): string =>
renderToStaticMarkup(
<ReactMarkdown remarkPlugins={[remarkMath, remarkGfm, remarkTag, remarkMention, remarkBreaks]}>{content}</ReactMarkdown>,
);
renderToStaticMarkup(<ReactMarkdown remarkPlugins={[remarkMath, remarkGfm, remarkMemoSyntax, remarkBreaks]}>{content}</ReactMarkdown>);
const renderMarkdownWithoutMath = (content: string): string =>
renderToStaticMarkup(<ReactMarkdown remarkPlugins={[remarkGfm, remarkTag, remarkMention, remarkBreaks]}>{content}</ReactMarkdown>);
renderToStaticMarkup(<ReactMarkdown remarkPlugins={[remarkGfm, remarkMemoSyntax, remarkBreaks]}>{content}</ReactMarkdown>);
describe("remarkTag", () => {
describe("remarkMemoSyntax", () => {
it("does not turn URL fragments inside autolinks into tags", () => {
const html = renderMarkdown("https://github.com/dmtrKovalenko/fff#pi-agent-extension\n\nProject #memo-tag");
@ -266,6 +263,28 @@ describe("remarkTag", () => {
expect(html).toContain('<em><a href="mailto:foo@example.com">foo@example.com</a> <span class="tag" data-tag="tag">#tag</span></em>');
});
it("renders only complete writable usernames and preserves their case", () => {
const maximum = `A${"b".repeat(35)}`;
const html = renderMarkdown(`@Alice-2 @1alice @123 @-alice @alice- @alice_smith @${maximum} @${maximum}c`);
expect(html).toContain('data-mention="Alice-2"');
expect(html).toContain('data-mention="1alice"');
expect(html).toContain(`data-mention="${maximum}"`);
expect(html).toContain('data-mention="123"');
expect(html).not.toContain('data-mention="-alice"');
expect(html).not.toContain('data-mention="alice-"');
expect(html).toContain('data-mention="alice"');
expect(html).not.toContain(`data-mention="${maximum}c"`);
});
it("uses Markdown structure for mention eligibility", () => {
const html = renderMarkdown("**@Alice** `@code` [@link](/x) https://example.com/@url $@math$ \\@escaped &#64;entity @ok");
expect(html).toContain('data-mention="Alice"');
expect(html).toContain('data-mention="ok"');
expect(html.match(/data-mention=/g)).toHaveLength(2);
});
it.each([
[
"__foo@example.com #tag__ @carol",
@ -331,7 +350,7 @@ describe("remarkTag", () => {
expect(strongHTML).toContain('data-mention="alice"');
expect(emphasisHTML).toContain('data-mention="alice"');
expect(tagHTML).not.toContain('data-mention="alice"');
expect(adjacentMentionsHTML.match(/data-mention=/g)).toHaveLength(1);
expect(adjacentMentionsHTML).toContain('data-mention="alice"');
expect(adjacentMentionsHTML).not.toContain('data-mention="bob"');
});

View file

@ -0,0 +1,30 @@
import { describe, expect, test } from "vitest";
import { isValidUsername, MAX_USERNAME_LENGTH } from "@/utils/username";
describe("isValidUsername", () => {
test.each([
["alice", true],
["Alice-2", true],
["1alice", true],
["a--b", true],
["a---b", true],
[`a${"b".repeat(MAX_USERNAME_LENGTH - 1)}`, true],
["", false],
["a".repeat(MAX_USERNAME_LENGTH + 1), false],
["123", true],
["123-456", true],
["00000000-0000-0000-0000-000000000000", true],
["-", false],
["---", false],
["-alice", false],
["alice-", false],
["alice_smith", false],
["alice@example.com", false],
[" alice ", false],
["alice smith", false],
["álîçé", false],
["张三", false],
])("validates %j", (username, expected) => {
expect(isValidUsername(username)).toBe(expected);
});
});