Skip to content

Commit 1646f71

Browse files
rustyconoverclaude
andcommitted
Make worker lint-clean at vgi-lint-check 0.26.0 (strict profile, 100/100)
Add the strict-profile metadata the 0.26.0 linter requires on every object, and make all examples execute cleanly against the worker (execute-on-by-default). - Add vgi_nlp/meta.py with object_tags()/source_url() helpers (mirrors the vgi-units reference): vgi.title (distinct from the machine name), narrative vgi.doc_llm and vgi.doc_md (distinct Markdown), vgi.keywords, vgi.source_url. - scalars.py: per-object tags on all 11 scalar functions; rewrite every example to be self-contained (string literals, no backend tables) so VGI901 examples bind/execute; switch the explicit-model examples to the installed en_core_web_sm. - tables.py: per-object tags on entities/tokens/sentences/noun_chunks; rename vgi.columns_md -> vgi.result_columns_md (VGI405); self-contained per-function examples; add vgi.executable_examples (VGI509/VGI906) with inline one-row input relations. - nlp_worker.py: rename vgi.description_llm/_md -> vgi.doc_llm/_md; add vgi.title + vgi.keywords on the catalog; add vgi.title/keywords/source_url/ example_queries plus bare-key domain/category/topic (VGI123) on the schema; use the LicenseRef- SPDX form for the source-available license (VGI013). Lint: 100/100, 0 findings at --fail-on info. pytest, ruff, mypy, pydoclint green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 3ac4a41 commit 1646f71

4 files changed

Lines changed: 537 additions & 36 deletions

File tree

nlp_worker.py

Lines changed: 41 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -70,8 +70,24 @@
7070
)
7171

7272
_SCHEMA_DESCRIPTION_MD = (
73-
"Classical NLP functions: language ID, sentiment, cleaning scalars, and "
74-
"entity/token/sentence/noun-chunk table functions."
73+
"# main\n\n"
74+
"Classical NLP functions exposed to SQL.\n\n"
75+
"- **Scalars:** language ID/confidence, sentiment score/label, lemmatize, "
76+
"strip stop-words, normalize.\n"
77+
"- **Table functions:** entities, tokens, sentences, noun_chunks "
78+
"(one text row in, N rows out, with an optional `id :=` passthrough)."
79+
)
80+
81+
# Representative, catalog-qualified example queries for the schema (VGI506).
82+
# All are self-contained so they bind/execute against the worker.
83+
_SCHEMA_EXAMPLE_QUERIES = (
84+
"SELECT nlp.main.detect_lang('Bonjour tout le monde');\n"
85+
"SELECT nlp.main.sentiment('I absolutely love this product!');\n"
86+
"SELECT nlp.main.sentiment_label('This was a terrible experience');\n"
87+
"SELECT nlp.main.lemmatize('The cats were running', 'en');\n"
88+
"SELECT nlp.main.normalize(' Café DELUXE ');\n"
89+
"SELECT * FROM nlp.main.entities("
90+
"(SELECT 1 AS id, 'Apple is based in California.' AS body), id := 'id', lang := 'en');"
7591
)
7692

7793
_NLP_CATALOG = Catalog(
@@ -80,11 +96,17 @@
8096
comment="Classical NLP (spaCy + fastText + VADER): language ID, sentiment, NER, tokenization for SQL.",
8197
source_url="https://github.com/Query-farm/vgi-nlp",
8298
tags={
83-
"vgi.description_llm": _CATALOG_DESCRIPTION_LLM,
84-
"vgi.description_md": _CATALOG_DESCRIPTION_MD,
99+
"vgi.title": "Classical NLP for SQL",
100+
"vgi.keywords": (
101+
"nlp, natural language processing, language detection, sentiment analysis, "
102+
"named entity recognition, ner, tokenization, lemmatize, stop words, "
103+
"noun chunks, spacy, fasttext, vader, text enrichment"
104+
),
105+
"vgi.doc_llm": _CATALOG_DESCRIPTION_LLM,
106+
"vgi.doc_md": _CATALOG_DESCRIPTION_MD,
85107
"vgi.author": "Query.Farm",
86108
"vgi.copyright": "Copyright 2026 Query Farm LLC - https://query.farm",
87-
"vgi.license": "Query Farm Source-Available License, Version 1.0",
109+
"vgi.license": "LicenseRef-QueryFarm-Source-Available-1.0",
88110
"vgi.support_contact": "https://github.com/Query-farm/vgi-nlp/issues",
89111
"vgi.support_policy_url": "https://github.com/Query-farm/vgi-nlp/blob/main/README.md",
90112
},
@@ -93,8 +115,20 @@
93115
name="main",
94116
comment="Classical NLP: language ID, sentiment, NER, tokenization for SQL",
95117
tags={
96-
"vgi.description_llm": _SCHEMA_DESCRIPTION_LLM,
97-
"vgi.description_md": _SCHEMA_DESCRIPTION_MD,
118+
"vgi.title": "NLP Functions (main)",
119+
"vgi.keywords": (
120+
"nlp, language detection, sentiment, ner, entities, tokens, "
121+
"sentences, noun chunks, lemmatize, strip stopwords, normalize, "
122+
"spacy, fasttext, vader"
123+
),
124+
# VGI123 classifying tags use BARE keys (not vgi.-namespaced).
125+
"domain": "text-analytics",
126+
"category": "natural-language-processing",
127+
"topic": "language-detection-sentiment-ner",
128+
"vgi.source_url": "https://github.com/Query-farm/vgi-nlp/blob/main/nlp_worker.py",
129+
"vgi.doc_llm": _SCHEMA_DESCRIPTION_LLM,
130+
"vgi.doc_md": _SCHEMA_DESCRIPTION_MD,
131+
"vgi.example_queries": _SCHEMA_EXAMPLE_QUERIES,
98132
},
99133
functions=[*SCALAR_FUNCTIONS, *TABLE_FUNCTIONS],
100134
),

vgi_nlp/meta.py

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
"""Shared helpers for the per-object discovery/description metadata that the
2+
``vgi-lint`` strict profile expects on **every** function and table.
3+
4+
Each function/table surfaces these in its ``Meta.tags``:
5+
6+
- ``vgi.title`` (VGI124) -- human-friendly display name (must differ from
7+
the machine name; add an extra word so VGI125 stays quiet)
8+
- ``vgi.doc_llm`` (VGI112) -- Markdown narrative aimed at LLMs/agents
9+
- ``vgi.doc_md`` (VGI113) -- Markdown narrative for human docs (distinct
10+
content from ``doc_llm``)
11+
- ``vgi.keywords`` (VGI126) -- comma-separated search terms/synonyms
12+
- ``vgi.source_url`` (VGI128) -- link to the implementing source file
13+
14+
``source_url(file)`` builds the canonical GitHub blob URL so every object points
15+
at exactly where it is implemented.
16+
""" # noqa: D205
17+
18+
from __future__ import annotations
19+
20+
#: Base GitHub blob URL for source files in this repo (pinned to ``main``).
21+
SOURCE_BASE = "https://github.com/Query-farm/vgi-nlp/blob/main"
22+
23+
24+
def source_url(relative_path: str) -> str:
25+
"""Build the implementation ``vgi.source_url`` for a repo-relative file.
26+
27+
e.g. ``source_url("vgi_nlp/scalars.py")``.
28+
"""
29+
return f"{SOURCE_BASE}/{relative_path}"
30+
31+
32+
def object_tags(
33+
title: str,
34+
doc_llm: str,
35+
doc_md: str,
36+
keywords: str,
37+
relative_path: str,
38+
) -> dict[str, str]:
39+
"""Build the five standard per-object discovery/description tags.
40+
41+
``relative_path`` is the implementing file relative to the repo root.
42+
"""
43+
return {
44+
"vgi.title": title,
45+
"vgi.doc_llm": doc_llm,
46+
"vgi.doc_md": doc_md,
47+
"vgi.keywords": keywords,
48+
"vgi.source_url": source_url(relative_path),
49+
}

0 commit comments

Comments
 (0)