id: TEDY.17 template: tool-feature module: tedy state: Built traces-from: [] source-refs:

• reference-editor/app/src/app/integration/import (dashboard + all importers)
• reference-editor/app/src/app/integration/import/file-import/file-analysis.service.ts
• reference-editor/app/src/app/resources/_lib/code-system/services/code-system-file-import.service.ts
• reference-editor/app/src/app/resources/_lib/value-set/services/value-set-file-import.service.ts
• reference-editor/app/src/app/resources/_lib/map-set/services/map-set-file-import.service.ts
• reference-editor/app/src/app/integration/_lib/model/integration-import-configuration.ts
• reference-editor branch feat/import-config-portable-file (portable config block)
• terminology-server/fileimporter (analyze + code-system/value-set/map-set process controllers)
• terminology-server job-logs (async job + poll)
• https://tx.example/tutorial/import author: TEDY bootstrap created: 2026-08-09 updated: 2026-08-09

TEDY.17 – File & Source Import (Common Spec)

Feature grouping

Overview / common spec for the Import family. Children: TEDY.17.1 Code System file import, TEDY.17.2 Value Set file import, TEDY.17.3 Map Set file import, TEDY.17.4 External source imports (ICD-10 · Orphanet · LOINC). This spec owns the shared import lifecycle, transport contract, job model, navigation (Integration dashboard), privileges, and mapping-reuse machinery; children do not duplicate them.

Description

TEDY imports terminology resources — Code Systems, Value Sets, Map Sets — from uploaded files or source URLs, and imports a set of well-known external terminologies (WHO ICD-10, Orphanet rare diseases, LOINC). All importers share one backend surface under /file-importer/* (generic file import) plus dedicated per-source endpoints (/icd10/import, /orphanet/import, /loinc/import/*), and all report progress through the same asynchronous job log.

The importers live in a standalone Integration → Import dashboard (a left system menu with a routed panel per importer), and each resource list’s Add ▾ → Import action deep-links into the matching importer.

Import lifecycle (the shared spine)

Every file importer follows the same three steps; per-source importers use a subset (no analyze/mapping).

1. Analyze (synchronous) — POST {txapi}/file-importer/analyze, multipart/form-data with parts request (JSON { link, type }) and optional file. Returns { columns: [{ columnName, columnType, columnTypeFormat, hasValues }] }. The server parses the file/URL and detects columns; CSV/TSV/XLSX parsing is server-side — the client never parses the file.
2. Map (client only) — the user assigns each detected column to a target (per-resource; see children). Reusable via saved mappings, templates, and a portable config file (below). No network calls.
3. Process (asynchronous job) — POST {txapi}/file-importer/{resource}/process, multipart/form-data with parts request (JSON, the resource’s processing DTO) and optional file. Returns { jobId } immediately.
4. Poll — GET {txapi}/job-logs/{jobId} every 3 s until execution.status !== 'running' (client timeout window ~60 s, cancellable). Result is a JobLog { errors[], warnings[], successes[] }. On errors/warnings the importer surfaces them inline and offers a downloadable log; on a successful non-dry-run it shows a success toast linking to the created/updated resource.

Transport contract (must match exactly)

• Multipart parts are named request (a JSON string) and file (the blob, sent with filename files). The file part is omitted when importing from a URL (link/url / sourceUrl carries it and the server fetches).
• releaseDate is serialized to a date string before send.
• CS/VS controllers URL-decode the request part before JSON parsing; TEDY sends plain JSON (tolerated). Map Set uses the field name url (not link).
• Auth: every endpoint is privilege-guarded (see Privileges). The client attaches the ambient session/bearer to each request (TEDY dev uses the guest superuser).

Endpoints

(ATC /atc/import and UCUM /api/v1/ucum/essence/import exist in the reference editor and are out of scope for this family; see TEDY.17.4 notes.)

Source formats

• Code System: csv, tsv, xlsx, json. For XLSX the concept worksheet must be named concepts; CSV delimiter may be ; or ,; UTF-8; header on line 1.
• Value Set: csv, tsv (the reference DTO also lists json, fsh).
• Map Set: CSV only, against the downloadable template.

Navigation / Information architecture

• Route family under Integration → Import (e.g. /tedy/integration/import/...): code-system, value-set, map-set, and (TEDY.17.4) icd-10, orphanet, loinc. A dashboard shell renders a left system menu grouped by section (generic file importers · LOINC · ICD-10 · Orphanet) and a routed panel on the right, mirroring the reference editor’s integration dashboard.
• Each importer panel is a card with a breadcrumb header, the importer form, and an inline JobLog result area; global success/error via toast.
• Deep links: the existing per-resource Add ▾ → Import (CS/VS/MS lists) routes to the matching importer, prefilling the target resource when known.

Mapping-reuse machinery (CS importer; see TEDY.17.1)

All three are client-side and carry over from the reference editor:

• Saved mappings — per-Code-System-id column mappings in localStorage (key helex.tedy.code-system-file-import.mapping:{csId}), with save/load.
• Predefined templates — JSON presets under assets/file-import-templates/{id}.json (+ index.json), selectable from a dropdown; applied over the analyzed columns.
• Portable config file — a machine-readable JSON block delimited by ===TEDY-IMPORT-CONFIG=== / ===END-TEDY-IMPORT-CONFIG===, tedyImportConfig: 1, carrying { codeSystem, codeSystemVersion, sourceFormat, sourceType, rows[] }. It is prepended to the downloaded log so one file doubles as a reusable config; a Load config control reads it client-side (FileReader) and prefills target CS + version + source format + column mapping (mapping applied after the next analyze).

Privileges

• Analyze, CS process, VS process, MS CSV template, ICD-10, Orphanet, LOINC: CodeSystem.write (CS_WRITE).
• Map Set process: MapSet.read (MS_READ) in the reference backend (noted; TEDY follows the backend’s guard).
• Import entry points and the dashboard are privilege-gated; in TEDY dev the guest profile is a superuser (['*']), so all are visible.

Data layer (TEDY)

• libs/tedy/src/lib/client/fileImportClient.ts — analyzeFile, processCodeSystemImport, processValueSetImport, processMapSetImport, downloadMapSetTemplate.
• libs/tedy/src/lib/client/integrationImportClient.ts — importIcd10, importOrphanet, listLoincArchives, listLoincArchiveFiles, importLoincFromArchive, uploadBobObject.
• libs/tedy/src/lib/client/jobClient.ts + hooks/useImportJob.ts — the 3 s poll loop as a hook ({ running, log, error }, timeout + cancel).
• libs/tedy/src/lib/types/fileImport.ts, types/integrationImport.ts — DTOs modeled on the server request objects (richer than the reference frontend’s: CS carries space, spacePackage, autoConceptOrder, valueSetProperties, admin, endorser, etc.) so no capability is dropped.

Shared UI

• SourcePicker (link/file toggle + format), JobLogPanel (errors/warnings/ successes + download), ImportOptions (dryRun/clean/replace/generateValueSet), ColumnMappingTable (CS/VS). New pieces vs. existing TEDY: mapping table, JobLog viewer, MS scope editor (TEDY.17.3), LOINC slot mapper (TEDY.17.4). Reused from existing TEDY: resource searches, semantic-version select, publication-status select, multi-language inputs.

Testing

Baseline (what exists upstream)

• Reference editor (frontend): no import tests. The whole Angular app carries only 3 *.spec.ts files, none touching import; there is no test runner wired for the app. So there is no frontend suite to port and none to lose by omission.
• terminology-server (backend): well covered for CS. Spock tests under fileimporter/ — CodeSystemImportServiceTest, …FileImportServiceDryRunTest, …FileImportMapperTest, …FileImportProcessorTest, …FileImportLargeFileTest, and the Csv/Tsv/Xlsx FileParserTest parsers. No VS/MS file-import tests on the backend. TEDY relies on this suite for parsing, dry-run diffing, and mapping correctness — TEDY does not re-test server behavior.

TEDY strategy

Add targeted unit tests for the pure-logic seams the port introduces (the parts that were never covered upstream), plus light component smoke tests. Tooling: Vitest (jsdom, globals) + @testing-library/react, co-located *.test.ts(x). The pure-logic pieces live in @helex/tedy, which now carries a test-only vite.config.mts (jsdom/globals, src/test-setup.ts) mirroring @helex/ui. @helex/tedy is source-aliased, not a registered Nx project, so the invocation is npx vitest run from libs/tedy (not nx test). The lib’s test config deliberately carries no env/proxy config — prefixing + the /txapi proxy stay in the module’s vite.config.ts (TEDY.18). Each build phase’s acceptance includes new unit tests green + tsc clean.

Unit tests (high value — pure logic)

1. useImportJob polling — start → running → finished transition on the 3 s interval (fake timers); resolves the JobLog once execution.status !== 'running'; honors the ~60 s timeout window; cancel/unmount stops polling (no further job-logs calls); surfaces a job/HTTP error. Mock jobClient.
2. Column-mapping validation (CS) — at most one identifier; date/dateTime rows require propertyTypeFormat; duplicate concept-code / hierarchical-concept warns; only import-checked rows are considered.
3. Portable config — parseConfig extracts the delimited ===TEDY-IMPORT-CONFIG=== block, falls back to whole-file JSON, returns undefined on invalid; buildConfigBlock round-trips; applyMappingRows merges saved rows onto analyzed columns by columnName (unmatched columns untouched).
4. Request builders — the multipart shape: parts named exactly request (JSON) + file (filename files); file omitted in link mode; releaseDate serialized; properties[] includes only import rows; Map Set uses url (not link); DTOs drop undefineds.
5. Default derivation — property-name list = defaults (concept-code, hierarchical-concept, is-a) + defined + CS properties, weight-ordered; type defaulted from the chosen property.

Component tests (light — RTL smoke)

• CS importer: analyze mocked → mapping table renders one row per column; Process disabled until validation passes; JobLogPanel renders errors/warnings/ successes and the download affordance. Keep minimal — no real network, no CSV parsing (that’s server-side).

Out of scope for TEDY tests

Actual CSV/TSV/XLSX parsing and end-to-end job execution (backend suite); heavy widget internals; the external-source server fetch/parse (ICD-10/Orphanet/LOINC) — covered by manual verification against htx.helex.dev/lmb per child acceptance.

Open questions

• Predefined templates: seed set + where the assets/file-import-templates live in the TEDY build (module assets vs @helex/*).
• Real privileges once TEDY auth moves off the dev guest superuser (see docs/TODO.md auth item).