Writeup from Claude:
DXL re-import: why exported elements duplicate, and what the outline round-trip agent does differently
Background
Dmytro reported that exported agents are duplicated when imported back into an existing database. Our working theory is that our export/cleaning step strips out IDs that the importer needs to match existing elements.
Justin has a separate, older piece of code that exports and cleans Outline DXL and re-imports it without producing duplicates. This note summarizes what that code does differently and what it implies for our fix. (The reference code is not attached and should not be added to this repo.)
What the reference code is
It is a Domino server agent, not a standalone CLI like our importer. It is driven by two environment variables (a source ID and a target name) and is specialized for Outline design elements only — not a general design importer.
Its model is a round-trip:
- Export an Outline as DXL.
- DOM-parse it and store each outline entry as a regular Domino document (a "DXL Outline Form" in a "DXL Outlines" view), so entries can be edited as data.
- Regenerate DXL from those documents.
- Re-import.
A test harness exists to confirm the regenerated DXL matches the original.
The part relevant to the duplication bug
The key difference is how matching is configured on import, in processDXLImport:
setDesignImportOption(DXLIMPORTOPTION_REPLACE_ELSE_CREATE) — find an existing element and replace it; only create if none is found.setReplicaRequiredForReplaceOrUpdate(false) — do not require the target DB to be a replica of the source to perform a replace/update.setReplaceDbProperties(false) and setAclImportOption(DXLIMPORTOPTION_IGNORE).
DxlImporter importer = session.createDxlImporter();
importer.setReplaceDbProperties( false);
importer.setReplicaRequiredForReplaceOrUpdate(false);
importer.setAclImportOption( DxlImporter.DXLIMPORTOPTION_IGNORE);
importer.setDesignImportOption( DxlImporter.DXLIMPORTOPTION_REPLACE_ELSE_CREATE);
importer.importDxl( stream, dxl_target_db);
With REPLACE_ELSE_CREATE and the replica requirement turned off, the importer matches the existing design element by name/alias, not by UNID or replica identity. So it overwrites in place rather than appending a copy.
This matters because the reference code strips IDs far more aggressively than we do. The DXL it re-imports is hand-built and minimal: only semantic attributes (label, type, level, links, code/formula). There is no UNID, no noteid, no $-fields, and no DTD — it disables the output DOCTYPE on export and strips the <!DOCTYPE … domino_9_0.dtd> line before parsing.
In other words: it discards essentially all identity metadata, yet does not create duplicates, because matching falls back to the element name.
Implication for our case
Duplication is the classic symptom of the importer failing to find a match and falling through to "create." Once IDs were stripped, the likely culprits are:
- the importer still requires a replica/UNID match for replace, so the stripped IDs break the match; or
- it is effectively in create/import mode rather than replace mode; or
- the element name/alias in our cleaned DXL no longer matches the target.
The reference approach sidesteps all of this by relying on name-based REPLACE_ELSE_CREATE with the replica requirement disabled. This suggests our fix may live in the import options, not in restoring the stripped IDs.
Caveat if this becomes an option
Name-based replace works cleanly for outlines because their names are stable and unique. For agents, matching is also by name/alias, so it should translate — but agents with duplicate or computed names, or cases where we actually want to preserve the original UNID for replication identity, would behave differently than this outline-only round-trip assumes. Recommend gating any name-based replace behavior behind an explicit option.
Writeup from Claude:
DXL re-import: why exported elements duplicate, and what the outline round-trip agent does differently
Background
Dmytro reported that exported agents are duplicated when imported back into an existing database. Our working theory is that our export/cleaning step strips out IDs that the importer needs to match existing elements.
Justin has a separate, older piece of code that exports and cleans Outline DXL and re-imports it without producing duplicates. This note summarizes what that code does differently and what it implies for our fix. (The reference code is not attached and should not be added to this repo.)
What the reference code is
It is a Domino server agent, not a standalone CLI like our importer. It is driven by two environment variables (a source ID and a target name) and is specialized for Outline design elements only — not a general design importer.
Its model is a round-trip:
A test harness exists to confirm the regenerated DXL matches the original.
The part relevant to the duplication bug
The key difference is how matching is configured on import, in
processDXLImport:setDesignImportOption(DXLIMPORTOPTION_REPLACE_ELSE_CREATE)— find an existing element and replace it; only create if none is found.setReplicaRequiredForReplaceOrUpdate(false)— do not require the target DB to be a replica of the source to perform a replace/update.setReplaceDbProperties(false)andsetAclImportOption(DXLIMPORTOPTION_IGNORE).With
REPLACE_ELSE_CREATEand the replica requirement turned off, the importer matches the existing design element by name/alias, not by UNID or replica identity. So it overwrites in place rather than appending a copy.This matters because the reference code strips IDs far more aggressively than we do. The DXL it re-imports is hand-built and minimal: only semantic attributes (label, type, level, links, code/formula). There is no UNID, no noteid, no
$-fields, and no DTD — it disables the output DOCTYPE on export and strips the<!DOCTYPE … domino_9_0.dtd>line before parsing.In other words: it discards essentially all identity metadata, yet does not create duplicates, because matching falls back to the element name.
Implication for our case
Duplication is the classic symptom of the importer failing to find a match and falling through to "create." Once IDs were stripped, the likely culprits are:
The reference approach sidesteps all of this by relying on name-based
REPLACE_ELSE_CREATEwith the replica requirement disabled. This suggests our fix may live in the import options, not in restoring the stripped IDs.Caveat if this becomes an option
Name-based replace works cleanly for outlines because their names are stable and unique. For agents, matching is also by name/alias, so it should translate — but agents with duplicate or computed names, or cases where we actually want to preserve the original UNID for replication identity, would behave differently than this outline-only round-trip assumes. Recommend gating any name-based replace behavior behind an explicit option.