AristaAvdDuplicateDataError message replaced by a TypeError under ansible-core 2.19+ (real conflict hidden)

[shoddynetwork]

On a build with a genuine duplicate in the data model, the useful Found duplicate objects with conflicting data… {A} conflicts with {B} message never reaches us — it's replaced by a constructor TypeError, so we can't see what actually conflicts. The cause looks like exception reconstruction in ansible-core 2.19+: AVD exceptions store only (message,) in Exception.args (via AristaAvdError.__init__), but AristaAvdDuplicateDataError.__init__(context, context_item_a, context_item_b, host=None) needs three positional args, so when core rebuilds the chained exception via type(exc)(*exc.args) it raises missing 2 required positional arguments. We only recovered the real conflict by temporarily patching the exception constructor to dump self.message to a file. Is this a known interaction, and would making AristaAvdError subclasses reconstructable (e.g. a base __reduce__) be the right fix? Happy to open an issue/PR if useful.

What we see:

[ERROR]: Task failed: AristaAvdDuplicateDataError.__init__() missing 2 required positional arguments: 'context_item_a' and 'context_item_b'

What was actually being reported (recovered via the patch):

Found duplicate objects with conflicting data while generating configuration for NetworkServices. {'name': 'Tenant_A', 'mac_vrf_vni_base': 40000, ...} conflicts with {'name': 'Tenant_A', 'mac_vrf_vni_base': 30000, ...}

Environment: pyavd 6.2.0 · arista.avd 6.2.0 · ansible-core 2.20.5 · Python 3.12

[gmuloc]

ugh thats not supposed to happen can you pinpoint the line with this?

We have not seen this one before and we have some extensive testing of negative issues (including duplicates) using ansible 2.19.

Can you share some repro inputs maybe? Though mac_vrf_vni_base conflicting should get me started I suppose.

thanks for reporting!

[gmuloc]

intersting, our tests do go through the ansible boundaries because they are run using molecule so we assert that we get the correct exception message from ansible in theory. Thanks for the repro I will investigate further about how we have been avoiding this!

[gmuloc]

Hmm I have been trying a bit and this is so far unreproducible from the ansible invocation of the role .
Can you give more info on how you have been using AVD?
In theory we suppress the exception chain towards ansible and only raise AnsibleFailAction so the exception does not need to be reconstructed.

Do you maybe have something specific in your playbook?

I still think we should investigate this further just want to make sure to understand how you hit this.

Thanks!

[shoddynetwork]

Ok, I pinned down the exact combination — and was able to reproduce using the public examples/campus-fabric out of the box, so you can clone and run it directly.

How to reproduce

In examples/campus-fabric, edit group_vars/DC1_NETWORK_SERVICES.yml to split the tenant across two network_services_keys with the same name but a differing tenant-level field (mac_vrf_vni_base):

---
network_services_keys:
  - name: tenants
  - name: tenants_b

tenants:
  - name: MY_FABRIC
    vrfs:
      - name: default
        # ... existing SVIs ...

tenants_b:
  - name: "MY_FABRIC" # same name, different key
    mac_vrf_vni_base: 40000
    mac_vrf_id_base: 0
    vrfs:
      - name: "inet"
        vrf_vni: 40000
        description: "inet"
        vtep_diagnostic:
          loopback: 40
          loopback_ip_pools:
            - pod: "INET"
              ipv4_pool: "10.4.0.0/24"

ansible-playbook build.yml then fails at Set eos_designs facts with the masked
AristaAvdDuplicateDataError.__init__() missing 2 required positional arguments: 'context_item_a' and 'context_item_b'.

The duplicate name has to span two different keys. If you instead put both MY_FABRIC entries in the single tenants: list, schema validation catches it cleanly at Validate eos_designs inputs (tenants[0].name ... not unique) — the masked path is only reached via the cross-key collision.

How I have mine

In production we don't keep tenants in one file. DC1_NETWORK_SERVICES is an inventory group the fabric nodes belong to:

DC1:
  children:
    DC1_NETWORK_SERVICES:
      children:
        DC1_LEAFS:
        DC1_SPINES:

…and instead of group_vars/DC1_NETWORK_SERVICES.yml we use a group_vars/DC1_NETWORK_SERVICES/ folder, so Ansible merges every .yml inside it. That gives each tenant its own file for easier management, stitched together by network_services_keys:

group_vars/
  DC1_NETWORK_SERVICES/
    _keys.yml          # network_services_keys: [{name: tenants}, {name: tenants_b}, ...]
    tenants.yml        # tenants:   [ - name: MY_FABRIC ... ]
    tenants_b.yml      # tenants_b: [ - name: MY_FABRIC ... ]   <- same name, different file/key

Whether the keys are split across files in a folder or written in one file is irrelevant to AVD — the per-tenant-file layout just naturally produces the cross-key duplicate that triggers this.

What we think is the problem

1. Cross-key duplicates bypass schema validation. Each key's list is validated for uniqueness independently, so a name duplicated across two keys slips past Validate eos_designs inputs. It only collides later, when filtered_tenants appends every key's tenants into one indexed list:

# shared_utils/filtered_tenants.py
for network_services_key in self.inputs._dynamic_keys.network_services:
    for original_tenant in network_services_key.value:
        filtered_tenants.append(tenant)   # append() RAISES on duplicate name — unguarded by schema validation

2. It's reached during facts (the run_once task). filtered_tenants is hit at facts via shared_utils/routing.py: bgp_as → bgp → bgp_in_network_services. In bgp:

self.uplink_type in ["p2p", "p2p-vrfs", "lan"]
and (
    self.underlay_routing_protocol == "ebgp"                                  # (A)
    or (overlay in ["ebgp","ibgp"] and evpn/mpls role in ["client","server"]) # (B)
    or self.bgp_in_network_services                                           # (C)
)

bgp_in_network_services (C) is only evaluated when (A) and (B) are both False. If your underlay is ebgp (A True), (C) short-circuits and the duplicate isn't caught until per-host structured config — where it surfaces normally as fatal: FAILED! => {"msg": "Found duplicate objects ..."} (real message, no mask). The campus-fabric l3spine design isn't ebgp-underlay, so (C) runs at facts → masked.

3. The exception can't survive serialization. Once it's raised in the run_once facts task, the result is serialized across hosts — and the exception object dies in a pickle/deepcopy round-trip, taking the real message with it:

import pickle, copy
from pyavd._errors import AristaAvdDuplicateDataError

e = AristaAvdDuplicateDataError("NetworkServices", "A", "B")
print(len(e.args))             # 1  -> only the rendered message
pickle.loads(pickle.dumps(e))  # TypeError: __init__() missing 2 required positional arguments
copy.deepcopy(e)               # same TypeError

BaseException.__reduce__ returns (cls, self.args), and AristaAvdError.__init__ stores only (message,) in args, while AristaAvdDuplicateDataError.__init__ needs three positional args — so reconstruction fails and the real message is replaced by the TypeError.

Happy to PR a fix (e.g. an AristaAvdError.__reduce__) + a pickle round-trip regression test.

[ClausHolbechArista]

We are still root-causing this. It is not as "simple" as this apparently. We will come back. Thank you for the analysis.

[gmuloc]

repro and root cause here: #7168 now we will discuss how to address this. thanks for the detailed info