Switch to WMO Instruments

[sfinkens]

The Plan

This is the first of a series of PRs for switching to WMO instrument names (AVHRR/1) in user facing places, namely dataset attributes and scene.sensor_names. The internal representation (avhrr-1) remains the same.

Furthermore, we agreed to deprecate sensor: str|set[str] in favor of instruments: set[str]. For backwards compatibility, a config switch will be added to get the legacy sensor attribute back.

1. (This PR) Change Satpy internals to use sets for instruments and prepare for the transition to instruments.
2. Deprecate sensor in favor of instruments
   2.1 Enhancements: WMO Instruments Part 2.1: Enhancements sfinkens/satpy#2
   2.2 File Handlers: WMO Instruments Part 2.2: File Handlers sfinkens/satpy#3
3. Rename enhancement/composite YAML files so that they match the internal representation of the WMO sensor name.
   3.1 Rename files: WMO Instruments Part 3.1: Rename enhancement and composite YAML files sfinkens/satpy#5
   3.2 Rename sensor_name property to instruments: WMO Instruments Part 3.2: Cosmetics sfinkens/satpy#6
4. Rename sensor to instrument in all method names and docstrings.

I hope this way the changes are less painful to review. PRs 2-4 are relative to this PR, which will be merged into main eventually.

Changes in this PR

• Add satpy._instruments module containing
  • A StrEnum with all WMO instrument names. Members behave like strings, even in sets

    len({"ABI", OSCAR.ABI}) == 1

    Internally the enum values are used directly. Dataset attributes are still converted to plain strings because that's easier for users (no custom type, simpler string repr).
  • Helper methods for
    • Getting instruments from dataset attributes. Both sensor and instruments are supported, with a deprecation warning for sensor.
    • Setting instrument attributes
    • Converting instrument names WMO <-> internal
• For backwards compatibility, add sensor attribute in scene.__getitem__ if it doesn't exist. To be removed in v1.0.

Breaking Changes

None so far. For v1.0 we would have to remove compatibility methods and deprecation warnings wrapped in 8< v1.0. Then Satpy wouldn't support the sensor attribute anymore.

Notes

It would be nice to have a custom Attributes class with getters and setters for the instrument, but xarray converts it back to plain dicts internally.

• Closes #xxxx
• Tests added
• Fully documented
• Add your name to AUTHORS.md if not there already

[codecov]

Codecov Report

❌ Patch coverage is 97.09302% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.35%. Comparing base (1adb2fa) to head (8f92ae5).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
satpy/enhancements/enhancer.py	92.00%	2 Missing ⚠️
satpy/readers/core/yaml_reader.py	80.00%	2 Missing ⚠️
satpy/readers/ghrsst_l3c_sst.py	0.00%	2 Missing ⚠️
satpy/readers/maia.py	33.33%	2 Missing ⚠️
satpy/decision_tree.py	90.90%	1 Missing ⚠️
satpy/readers/mirs.py	88.88%	1 Missing ⚠️
satpy/readers/nucaps.py	75.00%	1 Missing ⚠️
satpy/readers/scmi.py	75.00%	1 Missing ⚠️
satpy/readers/seviri_l1b_icare.py	66.66%	1 Missing ⚠️
satpy/readers/tropomi_l2.py	66.66%	1 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3390      +/-   ##
==========================================
+ Coverage   96.33%   96.35%   +0.01%     
==========================================
  Files         467      470       +3     
  Lines       59138    59391     +253     
==========================================
+ Hits        56973    57226     +253     
  Misses       2165     2165              

Flag	Coverage Δ
behaviourtests	3.76% <22.86%> (+0.17%)	⬆️
unittests	96.44% <97.09%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sfinkens]

Would it make sense to move those instrument helper methods to a dedicated module _instrumenty.py?

[djhoese]

I added the "backwards-incompatibility" label on this PR. Let me know if that isn't accurate. You should rebase this PR on main which now includes some fixes for CI.

[djhoese]

Looks pretty good. I had a couple questions inline. I think my biggest long term fear after seeing these changes is handling the "sensor" in the enhancement decision tree.

[mraspaud]

Wow, that's a big one! But very clean, I like it!

My only concern is instument vs sensor, not in the naming, but rather that it could be confusing, but I understand that it makes things more distinct. So nothing to change there.

[sfinkens]

@martin You mean that there's both sensor and instrument in the code base? Eventually I would rename all sensor occurences to instrument.

Ok so I'm going to merge PRs sfinkens#2 and sfinkens#3 into this one and continue with renaming enhancement/composite files so that they match the WMO name.

[djhoese]

How will you handle backwards compatibility with old filenames? Or search for both?

[sfinkens]

My idea was to create a link from the old filename to the new filename

[djhoese]

But what will users do before they have time to transition?

[djhoese]

Transition their custom YAMLs I mean

[sfinkens]

Transition their custom YAMLs I mean

Hmm, good point. I didn't think of that. Then I'll have so search for both the old and the new name.