Adding methods, metrics and attacks¶

A component is written once, in Python, and everything else is derived from it. The class holds the implementation and a card: its name, what it does, how it works, what its parameters mean, where it comes from and what it needs. From that single declaration:

Consumer	What it takes from the class
Experiments, `taf-eval`, Python API	the registered class and its constructor parameters or dataclass fields
HTTP API (`/api/catalog/methods`, `/metrics`, `/attacks`)	the card, the parameters, the group and its name, in English and Polish
Research UI	catalogue pages, method page, experiment editor: nothing to edit in `web/`
Documentation (this site)	the method, attack and metric reference pages, generated by `python -m taf.catalogue_docs`
README	the short lists of methods, attacks and metrics, generated by the same command
Tests	contract tests run on every registered component without being told about it

Group names and their order (method families, metric categories, attack families) also come from the backend (taf.models.card), so a new family appears in the UI without a translation file to update.

At a glance¶

Component	Write	Register	Automatically tested by
Method	`src/taf/methods/MyMethod.py`: `SteganographyMethod` subclass with a `MethodCard`	a `MethodType` member (`models/types.py`) and a `BUILTIN_METHOD_CLASSES` entry (`methods/factory.py`)	`test_methods_roundtrip.py`, `test_catalogue.py`
Metric	`src/taf/metrics/<group>/MyMetric.py`: `Metric` subclass with a `MetricCard`	a `MetricType` member and a `BUILTIN_METRICS` entry (`metrics/factory.py`)	`test_metric_outputs.py`, `test_catalogue.py`
Attack	a frozen dataclass in the module of its phenomenon (`src/taf/attacks/noise.py`, `temporal.py`, ...) with an `AttackCard`	`ATTACK_CLASSES` (`attacks/registry.py`) and severity levels (`attacks/presets.py`)	`test_attack_contract.py`, `test_catalogue.py`

Then regenerate the documentation and run the tests:

python -m taf.catalogue_docs          # rewrites the generated sections of docs/ and README.md
python -m pytest tests/test_catalogue.py tests/test_attack_contract.py tests/test_methods_roundtrip.py

tests/test_catalogue.py fails while a card is missing or incomplete, while a description names a parameter the class does not have, and while the generated pages differ from the cards.

The card¶

Cards are defined in taf.models.card. Every text accepts either a plain English string or Text(en=..., pl=...); clients fall back to English when Polish is missing. Packaged components must provide both languages, and the tests check it.

Field	Kind	Meaning
`title`	all	Name for people: `"QIM / ST-DM"`, `"White Gaussian noise"`.
`summary`	all	One sentence: what the method hides, the attack models or the metric measures.
`details`	all	How it works, what the parameters do, how to read the results, and where the implementation departs from the publication.
`abbreviation`	all	Short label for figures and tables; defaults to the title. Must be unique within its kind.
`references`	all	`Reference(citation, year, doi=..., url=...)`; the first one is shown in tables.
`requires`, `extra`	all	Importable modules needed beyond the core installation and the extra that installs them (`requires=("torch",), extra="neural"`). The experiment preview warns when they are missing and the UI marks the component not installed.
`family`	method	One of the method families (`lsb`, `transform`, `spread_spectrum`, `echo`, `phase`, `quantization`, `statistical`, `adaptive`, `reversible`, `learned`, `neural`).
`purpose`	method	`steganography` (covert communication) or `watermarking` (robust marking).
`strength_parameter`	method	The constructor parameter that scales embedding strength; trade-off curves sweep it.
`needs_long_input`	method	The method needs several seconds of audio for a short message.
`scale`, `category`, `intrusive`, `domain`	metric	Unit or range; group (`speech_quality`, `speech_intelligibility`, `speech_reverberation`, `ai_based`); whether it compares with the original; `speech` or `audio`.

Attacks take their family from category (an AttackCategory) and their parameters from their fields; metrics take their direction from higher_is_better, components and component_directions. These stay on the class because the engine uses them, and the card never restates them.

Writing the texts¶

Describe what this code does. Name deviations from the paper explicitly ("uses the first layer only", "the paper's sync code is not reproduced"), and never report a paper's results as local ones.
Refer to parameters by their exact names (step_scale, snr_db). The tests check that every compound name in details is a real parameter, so a renamed argument cannot leave a stale description.
Keep imperceptibility, robustness, capacity and detectability apart: a small perturbation is not evidence of robustness, and a quality score is not evidence of security.
The summary is one sentence; the details are one paragraph of four to six sentences.

Adding a method¶

# src/taf/methods/MyEchoMethod.py
from typing import List

import numpy as np

from taf.models.SteganographyMethod import SteganographyMethod
from taf.models.card import MethodCard, Reference, Text
from taf.models.errors import CapacityError


class MyEchoMethod(SteganographyMethod):
    """One-line summary for readers of the code."""

    card = MethodCard(
        title=Text("My echo", "Moje echo"),
        abbreviation="MyEcho",
        family="echo",
        purpose="watermarking",
        strength_parameter="alpha",
        references=(Reference("Doe & Roe", 2024, doi="10.0000/example"),),
        summary=Text(
            en="Represents each bit by the delay of a faint echo.",
            pl="Reprezentuje każdy bit opóźnieniem słabego echa.",
        ),
        details=Text(
            en="Splits the cover into one frame per bit and adds a copy delayed by one of two delays, "
               "scaled by alpha relative to the frame RMS. ...",
            pl="Dzieli nośnik na ramki, po jednej na bit, i dodaje kopię opóźnioną o jedno z dwóch "
               "opóźnień, przeskalowaną przez alpha względem RMS ramki. ...",
        ),
    )

    def __init__(self, sr: int = 16000, alpha: float = 0.3, key: int = 1):
        self.sr = sr
        self.alpha = alpha
        self.key = key

    def encode(self, data: np.ndarray, message: List[int]) -> np.ndarray:
        audio = np.asarray(data, dtype=np.float64)
        if len(message) > len(audio) // 1024:
            raise CapacityError(f"message too long: {len(message)} > {len(audio) // 1024} bits")
        stego = audio.copy()            # never write into the caller's array
        ...
        return stego                    # same length as the cover

    def decode(self, data_with_watermark: np.ndarray, watermark_length: int) -> List[int]:
        ...                             # must work on a fresh instance: no state from encode()

    def type(self) -> str:
        return "My echo method"         # label stored in result rows

Register it:

# src/taf/models/types.py
class MethodType(Enum):
    ...
    MY_ECHO_METHOD = auto()

# src/taf/methods/factory.py
from taf.methods.MyEchoMethod import MyEchoMethod
BUILTIN_METHOD_CLASSES = {
    ...
    MethodType.MY_ECHO_METHOD: MyEchoMethod,
}

The method is now available as MY_ECHO_METHOD in configurations, with parameters written as MY_ECHO_METHOD:alpha=0.5. tests/test_methods_roundtrip.py holds it to the method contract on real speech: the message returns bit for bit through a fresh decoder, encode() neither modifies nor resizes the cover, an over-capacity message raises CapacityError, and synthetic audio does not crash it. Express strengths relative to a quantity the decoder can recompute (frame RMS, band energy); absolute constants break on quiet material and after any volume change. Import optional heavy dependencies inside the methods that use them and declare them in requires.

Adding a metric¶

# src/taf/metrics/speech_quality/MyDistanceMetric.py
import numpy as np

from taf.models.Metric import Metric
from taf.models.card import MetricCard, Reference, Text


class MyDistanceMetric(Metric):

    card = MetricCard(
        title="My spectral distance",
        abbreviation="MSD",
        category="speech_quality",
        scale="dB",
        references=(Reference("Doe", 2023, doi="10.0000/example"),),
        summary=Text(en="Measures ...", pl="Mierzy ..."),
        details=Text(en="Computes ... Lower is closer. ...", pl="Oblicza ... Mniej znaczy bliżej. ..."),
    )

    higher_is_better = False            # required: rankings never guess a direction from the name

    def calculate(self, samples_original, samples_processed, fs, frame_len=0.03, overlap=0.75):
        ...

    def name(self) -> str:
        return "My spectral distance (MSD)"   # label stored in result rows

A metric that returns several numbers declares components = ("a", "b") and, for entries that are not scores, component_directions; each component is then reported on its own rather than averaged. Register the class in MetricType and BUILTIN_METRICS. A missing optional dependency must raise, so the trial records a metric error instead of a plausible score.

Adding an attack¶

# src/taf/attacks/filtering.py (the module of the phenomenon it models)
@dataclass(frozen=True)
class TiltFilter(Attack):
    """First-order spectral tilt around a pivot frequency."""

    pivot_hz: float = 1000.0
    tilt_db: float = -6.0

    name = "tilt"
    category = AttackCategory.FILTERING
    card = AttackCard(
        title=Text("Spectral tilt", "Pochylenie widma"),
        summary=Text(en="Tilts the spectrum ...", pl="Pochyla widmo ..."),
        details=Text(en="... pivot_hz sets ... tilt_db ...", pl="... pivot_hz ustala ... tilt_db ..."),
    )

    def _process(self, audio, sample_rate):
        validate_cutoff(self.pivot_hz, sample_rate, "pivot_hz")   # never assume a rate
        processed = ...
        return processed, sample_rate, {"measured_tilt_db": ...}

Every field is a parameter recorded in the result row. Stochastic attacks take a seed field and draw only from np.random.default_rng(self.seed); experiments replace it per trial. Nothing may normalise its own effect away. Add the class to ATTACK_CLASSES in attacks/registry.py, and give it severity levels that resolve to explicit numbers, in severity_parameters (attacks/presets.py) or on the class:

    @classmethod
    def severity_levels(cls, severity, sample_rate):
        return {"tilt_db": {"mild": -1.5, "moderate": -3.0, "strong": -6.0, "extreme": -12.0}[severity.value]}

Rate-dependent values (cutoffs, target rates) must be derived from sample_rate. A robustness-curve ladder can be added to sweep_presets. tests/test_attack_contract.py then runs the attack on real speech with its defaults and at every severity, checks that the caller's array is untouched, that the output is finite and keeps its length unless the class sets changes_length_or_rate, that parameters reach the metadata, and that a seeded attack repeats exactly.

Plugins¶

Methods, metrics and attacks from other distributions are discovered through entry points, without modifying this package. A registered component appears in the catalogue, the UI and experiments under its entry-point name and is subject to the same method contract:

# pyproject.toml of the distribution that provides the components
[project.entry-points."taf.methods"]
MY_METHOD = "my_package.method:MyMethod"      # callable taking the sampling rate

[project.entry-points."taf.metrics"]
MY_METRIC = "my_package.metric:MyMetric"      # callable taking no arguments

[project.entry-points."taf.attacks"]
my_attack = "my_package.attack:MyAttack"      # Attack subclass

A plugin class declares its card exactly like a packaged one and is described in the UI in the same way. A plugin without a card is still listed: its title is its entry-point name and its summary the first paragraph of its class docstring. A plugin attack declares its severity levels with severity_levels. Plugins do not change the generated documentation pages, which describe the packaged components only.

Packaged names take precedence, so a published result that names a packaged method always refers to the packaged implementation (taf.plugins). An automated test enforces the package layering: the building blocks (methods, metrics, attacks, models, audio, steganalysis) never import the evaluation engine, the experiment layer or the HTTP layer. The engine in turn never imports the HTTP layer.

Interfaces¶

class SteganographyMethod(ABC):
    card: ClassVar[MethodCard | None] = None

    @abstractmethod
    def encode(self, data: np.ndarray, message: List[int]) -> np.ndarray: ...

    @abstractmethod
    def decode(self, data_with_watermark: np.ndarray, watermark_length: int) -> List[int]: ...

    @abstractmethod
    def type(self) -> str: ...


class Metric(ABC):
    card: ClassVar[MetricCard | None] = None
    higher_is_better: bool | None = None
    components: tuple[str, ...] = ()
    component_directions: tuple[bool | None, ...] = ()

    @abstractmethod
    def calculate(self, samples_original: np.ndarray, samples_processed: np.ndarray, fs: int,
                  frame_len: float = 0.03, overlap: float = 0.75) -> Number | np.ndarray: ...

    @abstractmethod
    def name(self) -> str: ...

Attacks subclass taf.attacks.base.Attack, a frozen dataclass whose fields are its parameters, and implement _process(audio, sample_rate) -> (audio, sample_rate, metadata); apply() handles dtype conversion, validation and the reproducibility metadata.