diff --git a/docs/api_reference.rst b/docs/api_reference.rst index ff3baff7..412ffc99 100644 --- a/docs/api_reference.rst +++ b/docs/api_reference.rst @@ -37,7 +37,7 @@ Domain Specification (``dpsynth.domain``) The ``domain`` module provides dataclasses for describing the schema of a tabular dataset. Each column is represented by one of the attribute types below. Pass a mapping of column names to attribute objects as the ``domains`` -argument to :class:`~dpsynth.TabularSynthesizer`. +argument to :class:`~dpsynth.TabularConfig`. .. autosummary:: :toctree: _autosummary @@ -59,7 +59,7 @@ Cross-Attribute Constraints (``dpsynth.constraints``) The ``constraints`` module lets you express known relationships between columns so that the synthetic data honours them. Pass a list of :class:`~dpsynth.constraints.Constraint` objects as -``cross_attribute_constraints`` to :class:`~dpsynth.TabularSynthesizer`. +``cross_attribute_constraints`` to :class:`~dpsynth.TabularConfig`. .. autosummary:: :toctree: _autosummary @@ -88,13 +88,30 @@ protocol shared by all DPSynth mechanisms. ---- +Tabular Synthesis (``dpsynth``) +=============================== + +.. currentmodule:: dpsynth + +The primary entry point for generating differentially private synthetic data from standard tabular datasets (such as Pandas DataFrames). + +.. autosummary:: + :toctree: _autosummary + :nosignatures: + :template: autosummary/class.rst + + TabularConfig + TabularMechanism + +---- + Discrete Mechanisms (``dpsynth.discrete_mechanisms``) ====================================================== .. currentmodule:: dpsynth.discrete_mechanisms Discrete mechanisms operate on pre-discretized integer datasets -(:class:`mbi.Dataset`). :class:`TabularSynthesizer` applies them internally +(:class:`mbi.Dataset`). :class:`~dpsynth.TabularConfig` applies them internally after encoding your DataFrame. Use them directly only if you already have a discrete dataset. @@ -102,7 +119,7 @@ Mechanism Configs ----------------- Each config class corresponds to a published DP synthesis algorithm. Pass one -as the ``discrete_mechanism`` argument to :class:`~dpsynth.TabularSynthesizer`, +as the ``discrete_mechanism`` argument to :class:`~dpsynth.TabularConfig`, or use :class:`~dpsynth.discrete_mechanisms.DiscreteConfig` to add one-way marginal measurement and domain compression. @@ -118,12 +135,13 @@ marginal measurement and domain compression. SWIFTConfig AIMGDPConfig -DiscreteConfig --------------- +DiscreteConfig and DiscreteMechanism +------------------------------------ :class:`DiscreteConfig` wraps any of the mechanism configs above with one-way -marginal pre-measurement and optional domain compression. It is the recommended -entry point when you have a pre-discretized table. +marginal pre-measurement and optional domain compression. When calibrated, it +produces a runnable :class:`DiscreteMechanism`. This is the recommended entry +point when you have a pre-discretized table. .. autosummary:: :toctree: _autosummary @@ -131,3 +149,4 @@ entry point when you have a pre-discretized table. :template: autosummary/class.rst DiscreteConfig + DiscreteMechanism diff --git a/docs/in_memory_api.md b/docs/in_memory_api.md index e3140e7c..5a16d3e1 100644 --- a/docs/in_memory_api.md +++ b/docs/in_memory_api.md @@ -11,13 +11,13 @@ within a single machine's RAM. -------------------------------------------------------------------------------- -## Python API: `dpsynth.TabularSynthesizer` +## Python API: `dpsynth.TabularConfig` The primary entry point for in-memory synthesis is -`dpsynth.TabularSynthesizer`. It accepts a dictionary of attribute domains, -is calibrated with a privacy budget, and generates a fully synthetic, -differentially private DataFrame matching the exact schema and data types of -your input. +`dpsynth.TabularConfig`. It accepts a dictionary of attribute domains and +mechanism options, is calibrated with a privacy budget to produce a +`dpsynth.TabularMechanism`, and generates a fully synthetic, differentially +private DataFrame matching the exact schema and data types of your input. ### Usage @@ -27,43 +27,45 @@ from dpsynth import discrete_mechanisms import numpy as np import pandas as pd -synth = dpsynth.TabularSynthesizer( +config = dpsynth.TabularConfig( domains=domains, discrete_mechanism=discrete_mechanisms.MSTConfig(), ) -result = synth.calibrate( - epsilon=1.0, - delta=1e-6, -)(np.random.default_rng(), sensitive_df) +mechanism = config.calibrate(epsilon=1.0, delta=1e-6) +result = mechanism(np.random.default_rng(), sensitive_df) synthetic_df = result.synthetic_data ``` -### Key Arguments +### Key Configuration Arguments + +When initializing `dpsynth.TabularConfig`: -* `data`: The sensitive input `pd.DataFrame`. * `domains`: Mapping of column names to domain specifications ([`CategoricalAttribute`, `NumericalAttribute`, or `OpenSetCategoricalAttribute`](data_and_terminology.md)). Every key must exist in `data.columns`. -* `epsilon`, `delta`: Total differential privacy budget parameters. * `discrete_mechanism`: Configuration object specifying which DP synthesis mechanism to run (e.g., `MSTConfig()`, `AIMConfig()`, `IndependentConfig()`). * `numerical_bins`: Number of equal-frequency quantile buckets used to discretize continuous numerical columns (default: `32`). -* `one_way_marginal_budget_fraction`: Fraction of total `(epsilon, delta)` - allocated for one-way marginal measurements and domain compression (default: - `0.1`). -* `skip_compression`: If `True`, bypasses the rare-category merging phase. - Note: Compression cannot currently be used simultaneously with - cross-attribute constraints. +* `init_budget_fraction`: Fraction of total `(epsilon, delta)` budget + allocated for per-column initialization such as bounds computation and + partition selection (default: `0.1`). +* `cross_attribute_constraints`: Optional sequence of constraints to enforce + on generated data. + +When calling `config.calibrate(...)`: + +* `epsilon`, `delta`: Total differential privacy budget parameters. Returns a + runnable `TabularMechanism`. -------------------------------------------------------------------------------- -## End-to-End Python Example +## Standalone End-to-End Python Example -Here is a complete Python script demonstrating how to load data, parse a domain -YAML file, configure the AIM mechanism, set a fixed random seed, and generate -synthetic records. +Here is a complete, self-contained Python script demonstrating how to specify a +domain, set up a `TabularConfig`, calibrate the mechanism with a privacy budget, +load sensitive data, synthesize records, and print the first few rows. ```python import dpsynth @@ -72,34 +74,64 @@ from dpsynth import domain import numpy as np import pandas as pd -# 1. Load sensitive tabular data into Pandas -sensitive_df = pd.read_csv("sensitive_transactions.csv") - -# 2. Load domain schema from YAML -attribute_domains = domain.from_yaml_file("transaction_domain.yaml") +# 1. Domain Specification: Define the schema of the tabular dataset +attribute_domains = { + "age": domain.NumericalAttribute(lower_bound=18, upper_bound=90), + "workclass": domain.CategoricalAttribute( + allowed_values=["Private", "Self-emp", "Gov", "Other"] + ), + "education": domain.CategoricalAttribute( + allowed_values=["HS-grad", "Bachelors", "Masters", "PhD"] + ), +} -# 3. Configure and calibrate the synthesizer (AIM) -synth = dpsynth.TabularSynthesizer( +# 2. Setup Config: Configure synthesizer with domain and mechanism choices +config = dpsynth.TabularConfig( domains=attribute_domains, - discrete_mechanism=discrete_mechanisms.AIMConfig( - max_rounds=50, - pgm_iters=1000, - ), -) -calibrated = synth.calibrate( - epsilon=1.0, - delta=1e-6, - numerical_bins=16, # Use 16 quantile buckets for numerical columns + discrete_mechanism=discrete_mechanisms.MSTConfig(), + numerical_bins=16, ) -# 4. Generate Differentially Private synthetic data -seed = 42 -result = calibrated(np.random.default_rng(seed), sensitive_df) +# 3. Calibrate Mechanism: Allocate privacy budget to get runnable mechanism +mechanism = config.calibrate(epsilon=1.0, delta=1e-5) + +# 4. Load Data: Create sensitive input DataFrame matching the domain schema +sensitive_df = pd.DataFrame({ + "age": [25, 42, 30, 55, 62, 29, 38, 47, 51, 33], + "workclass": [ + "Private", + "Gov", + "Private", + "Self-emp", + "Other", + "Private", + "Gov", + "Private", + "Self-emp", + "Private", + ], + "education": [ + "Bachelors", + "Masters", + "HS-grad", + "PhD", + "HS-grad", + "Bachelors", + "HS-grad", + "Masters", + "Bachelors", + "HS-grad", + ], +}) + +# 5. Synthesize Data: Run the calibrated mechanism on the sensitive data +rng = np.random.default_rng(seed=42) +result = mechanism(rng, sensitive_df) synthetic_df = result.synthetic_data -# 5. Save the synthetic dataframe -synthetic_df.to_csv("synthetic_transactions.csv", index=False) -print("Synthetic data successfully generated!") +# 6. Print the first few rows of the generated synthetic dataset +print("Generated Synthetic Data:") +print(synthetic_df.head()) ``` -------------------------------------------------------------------------------- @@ -141,7 +173,7 @@ python3 bin/main.py \ ## Under the Hood: The In-Memory Lifecycle -When you invoke `TabularSynthesizer`, the library performs the following +When you configure and run `TabularConfig`, the library performs the following single-machine pipeline: 1. **Discretization**: Continuous numerical columns are bucketed into @@ -152,11 +184,11 @@ single-machine pipeline: 3. **Domain Compression**: DPSynth measures 1-way marginals with Gaussian noise and merges rare categories into an `"Other"` bucket, producing an un-noised discrete dataset (`mbi.Dataset`). -4. **Mechanism Execution**: Calls `discrete_mechanisms.run_mechanism()` to - execute the selected algorithm (`AIM`, `MST`, etc.) on the discrete dataset. - The mechanism fits a Markov Random Field (`mbi.MarkovRandomField`) via - Private-PGM mirror descent. +4. **Mechanism Execution**: Calls the configured discrete mechanism (`AIM`, + `MST`, etc.) on the discrete dataset. The mechanism fits a Markov Random + Field (`mbi.MarkovRandomField`) via Private-PGM mirror descent. 5. **Sampling & Inversion**: Samples synthetic integer records from the graphical model, unpacks `"Other"` categories, and inverts the integer encoding back to original Pandas dtypes (strings, integers, floating points). + diff --git a/docs/index.md b/docs/index.md index d336613e..3576de82 100644 --- a/docs/index.md +++ b/docs/index.md @@ -58,7 +58,7 @@ dataframes to massive distributed datasets across computing clusters: └────────────────────────────────────────┘ ``` -### 1. In-Memory DataFrame API (`dpsynth.TabularSynthesizer`) +### 1. In-Memory DataFrame API (`dpsynth.TabularConfig`) Optimized for rapid prototyping, research experimentation, and datasets that easily fit within single-machine memory. @@ -128,3 +128,30 @@ APIs: ## Contact -------------------------------------------------------------------------------- + +```{toctree} +:maxdepth: 2 +:caption: Getting Started +:hidden: + +data_and_terminology +in_memory_api +scalable_pipeline_api +``` + +```{toctree} +:maxdepth: 2 +:caption: Guides +:hidden: + +processing_lifecycle +contributors_guide +``` + +```{toctree} +:maxdepth: 2 +:caption: API Reference +:hidden: + +api_reference +``` diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index 96dce10a..00000000 --- a/docs/index.rst +++ /dev/null @@ -1,40 +0,0 @@ -.. Copyright 2026 Google LLC -.. -.. Licensed under the Apache License, Version 2.0 (the "License"); -.. you may not use this file except in compliance with the License. -.. You may obtain a copy of the License at -.. -.. http://www.apache.org/licenses/LICENSE-2.0 -.. -.. Unless required by applicable law or agreed to in writing, software -.. distributed under the License is distributed on an "AS IS" BASIS, -.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -.. See the License for the specific language governing permissions and -.. limitations under the License. - -################################################## - DPSynth: Differentially Private Synthetic Data -################################################## - -.. include:: index.md - :parser: myst_parser.sphinx_ - -.. toctree:: - :maxdepth: 2 - :caption: Getting Started - - data_and_terminology - in_memory_api - -.. toctree:: - :maxdepth: 2 - :caption: Guides - - processing_lifecycle - contributors_guide - -.. toctree:: - :maxdepth: 2 - :caption: API Reference - - api_reference diff --git a/docs/sitemap.md b/docs/sitemap.md index 058eb24c..173bbb94 100644 --- a/docs/sitemap.md +++ b/docs/sitemap.md @@ -11,7 +11,7 @@ * [Why DPSynth?](index.md#why-dpsynth) * [Core APIs and Execution Models](index.md#core-apis-and-execution-models) - * [1. In-Memory DataFrame API (`dpsynth.TabularSynthesizer`)](index.md#1-in-memory-dataframe-api-dpsynth-tabularsynthesizer) + * [1. In-Memory DataFrame API (`dpsynth.TabularConfig`)](index.md#1-in-memory-dataframe-api-dpsynth-tabularconfig) * [2. Scalable PipelineBackend API (`dpsynth.data_generation`)](index.md#2-scalable-pipelinebackend-api-dpsynthdata_generation) * [Documentation Sitemap & Navigation](index.md#documentation-sitemap--navigation) * [Supported Synthesis Algorithms](index.md#supported-synthesis-algorithms) @@ -46,7 +46,7 @@
📁 In-Memory DataFrame API Guide -* [Python API: `dpsynth.TabularSynthesizer`](in_memory_api.md#python-api-dpsynth-tabularsynthesizer) +* [Python API: `dpsynth.TabularConfig`](in_memory_api.md#python-api-dpsynth-tabularconfig) * [Function Signature](in_memory_api.md#function-signature) * [Key Arguments](in_memory_api.md#key-arguments) * [End-to-End Python Example](in_memory_api.md#end-to-end-python-example) diff --git a/dpsynth/__init__.py b/dpsynth/__init__.py index 6e24f6f8..dac7c8d2 100644 --- a/dpsynth/__init__.py +++ b/dpsynth/__init__.py @@ -24,6 +24,8 @@ from dpsynth.data_generation_v3 import TabularConfig from dpsynth.data_generation_v3 import TabularMechanism from dpsynth.data_generation_v3 import TabularSynthesizer +from dpsynth.discrete_mechanisms.discrete import DiscreteConfig +from dpsynth.discrete_mechanisms.discrete import DiscreteMechanism from dpsynth.domain import CategoricalAttribute from dpsynth.domain import FreeFormTextAttribute from dpsynth.domain import NumericalAttribute