debut du background sur ZF

5 files changed, 340 insertions, 0 deletions

diff --git a/background/conf.tex b/background/conf.tex
new file mode 100644
index 0000000..4ee8d9f
--- /dev/null
+++ b/background/conf.tex
@@ -0,0 +1,101 @@
+
+%Attacks which violate privacy and confidentiality in ML infer potentially sensitive unobservable information from observable information (e.g., model predictions). 
+\label{sec:bck_aia}
+
+Attacks which violate privacy and confidentiality in ML infer potentially sensitive information from observable information (e.g., model predictions). 
+This leakage of information is a privacy risk if adv learns something about $traindata$ -or the inputs- which would be impossible to learn without access to $targetmodel$. This differentiates between a privacy risk and simple statistical inference~\cite{cormode}. 
+Among the various privacy risks explored in literature pertaining to ML models, attribute inference attacks~\cite{fredrikson2,Mahajan2020DoesLS,yeom,Song2020Overlearning,malekzadeh2021honestbutcurious,MehnazAttInf} infer the specific value of a sensitive attribute for a specific input to ML model given some model observables (e.g., model predictions, parameters, intermediate layerwise outputs) and background information. Based on attack surface being exploited, aia{s} can be categorized into (a) imputation-based attacks and (b) representation-based attacks. 
+
+Let's introduce some notations to guide us in understanding the zoology of those attacks. 
+
+We have a dataset $d:I\rightarrow \mathcal{X}\times\mathcal{S}\times\mathcal{Y}$ containing as column: the features, the sensitive attribute and the ground truth.
+$I$ is a finite set of indices. 
+To access features, sensitive attribute and labels from there indices, we define respectively the following functions:
+\begin{itemize}
+    \item $X:I\rightarrow \mathcal{X},~i\mapsto (d(i))_0$
+    \item $S:I\rightarrow \mathcal{S},~i\mapsto (d(i))_1$
+    \item $Y:I\rightarrow \mathcal{Y},~i\mapsto (d(i))_2$
+\end{itemize}
+Let $(I_0,I_1)$ be a partition of $I$.
+$d$ is split in two datasets $d_0 = d_{{|I_0}}$ and $d_1 = d_{{|I_1}}$ which we call respectively the target dataset and the auxiliary dataset.
+$d_0$ is used to train a machine learning model to infer the ground truth from the features: we call it the target model $targetmodel$.
+
+Regarding attribute inference attack, we differentiate between training time attacks that target $d_0$: the dataset used in training.
+And inference time attack that target data used as input of an already trained target model.
+Our work focuses on the later (see figure \ref{fig:tm2}) but for clear positioning of our contributions, we are going to present both types of attack in this background section.
+
+\noindent\textbf{\underline{Imputation-based attacks}} assume adv has access to non-sensitive attributes in addition to model's predictions and background information (e.g., marginal prior over sensitive attribute and confusion matrix). We review these different imputation-based attacks below:
+
+
+
+\setlength\tabcolsep{3pt}
+\begin{table*}[!htb]
+\caption{Comparison of prior work based on: attack surface exploited (e.g., model predictions ($targetmodel(X(i))$), $X(i)$, $Y(i)$, distribution over $S(i)$ ($P_S$) and confusion matrix between true and predicted output across all training data records ($C(Y(i),targetmodel(X(i)))$), whether $S(i)$ is censored, i.e., included in $traindata$ or inputs, whether they account for class imbalance in $S(i)$, whether adv is active or passive and whether the threat model is blackbox or whitebox. All the attacks assume the knowledge of auxiliary data $auxdata$.}
+\begin{center}
+\footnotesize
+\begin{tabular}{ |c|c|c|c|c|c| }
+ \hline
+ \textbf{Literature} & \textbf{Attack Vector} & \textbf{$S$ is censored?} & \textbf{Imbalance in $S$?} & \textbf{adv} & \textbf{Threat Model} \\
+ \hline
+ \multicolumn{6}{|c|}{\textbf{Imputation-based Attacks}}\\
+ \hline
+ \textbf{Fredrikson et al.}~\cite{fredrikson2} & $X$, $Y$, $targetmodel(X(i))$, \textbf{$P_S$}, $C(Y(i),targetmodel(X(i)))$ & $\checkmark$ & $\times$ &  Passive & Blackbox\\
+ \textbf{Yeom et al.}~\cite{yeom} & $X(i)$, $Y(i)$, $targetmodel()$, \textbf{$P_S$} & $\checkmark$  & $\times$ &  Passive & Blackbox\\
+  \textbf{Mehnaz et al.}~\cite{MehnazAttInf} & $X(i)$, $Y(i)$, $targetmodel()$, \textbf{$P_S$}, $C(Y(i),targetmodel(X(i)))$ & $\checkmark$ & $\times$ &  Passive & Blackbox\\
+  \textbf{Jayaraman and Evans}~\cite{jayaraman2022attribute} & $X(i)$, $Y(i)$, $targetmodel()$, \textbf{$P_S$}, $C(Y(i),targetmodel(X(i)))$ & $\times$, $\checkmark$ & $\times$ &  Passive & Whitebox\\
+  \hline
+   \multicolumn{6}{|c|}{\textbf{Representation-based Attacks}}\\
+  \hline
+ \textbf{Song et al.}~\cite{Song2020Overlearning} & $targetmodel(X(i))$ & $\times$ & $\times$ &  Passive & Both\\
+ \textbf{Mahajan et al.}~\cite{Mahajan2020DoesLS} & $targetmodel(X(i))$ & $\checkmark$ & $\times$ &  Passive & Blackbox\\
+ \textbf{Malekzadeh et al.}~\cite{malekzadeh2021honestbutcurious} & $targetmodel(X(i))$ & $\times$ & $\times$ &  Active & Blackbox\\
+ \textbf{Our Work} & $targetmodel(X(i))$ & $\times$, $\checkmark$ & $\checkmark$ &  Passive & Blackbox \\
+ \hline
+\end{tabular}
+\end{center}
+\label{tab:summary}
+\end{table*}
+
+\label{sec:bck_aia}
+
+\begin{itemize}
+    \item \textbf{Fredrikson et al.~\cite{fredrikson2}} assumes that adv has access to $targetmodel(X(i))$. 
+    For this attack it is required that $X$ can be written $X(i) = (\cdots,S(i),\cdots)$.
+    We will refer to this case as "\textit{S is in the input}".
+    Fredrikson et al. attack generates an input with different possible values of the sensitive attribute
+    It then chooses the most likely value based on $targetmodel(X(i))$.
+
+    \item \noindent\textbf{Yeom et al.~\cite{yeom}} assumes a distribution $P_S$ over $S$ which is used to estimate the value of $S$ for an arbitrary data record. They propose three different variants of AS based on assumptions on $P_S$: Attack 1 leverages membership oracle to determine the value of $S(i)$ and Attack 2 and 3 assume different types of distributions over $S$.
+    For this attack to work, $S$ is in the input and the data points being attacked belong to the target dataset 
+    
+    \item \textbf{Mehnaz et al.~\cite{MehnazAttInf}} improves upon Fredrikson et al.~\cite{fredrikson1,fredrikson2} by exploiting $targetmodel\circ X$ and $X$, with $S$ in the input. The attack relies on the intuition that $targetmodel$'s output confidence is higher when the input has the correct value of $S$ as $targetmodel$ encountered the target record with that attribute during training. Their attack involves generating multiple instances of input with different values of $S(i)$ (similar to Fredrikson et al.~\cite{fredrikson1,fredrikson2}) and identifying the most likely value of $S$.
+\end{itemize}
+
+An appropriate baseline to identify whether such attacks are indeed a privacy risk is to use data imputation, i.e., train an ML model to infer value of missing attribute from other non-sensitive attributes without $targetmodel(X(i))$~\cite{jayaraman2022attribute}. Jayaraman and Evans~\cite{jayaraman2022attribute} find that existing blackbox imputation-based attacks~\cite{yeom,fredrikson2,MehnazAttInf} do not perform any better than data imputation. In other words, the perceived privacy risk is actually stemming from statistical inference and hence not an actual privacy risk.
+
+To address this, Jayaraman and Evans~\cite{jayaraman2022attribute} propose a whitebox aia which outperforms prior blackbox attacks as well as data imputation in the setting where there is limited knowledge of data for adv. However, since the attack is in a whitebox setting, we omit a detailed description of the attack. All these attacks require that: 
+
+\begin{itemize}
+    \item $S$ is in the input data records which is not always the case in realistic settings,
+    \item $X(i)$ being attacked belong to the target dataset.
+\end{itemize}
+
+\noindent\textbf{\underline{Representation-based attacks}} exploit the distinguishable intermediate layer outputs or predictions for different values of sensitive attributes~\cite{Song2020Overlearning,Mahajan2020DoesLS,malekzadeh2021honestbutcurious}. For instance, the distribution of $targetmodel\circ X$ for \textit{males} is different from the output prediction distribution for \textit{females}. We describe the existing attacks of this category below:
+
+\begin{itemize}
+\item \textbf{Song et al.~\cite{Song2020Overlearning} / Mahajan et al.~\cite{Mahajan2020DoesLS}} assume that $S$ is not in the input. adv only observes $targetmodel\circ X$. adv trains an ML attack model $ackmodel$ to map the output predictions $targetmodel(X(i))$ to $S(i)$.
+In other words, the statistic $\hat{S}$ used to infer $S$ is of the form: $ \hat{S} = 1_{[0.5,1]}\circ ackmodel\circ targetmodel\circ X$, where $attackmodel: [0,1]\rightarrow[0,1]$.
+
+
+\item \textbf{Malekzadeh et al.~\cite{malekzadeh2021honestbutcurious}} considers the setting where adv trains $targetmodel$ with a special loss function to explicitly encode information about $S(i)$ in $targetmodel(X(i))$.
+It makes it easier to extract the sensitive attribute during inference. In this setting, the model builder is malicious and actively introduces a ``backdoor''.
+\end{itemize}
+
+Our work focuses on representation-based aia in a blackbox setting at inference time. We focus on Song et al.~\cite{Song2020Overlearning} and Mahajan et al.~\cite{Mahajan2020DoesLS} as our baselines. 
+These attacks do not account for class imbalance in sensitive attribute commonly present in data from real-world applications which could effect adv's attack success~\cite{classIMb1,classIMb2}.
+In our evaluation, we consider an aia using an adaptive threshold which outperforms these baselines attacks (Section~\ref{sec:evalAIA}).
+Malekzadeh et al.~\cite{malekzadeh2021honestbutcurious} has a different threat model where adv explicitly modifies the training to enhance the leakage of $S$. 
+We do not assume such access to $targetmodel$ in a blackbox setting. 
+In addition, these attacks did not take into consideration the possibility to infer the sensitive attribute solely from the hard labels.
+We summarize relevant prior work in Table~\ref{tab:summary}.
+
diff --git a/background/eq.tex b/background/eq.tex
new file mode 100644
index 0000000..8a76ee7
--- /dev/null
+++ b/background/eq.tex
@@ -0,0 +1,42 @@
+
+\label{sec:bck_fair}
+Algorithmic fairness aims at reducing biases in ML model predictions.
+Indeed, data records belonging to certain subgroups influence $targetmodel$'s predictions more than others.
+For instance in criminal justice, the ethnicity of a culprit plays a non-negligible role in the prediction of them reoffending~\cite{fairjustice}. Generally, data records in the minority subgroup face unfair prediction behaviour compared to data records in the majority subgroup. These subgroups are identified based on a sensitive attribute (e.g., race or sex).
+Those biases are learnt by $targetmodel$ as they are part of the distribution of the training dataset. 
+There is two main categories of fairness of a ML model: 
+
+\textbf{Individual fairness} ensures that two data records with same attributes except for $S$ have the same model prediction.
+This notion does not dwell on sensitive attribute and as such is not really useful in our goal of mitigating attribute inference attack at inference time. 
+So we set it aside for the rest of the paper.
+
+\textbf{Group fairness} comes from the idea that different subgroups defined by an attribute such a skin color or gender should be treated equally. 
+We focus our study on group fairness where $S$ represents either sex or race (i.e., $S(i)$ equals to 0 for woman, 1 for man, and 0 for black, 1 for white, respectively). 
+There are different definitions of group fairness which have been introduced in prior work. 
+We discuss two well-established and commonly used metrics: demographic parity and equality of odds. 
+
+\begin{definition}
+\label{def:dp}
+    $\hat{Y}$ satisfies demparity for $S$ if and only if: $P(\hat{Y}=0 | S=0) = P(\hat{Y}=0 | S=1)$.
+    From that, we will call $|P(\hat{Y}=0 | S=0) - P(\hat{Y}=0 | S=1)|$ the demPar-level of $\hat{Y}$.
+\end{definition}
+
+demparity is the historical definition of fairness.
+Legally, disparate impact is the fairness definition recognized by law, where 80\% disparity is an agreed upon tolerance decided in the legal arena.
+demparity ensures that the number of correct prediction is the same for each population.
+However, this may result in different false positive and true positive rates if the true outcome does actually vary with $S$~\cite{dpbad}.
+Hardt et al.~\cite{fairmetric2} proposed eo as a modification of demparity to ensure that both the true positive rate and false positive rate will be the same for each population.
+
+\begin{definition}
+    \label{def:eo}
+    $\hat{Y}$, classifier of $Y$, satisfies equality of odds for $S$ if and only if: $\forall (\hat{y},y)\in\{0,1\}^2 \quad 
+        P(\hat{Y}=\hat{y} | S=0,Y=y) = P(\hat{Y}=\hat{y} | S=1,Y=y)$.
+\end{definition}
+
+The above fairness definitions can be achieved using three main fairness mechanisms: (a) pre-processing, (b) in-processing and (c) post-processing. \textit{Pre-processing} algorithms such as reweighing requires access to the training data and assigns weights to the data records to remove discrimination~\cite{preprocessing}.
+\textit{In-processing} algorithms such as advdebias~\cite{debiase} and egd~\cite{reductions} add constraint during $targetmodel$'s training to ensure fairness. %reductions
+\textit{Post-processing} techniques, in turn, hide the bias in output predictions to satisfy the above fairness constraints but the underlying model is still biased. 
+Similar to previous work~\cite{chang2021privacy}, we focus on in-processing algorithms.
+
+Our work focuses on the theoretical guaranties on attribute inference attacks given by the different fairness notions and not so much on how to implement in-processing fairness mechanism.
+Nevertheless in the experiment section we try production ready state of the art implementations of those fairness constraints along unconstrained ML algorithm.
diff --git a/background/main.tex b/background/main.tex
new file mode 100644
index 0000000..93ee12b
--- /dev/null
+++ b/background/main.tex
@@ -0,0 +1,78 @@
+Nous présentons dans ce chapitre les différentes théories et concepts sur les quelles se basent nos développements.
+\section{Mathématiques}
+L'originie de l'IA est mathématique~\cite{dartmouth,lecun2019quand}.
+Nous utilisons dans ce manuscrit principalement deux théories : l'optimisation pour entraîner les modèles et les probabilitées pour les évaluer.
+Ainsi nous présentons dans cette section les prérequi necessaire pour comprendre les prochains dévelopements. 
+Cette section ne serai être en cours exhaustif mais a pour but de mettre en place les définitions et les principaux théorèmes qui nous allons utiliser.
+Nous supposons que le lecteur est familier du clacul des prédicats.
+Nous utiliserons les quantificateurs $\forall$ (pour tout) et $\exists$ (il existe tel que).
+Nous utiliserons aussi les opératuer logiques suivant que nous définissons par leur tables de véritée :
+\begin{equation}
+\begin{matrix}
+a & b & a\iff b & a\implies b & a\wedge b & a\vee b & \neg a\\
+0 & 0 & 1 & 1 & 0 & 0 & 1\\
+0 & 1 & 0 & 1 & 0 & 1 & 1\\
+1 & 0 & 0 & 0 & 0 & 1 & 0\\
+1 & 1 & 1 & 1 & 1 & 1 & 0\\
+\end{matrix}
+\end{equation}
+\subsection{Ensembles et fonctions}
+    \input{background/set}
+
+\subsection{Algèbre linéaire}
+    \subsubsection{Espace vectoriel}
+    \subsubsection{Application linéaires}
+    \subsubsection{Matrices}
+
+\subsection{Mesurer le hasard pour prédire et inférer}
+    \label{sec:background-proba}
+    \input{background/proba}
+    %\subsection{Théorie de la mesure}
+    %\subsection{Probabilitées}
+    %\subsection{Statistiques}
+
+\subsection{Topologie}
+    \subsubsection{Distances et normes}
+    \subsubsection{Espaces topologiques}
+    \subsubsection{Application aux fonctions}
+
+\subsection{Calcul différentiel}
+    \subsubsection{Différentiel}
+    \subsubsection{Gradient}
+
+\subsection{Optimisation}
+    \label{sec:background-opti}
+    \subsubsection{Multiplicateurs de Lagrange}
+
+    \subsubsection{Descente de gradient}
+        \paragraph{Descente de gradient stochastique}
+
+        \paragraph{Descente de gradient exponentiée}
+
+\section{Apprentissage automatique}
+    \label{sec:background-ml}
+    \subsection{Principe}
+    \subsection{Entraîner un modèle}
+        \subsubsection{Fonction de coût}
+    \subsection{Evaluer un modèle}
+        \subsubsection{Classification}
+            \paragraph{La courbe ROC}
+            \paragraph{La courbe de precision/recall} 
+        \subsubsection{Regression}
+    \subsection{Décentralisation}
+        \subsubsection{Federated learning}
+    \subsection{Modèles génératifs}
+    \label{sec:background-generation}
+
+\section{Equité}
+    \label{sec:background-eq}
+    \input{background/eq}
+    %\subsection{Différentes notions d'équité}
+
+\section{Confidentialité}
+    \label{sec:background-conf}
+    \input{background/conf}
+    %\subsection{Mitiger l'inéquitée}
+        %\subsubsection{Preprocessing}
+       % \subsubsection{Inprocessing}
+       %\subsubsection{Postprocessing}
diff --git a/background/proba.tex b/background/proba.tex
new file mode 100644
index 0000000..bea43e7
--- /dev/null
+++ b/background/proba.tex
@@ -0,0 +1,42 @@
+
+Probability theory is deeply linked with machine learning and most of the properties of machine learning, such as differential privacy, fairness definitions, utility metrics... are often mathematically written within this framework.
+This paper does not differ and hence we provide a short background of this field and how it connects with the previously defined notions of ML introduced in section \ref{sec:ml}.
+
+Soit $A$ un ensemble.
+L'ensemble des parties de $A$ est $\mathcal{P}(A)$. 
+Chaque élément $a \in \mathcal{P}(A)$ est tel que $a \subset A$.
+Une tribue $\mathcal{A}$ est un sous esemble de $\mathcal{P}(A)$ qui contien $\emptyset$, $A$ par complémentaire est union dénombrable.
+Nous disons que $(A,\mathcal{A})$ est un espace mesurable.
+Une mesure $d$ est une fonction $d$:$\mathcal{A}$ $\rightarrow$ $[0,+\infty]$ telle que $d(\emptyset) = 0$ et $d\left(\bigcup_{i\in \mathbb{N}} A_i\right) = \sum_{i\in \mathbb{N}}d(A_i)$ pour chaque $(A_1, A_2, \cdots) \in \mathcal{A}^\mathbb{N} $ avec $\forall (i,j) A_i\cap A_j = \emptyset$.
+Nous disons alors que $(A, \mathcal{A}, d)$ est un espace mesuré.
+Nous appelons fonction mesurable un fonction de $A$ à $B$ telle que  $\forall b\in\mathcal{B}$~$f^{-1}(b)\in\mathcal{A}$.
+Nous notons alors $f:(A, \mathcal{A})\rightarrow (B, \mathcal{B})$ ou $f:(A, \mathcal{A},d)\rightarrow (B, \mathcal{B})$
+
+Dans le cas particulier où $d(A) = 1$, nous appelons $d$ une mesure de probabilité.
+ $(A,\mathcal{A},d)$ est alors un espace probailisé et les fonctions mesurables sur cet espace sont appelés variables aléatoires.
+Le loi de probabilité d'une variable aléatoire $f$ sur $(X,\mathcal{X})$ est la mesure de probabilite suivante :
+$d_X :\mathcal{X}\rightarrow [0,1]$, $x\mapsto d(X^{-1}(x))$.
+
+Having introduced probability theory, we explicit the relation with the ML theory described previously.
+Let $I$ a finite set, $\mathcal{X}$, $\mathcal{S}$ and $\mathcal{Y}$ the sets of features, sensitive attribute and label.
+Let $d:I\rightarrow \mathcal{X}\times\mathcal{S}\times\mathcal{Y}$ a dataset.
+Let $\#$ be the measure on $(I,\mathcal{P}(I))$ which maps to every $a$ in $\mathcal{P}(I)$ the number of elements of $a$.
+Let $P:\mathcal{P}(I)\rightarrow [0,1]$, $a\mapsto \frac{\#(a)}{\#(I)}$.
+Then $(I, \mathcal{P}(I), P)$ is a probability space.
+On this space we can define the following random variables:
+\begin{itemize}
+    \item $X:I\rightarrow \mathcal{X},~i\mapsto (d(i))_0$
+    \item $S:I\rightarrow \mathcal{S},~i\mapsto (d(i))_1$
+    \item $Y:I\rightarrow \mathcal{Y},~i\mapsto (d(i))_2$
+\end{itemize}
+Where for a vector $u$, $u_j$ refers to the $j$th element of $u$.
+
+From there we can define various random variables that will be useful in the rest of the paper.
+For instance $\hat{Y}=f\circ X$ is random variable that represents the prediction of a trained machine learning model $f$. 
+We can use it to write the accuracy in a compact way: $P(\hat{Y}=Y)$ by using the well accepted abuse of notations that for a random variable $A$ and an event $a$, 
+$\{A\in a\} = \{i\in\mathcal{P}(I)~|~A(i)\in a\} = A^{-1}(a)$.
+The accuracy is a reliable metric of a trained model's utility when $P(Y=0) = P(Y=1) = \frac{1}{2}$ but not so much when there is unbalance in $Y$. 
+To take into account an eventual unbalanced distribution of the labels, we will consider the balanced accuracy : 
+$\frac{P(\hat{Y}=0|Y=0) + P(\hat{Y}=1|Y=1)}{2}$.
+
+Finally in the context of attribute inference attack at inference time, we define the random variable $\hat{S}=a\circ \hat{Y}$ where here $a$ is a machine learning model trained to infer sensitive attribute from model's output. 
diff --git a/background/set.tex b/background/set.tex
new file mode 100644
index 0000000..c058648
--- /dev/null
+++ b/background/set.tex
@@ -0,0 +1,77 @@
+Commencons donc cette section préliminaire avec les définitions et quelques porpiété des ensemble et de fonctions.
+Commencons par les ensembles.
+Nous utilisons ici la théorie des ensembles Zermelo–Fraenkel (ZF).
+Si nous avons, dans ce manuscrit, besoin d'objet plus grand que les ensembles, nous les appelerons classes bien qu'il soit hors de propos de présenter ici la théorie Von Neumann–Bernays–Gödel (NBG).
+Nous allons présenter ZF de manière assez succinte, juste suffisante pour réaliser les clalculs du Chapitre~\ref{sec:fini}.
+Si le lecteur souhaite plus de détail sur ces théories nous le renvoyons à \textit{Elements of set thoery} de Herbert B. Enderton~\cite{enderton1977elements}.
+
+\subsubsection{Axiomes de la théroie ZF}
+Nous présentons dans cette section les axiomes de la théorie ZF.
+Ces axiomes sont la pierre angulaire des tous les dévleppoements mathématiques que nous ferons dans ce manuscrit.
+Pour un lecteur qui ne serai pas familier de cette théorie, disons qu'il s'agit de modéliser formellement le principe d'ensemble.
+C'est à dire le principe de ranger des choses, les éléments, dans des boîtes, les ensembles.
+
+\paragraph{Axiome d'Extensionnalité} 
+Deux ensemble $A$ et $B$ sont égaut si et seulement si ils ont les mêmes éléments.
+\begin{equation}
+\forall A\forall B (\forall x~x\in A \iff x\in B) \implies A=B
+\end{equation}
+
+\paragraph{Axiome de l'Ensemble vide}
+Il exite un ensemble qui ne contient aucun élément.
+Nous le notons donc $\{\}$ ou $\emptyset$.
+
+\paragraph{Axiome de la Paire}
+\begin{equation}
+\forall A \forall B \exists \{A,B\}\forall c(c\in \{A,B\}\iff c=A\vee c=B)
+\end{equation}
+
+\paragraph{Axiome de l'Union}
+Pour tout ensembles $A$, il exist un ensemble $\bigcup A$ qui soit exactement composé des éléments de chaque élément de $A$.
+\begin{equation}
+\forall A \exists \bigcup A \forall b \left(b\in\bigcup A\iff \exists a\in A~ b\in a\right)
+\end{equation}
+
+\paragraph{Axiome de l'ensemble des parties}
+Pour tout ensemble $A$ il existe un ensemble $\mathcal{P}(A)$ qui est l'ensemble des sous-ensembles (ou parties) de $A$.
+\begin{equation}
+\forall A \exists \mathcal{P}(A) ~ P\subset A \iff P\in \mathcal{P}(A)
+\end{equation}
+
+\paragraph{Axiome \textit{Aussonderung}}
+Pour toute formule $F$ (au sens du clacul des prédicats et du vocabulaire $\in$, $=$) qui ne pédend pas de $B$ et tout ensemble A, il existe un ensemble $B = \{a\in A | F\}$ qui est tel que 
+$\forall b\in B (b\in A \wedge F)$
+
+\paragraph{Axiome du choix}
+\begin{definition}[Fonction]
+qsdf
+\end{definition}
+
+\paragraph{Axiome de l'infini}
+\begin{equation}
+\exists A\forall a\in A~(\emptyset \in A \wedge a^+\in A)
+\end{equation}
+Où $a^+ = a\cup \{a\}$.
+Nous appelons un tel $A$, un ensemble récursif.
+
+\begin{definition}[Ensemble usuels]
+Soit $C$ la classe des ensembles récursif.
+Soit $A$ un ensemble récursif.
+Nous appelons $\mathbb{N}$ l'ensemble des entier naturels que nous définissons comme suit :
+\begin{equation}
+\mathbb{N} = \{n\in A~|~\forall c\in C~n\in c\}
+\end{equation}
+$\mathbb{N}$ est bien en ensemble d'après l'axiome Aussonderung.
+Cette construction permet de définir les opérations d'addition et de multiplication~\cite{enderton1977elements} ainsi que les autres ensembles usuels qui nous utiliserons dans ce manuscrit.
+Ainsi nous définisson $\mathbb{Z} = \{$ : l'ensemble des entiers relatifs l'union de $\mathbb{N}$ et de $-\mathbb{N} = \{$
+
+\end{definition}
+
+\paragraph{Axiome de remplacement}
+
+\paragraph{Axiome de régularitée}
+
+
+
+
+