Functional definition sets
Haskell implementation of the Overview/Functional definition sets
Sections
One functional definition sets
Substrate functional definition sets
Definition
A functional definition set $F \in \mathcal{F}$ is a set of unit functional transforms, $\forall T \in F~(T \in \mathcal{T}_{\mathrm{f}})$. Functional definition sets are also called fuds. The Fud type is defined as a set of Transform,
newtype Fud = Fud (Set.Set Transform)
A fud is constructed from a set of transforms,
setTransformsFud :: Set.Set Transform -> Maybe Fud
Consider the deck of cards example,
let lluu ll = fromJust $ listsSystem [(v,Set.fromList ww) | (v,ww) <- ll]
let [suit,rank] = map VarStr ["suit","rank"]
[hearts,clubs,diamonds,spades] = map ValStr ["hearts","clubs","diamonds","spades"]
[jack,queen,king,ace] = map ValStr ["J","Q","K","A"]
let uu = lluu [
(suit, [hearts, clubs, diamonds, spades]),
(rank, [jack,queen,king,ace] ++ map ValInt [2..10])]
let vv = Set.fromList [suit, rank]
rp uu
"{(rank,{A,J,K,Q,2,3,4,5,6,7,8,9,10}),(suit,{clubs,diamonds,hearts,spades})}"
rp vv
"{rank,suit}"
let aa = unit (cart uu vv)
rpln $ aall aa
"({(rank,A),(suit,clubs)},1 % 1)"
"({(rank,A),(suit,diamonds)},1 % 1)"
"({(rank,A),(suit,hearts)},1 % 1)"
"({(rank,A),(suit,spades)},1 % 1)"
"({(rank,J),(suit,clubs)},1 % 1)"
"({(rank,J),(suit,diamonds)},1 % 1)"
...
"({(rank,9),(suit,hearts)},1 % 1)"
"({(rank,9),(suit,spades)},1 % 1)"
"({(rank,10),(suit,clubs)},1 % 1)"
"({(rank,10),(suit,diamonds)},1 % 1)"
"({(rank,10),(suit,hearts)},1 % 1)"
"({(rank,10),(suit,spades)},1 % 1)"
A transform $T_{\mathrm{c}}$ can be constructed relating the suit to the colour,
let lltt kk ww qq = trans (unit (Set.fromList [llss (zip (kk ++ ww) ll) | ll <- qq])) (Set.fromList ww)
let colour = VarStr "colour"
red = ValStr "red"; black = ValStr "black"
let ttc = lltt [suit] [colour] [
[hearts, red],
[clubs, black],
[diamonds, red],
[spades, black]]
rpln $ aall $ ttaa ttc
"({(colour,black),(suit,clubs)},1 % 1)"
"({(colour,black),(suit,spades)},1 % 1)"
"({(colour,red),(suit,diamonds)},1 % 1)"
"({(colour,red),(suit,hearts)},1 % 1)"
rp $ und ttc
"{suit}"
rp $ der ttc
"{colour}"
Another transform $T_{\mathrm{t}}$ can be constructed relating the rank to whether it is a pip card or a face card,
let pip_or_face = VarStr "pip_or_face"
pip = ValStr "pip"; face = ValStr "face"
let ttt = lltt [rank] [pip_or_face] ([
[ace,pip],
[king, face],
[queen, face],
[jack, face]] ++
[[ValInt i, pip] | i <- [2..10]])
rpln $ aall $ ttaa ttt
"({(pip_or_face,face),(rank,J)},1 % 1)"
"({(pip_or_face,face),(rank,K)},1 % 1)"
"({(pip_or_face,face),(rank,Q)},1 % 1)"
"({(pip_or_face,pip),(rank,A)},1 % 1)"
"({(pip_or_face,pip),(rank,2)},1 % 1)"
"({(pip_or_face,pip),(rank,3)},1 % 1)"
"({(pip_or_face,pip),(rank,4)},1 % 1)"
"({(pip_or_face,pip),(rank,5)},1 % 1)"
"({(pip_or_face,pip),(rank,6)},1 % 1)"
"({(pip_or_face,pip),(rank,7)},1 % 1)"
"({(pip_or_face,pip),(rank,8)},1 % 1)"
"({(pip_or_face,pip),(rank,9)},1 % 1)"
"({(pip_or_face,pip),(rank,10)},1 % 1)"
rp $ und ttt
"{rank}"
rp $ der ttt
"{pip_or_face}"
Now a fud $F$ can be constructed with the two transforms, $F = \{T_{\mathrm{c}},T_{\mathrm{t}}\}$,
let llff = fromJust . setTransformsFud . Set.fromList
ffll = Set.toList . fudsSetTransform
let ff = llff [ttc, ttt]
rpln $ ffll ff
"({({(colour,black),(suit,clubs)},1 % 1),({(colour,black),(suit,spades)},1 % 1),({(colour,red),(suit,diamonds)},1 % 1),({(colour,red),(suit,hearts)},1 % 1)},{colour})"
"({({(pip_or_face,face),(rank,J)},1 % 1),({(pip_or_face,face),(rank,K)},1 % 1),...,(rank,10)},1 % 1)},{pip_or_face})"
Fuds are constrained such that derived variables can appear in only one transform. That is, the sets of derived variables are disjoint, \[ \forall F \in \mathcal{F}~\forall T_1,T_2 \in F~(T_1 \neq T_2 \implies \mathrm{der}(T_1) \cap \mathrm{der}(T_2) = \emptyset) \]
and [der tt1 `Set.intersection` der tt2 == Set.empty | tt1 <- ffll ff, tt2 <- ffll ff, tt1 /= tt2]
True
The set of fud histograms is $\mathrm{his}(F) := \{\mathrm{his}(T) : T \in F\}$.
The set of fud variables is $\mathrm{vars}(F) := \bigcup \{\mathrm{vars}(X) : X \in \mathrm{his}(F)\}$.
The fud derived is $\mathrm{der}(F) := \bigcup_{T \in F} \mathrm{der}(T) \setminus \bigcup_{T \in F} \mathrm{und}(T)$.
The fud underlying is $\mathrm{und}(F) := \bigcup_{T \in F} \mathrm{und}(T) \setminus \bigcup_{T \in F} \mathrm{der}(T)$,
fudsSetHistogram :: Fud -> Set.Set Histogram
fudsVars :: Fud -> Set.Set Variable
fudsDerived :: Fud -> Set.Set Variable
fudsUnderlying :: Fud -> Set.Set Variable
For example,
let fhis = fudsSetHistogram
fvars = fudsVars
fder = fudsDerived
fund = fudsUnderlying
rpln $ Set.toList $ fhis ff
"{({(colour,black),(suit,clubs)},1 % 1),({(colour,black),(suit,spades)},1 % 1),({(colour,red),(suit,diamonds)},1 % 1),({(colour,red),(suit,hearts)},1 % 1)}"
"{({(pip_or_face,face),(rank,J)},1 % 1),({(pip_or_face,face),(rank,K)},1 % 1),...,({(pip_or_face,pip),(rank,10)},1 % 1)}"
rp $ fvars ff
"{colour,pip_or_face,rank,suit}"
rp $ fund ff
"{rank,suit}"
rp $ fder ff
"{colour,pip_or_face}"
Now consider a third transform $T_{\mathrm{rf}}$ with a derived variable red_face which has value yes for cards which are both red and face, and value no otherwise,
let red_face = VarStr "red_face"
yes = ValStr "yes"; no = ValStr "no"
let ttrf = lltt [colour,pip_or_face] [red_face] [
[red, face, yes],
[red, pip, no],
[black, face, no],
[black, pip, no]]
rpln $ aall $ ttaa ttrf
"({(colour,black),(pip_or_face,face),(red_face,no)},1 % 1)"
"({(colour,black),(pip_or_face,pip),(red_face,no)},1 % 1)"
"({(colour,red),(pip_or_face,face),(red_face,yes)},1 % 1)"
"({(colour,red),(pip_or_face,pip),(red_face,no)},1 % 1)"
rp $ und ttrf
"{colour,pip_or_face}"
rp $ der ttrf
"{red_face}"
The underlying of the third transform, $T_{\mathrm{rf}}$, equals the derived of the other transforms, $T_{\mathrm{c}}$ and $T_{\mathrm{t}}$,
und ttrf == der ttc `Set.union` der ttt
True
Now let fud $G$ contain all three transforms, $G = F \cup \{T_{\mathrm{rf}}\} = \{T_{\mathrm{c}},T_{\mathrm{t}},T_{\mathrm{rf}}\}$,
let gg = llff (ffll ff ++ [ttrf])
rpln $ Set.toList $ fhis gg
"{({(colour,black),(pip_or_face,face),(red_face,no)},1 % 1),...,({(colour,red),(pip_or_face,pip),(red_face,no)},1 % 1)}"
"{({(colour,black),(suit,clubs)},1 % 1),({(colour,black),(suit,spades)},1 % 1),({(colour,red),(suit,diamonds)},1 % 1),({(colour,red),(suit,hearts)},1 % 1)}"
"{({(pip_or_face,face),(rank,J)},1 % 1),({(pip_or_face,face),(rank,K)},1 % 1),...,({(pip_or_face,pip),(rank,10)},1 % 1)}"
rp $ fvars gg
"{colour,pip_or_face,rank,red_face,suit}"
rp $ fund gg
"{rank,suit}"
rp $ fder gg
"{red_face}"
fvars ff `Set.isSubsetOf` fvars gg
True
and [der tt1 `Set.intersection` der tt2 == Set.empty | tt1 <- ffll gg, tt2 <- ffll gg, tt1 /= tt2]
True
The intermediate variables, colour and pip_or_face, are now hidden in $G$, appearing in neither the underlying nor the derived,
rp $ fvars gg `Set.difference` fund gg `Set.difference` fder gg
"{colour,pip_or_face}"
Conversion to transform
A functional definition set is a model, so it can be converted to a functional transform, \[ \begin{eqnarray} F^{\mathrm{T}} &:=& (\prod \mathrm{his}(F)~\%~(\mathrm{der}(F) \cup \mathrm{und}(F)),~\mathrm{der}(F)) \end{eqnarray} \]
fudsTransform :: Fud -> Transform
For example,
let ssplit ll qq = Set.toList (setVarsSetStatesSplit (Set.fromList ll) qq)
let fftt = fudsTransform
rpln $ aall $ ttaa $ fftt ff
"({(colour,black),(pip_or_face,face),(rank,J),(suit,clubs)},1 % 1)"
"({(colour,black),(pip_or_face,face),(rank,J),(suit,spades)},1 % 1)"
"({(colour,black),(pip_or_face,face),(rank,K),(suit,clubs)},1 % 1)"
"({(colour,black),(pip_or_face,face),(rank,K),(suit,spades)},1 % 1)"
"({(colour,black),(pip_or_face,face),(rank,Q),(suit,clubs)},1 % 1)"
"({(colour,black),(pip_or_face,face),(rank,Q),(suit,spades)},1 % 1)"
"({(colour,black),(pip_or_face,pip),(rank,A),(suit,clubs)},1 % 1)"
"({(colour,black),(pip_or_face,pip),(rank,A),(suit,spades)},1 % 1)"
...
"({(colour,red),(pip_or_face,pip),(rank,8),(suit,diamonds)},1 % 1)"
"({(colour,red),(pip_or_face,pip),(rank,8),(suit,hearts)},1 % 1)"
"({(colour,red),(pip_or_face,pip),(rank,9),(suit,diamonds)},1 % 1)"
"({(colour,red),(pip_or_face,pip),(rank,9),(suit,hearts)},1 % 1)"
"({(colour,red),(pip_or_face,pip),(rank,10),(suit,diamonds)},1 % 1)"
"({(colour,red),(pip_or_face,pip),(rank,10),(suit,hearts)},1 % 1)"
rpln $ ssplit [rank,suit] (states (ttaa (fftt ff)))
"({(rank,A),(suit,clubs)},{(colour,black),(pip_or_face,pip)})"
"({(rank,A),(suit,diamonds)},{(colour,red),(pip_or_face,pip)})"
"({(rank,A),(suit,hearts)},{(colour,red),(pip_or_face,pip)})"
"({(rank,A),(suit,spades)},{(colour,black),(pip_or_face,pip)})"
"({(rank,J),(suit,clubs)},{(colour,black),(pip_or_face,face)})"
"({(rank,J),(suit,diamonds)},{(colour,red),(pip_or_face,face)})"
...
"({(rank,9),(suit,hearts)},{(colour,red),(pip_or_face,pip)})"
"({(rank,9),(suit,spades)},{(colour,black),(pip_or_face,pip)})"
"({(rank,10),(suit,clubs)},{(colour,black),(pip_or_face,pip)})"
"({(rank,10),(suit,diamonds)},{(colour,red),(pip_or_face,pip)})"
"({(rank,10),(suit,hearts)},{(colour,red),(pip_or_face,pip)})"
"({(rank,10),(suit,spades)},{(colour,black),(pip_or_face,pip)})"
rpln $ aall $ ttaa $ fftt gg
"({(rank,A),(red_face,no),(suit,clubs)},1 % 1)"
"({(rank,A),(red_face,no),(suit,diamonds)},1 % 1)"
"({(rank,A),(red_face,no),(suit,hearts)},1 % 1)"
"({(rank,A),(red_face,no),(suit,spades)},1 % 1)"
"({(rank,J),(red_face,no),(suit,clubs)},1 % 1)"
"({(rank,J),(red_face,no),(suit,spades)},1 % 1)"
"({(rank,J),(red_face,yes),(suit,diamonds)},1 % 1)"
"({(rank,J),(red_face,yes),(suit,hearts)},1 % 1)"
...
"({(rank,10),(red_face,no),(suit,clubs)},1 % 1)"
"({(rank,10),(red_face,no),(suit,diamonds)},1 % 1)"
"({(rank,10),(red_face,no),(suit,hearts)},1 % 1)"
"({(rank,10),(red_face,no),(suit,spades)},1 % 1)"
rpln $ ssplit [rank,suit] (states (ttaa (fftt gg)))
"({(rank,A),(suit,clubs)},{(red_face,no)})"
"({(rank,A),(suit,diamonds)},{(red_face,no)})"
"({(rank,A),(suit,hearts)},{(red_face,no)})"
"({(rank,A),(suit,spades)},{(red_face,no)})"
"({(rank,J),(suit,clubs)},{(red_face,no)})"
"({(rank,J),(suit,diamonds)},{(red_face,yes)})"
...
"({(rank,9),(suit,hearts)},{(red_face,no)})"
"({(rank,9),(suit,spades)},{(red_face,no)})"
"({(rank,10),(suit,clubs)},{(red_face,no)})"
"({(rank,10),(suit,diamonds)},{(red_face,no)})"
"({(rank,10),(suit,hearts)},{(red_face,no)})"
"({(rank,10),(suit,spades)},{(red_face,no)})"
The resultant transform has the same derived and underlying variables as the fud, $\mathrm{der}(F^{\mathrm{T}}) = \mathrm{der}(F)$ and $\mathrm{und}(F^{\mathrm{T}}) = \mathrm{und}(F)$,
der (fftt ff) == fder ff
True
und (fftt ff) == fund ff
True
der (fftt gg) == fder gg
True
und (fftt gg) == fund gg
True
One functional definition sets
The set of one functional definition sets $\mathcal{F}_{U,\mathrm{1}}$ in system $U$ is the subset of the functional definition sets, $\mathcal{F}_{U,\mathrm{1}} \subset \mathcal{F}$, such that all transforms are one functional and the fuds are not circular. The transform of a one functional definition set is a one functional transform, $\forall F \in \mathcal{F}_{U,1}~(F^{\mathrm{T}} \in \mathcal{T}_{U,\mathrm{f},1})$. In the case of the deck of cards example the fuds, $F$ and $G$, are one functional definition sets, so the reduction of the fud transform histogram to underlying variables is cartesian, $\mathrm{his}(F^{\mathrm{T}})\%V = V^{\mathrm{C}}$ and $\mathrm{his}(G^{\mathrm{T}})\%V = V^{\mathrm{C}}$,
(ttaa (fftt ff)) `ared` vv == unit (cart uu vv)
True
(ttaa (fftt gg)) `ared` vv == unit (cart uu vv)
True
A dependent variable of a one functional definition set $F \in \mathcal{F}_{U,1}$ is any variable that is not a fud underlying variable, $\mathrm{vars}(F) \setminus \mathrm{und}(F)$,
rp $ fvars ff `Set.difference` fund ff
"{colour,pip_or_face}"
rp $ fvars gg `Set.difference` fund gg
"{colour,pip_or_face,red_face}"
Each dependent variable depends on an underlying subset of the fud, $\mathrm{depends} \in \mathcal{F} \times \mathrm{P}(\mathcal{V}) \to \mathcal{F}$ such that $\forall w \in \mathrm{vars}(F) \setminus \mathrm{und}(F)~(\mathrm{depends}(F,\{w\}) \subseteq F)$,
fudsVarsDepends :: Fud -> Set.Set Variable -> Fud
For example,
let depends ff v = fudsVarsDepends ff (Set.singleton v)
depends gg colour == llff [ttc]
True
depends gg pip_or_face == llff [ttt]
True
depends gg red_face == gg
True
Each dependent variable is in a layer. The layer is the length of the longest path of underlying transforms to the dependent variable. Given fud $F \in \mathcal{F}_{U,\mathrm{1}}$, let $l$ be the highest layer, $l = \mathrm{layer}(F,\mathrm{der}(F))$, where $\mathrm{layer} \in \mathcal{F} \times \mathrm{P}(\mathcal{V}) \to \mathbf{N}$ is defined in terms of $\mathrm{depends} \in \mathcal{F} \times \mathrm{P}(\mathcal{V}) \to \mathcal{F}$,
fudsSetVarsLayer :: Fud -> Set.Set Variable -> Integer
For example,
let layer ff v = fudsSetVarsLayer ff (Set.singleton v)
layer gg colour
1
layer gg pip_or_face
1
layer gg red_face
2
Let $F_i$ be the subset of the fud in a particular layer, $F_i = \{T : T \in F,~\mathrm{layer}(F,\mathrm{der}(T))=i\}$. Then $F = \bigcup_{i \in \{1 \ldots l\}} F_i$,
let l = fudsSetVarsLayer gg (fder gg)
[length (ffll ffi) | i <- [1..l], let ffi = llff [tt | tt <- ffll gg, fudsSetVarsLayer gg (der tt) == i]]
[2,1]
A one functional definition set $F \in \mathcal{F}_{U,1}$ is non-overlapping if the sets of variables of the underlying transforms of each of the fud derived variables are disjoint, $\forall v,w \in \mathrm{der}(F)~(v \neq w~\wedge~\mathrm{vars}(\mathrm{depends}(F,\{v\})) \cap \mathrm{vars}(\mathrm{depends}(F,\{w\})) = \emptyset)$,
fudsOverlap :: Fud -> Bool
For example, neither fud $F$ nor fud $G$ are overlapping,
let isnonoverlap = not . fudsOverlap
isnonoverlap ff
True
rp $ fder ff
"{colour,pip_or_face}"
fvars (depends ff colour) `Set.intersection` fvars (depends ff pip_or_face) == Set.empty
True
isnonoverlap gg
True
rp $ fder gg
"{red_face}"
Let a fourth transform $T_{\mathrm{op}}$ with a derived variable odd_pip have value yes for odd pip cards, and value no otherwise,
let odd_pip = VarStr "odd_pip"
let ttop = lltt [rank] [odd_pip] ([
[ace, yes],
[king, no],
[queen, no],
[jack, no]] ++
[[ValInt i, no] | i <- [2,4..10]] ++
[[ValInt i, yes] | i <- [3,5..9]])
rpln $ aall $ ttaa ttop
"({(odd_pip,no),(rank,J)},1 % 1)"
"({(odd_pip,no),(rank,K)},1 % 1)"
"({(odd_pip,no),(rank,Q)},1 % 1)"
"({(odd_pip,no),(rank,2)},1 % 1)"
"({(odd_pip,no),(rank,4)},1 % 1)"
"({(odd_pip,no),(rank,6)},1 % 1)"
"({(odd_pip,no),(rank,8)},1 % 1)"
"({(odd_pip,no),(rank,10)},1 % 1)"
"({(odd_pip,yes),(rank,A)},1 % 1)"
"({(odd_pip,yes),(rank,3)},1 % 1)"
"({(odd_pip,yes),(rank,5)},1 % 1)"
"({(odd_pip,yes),(rank,7)},1 % 1)"
"({(odd_pip,yes),(rank,9)},1 % 1)"
rp $ und ttop
"{rank}"
rp $ der ttop
"{odd_pip}"
Now let fud $H$ contain all three transforms, $H = G \cup \{T_{\mathrm{op}}\} = \{T_{\mathrm{c}},T_{\mathrm{t}},T_{\mathrm{rf}},T_{\mathrm{op}}\}$,
let hh = llff (ffll gg ++ [ttop])
Set.size $ fhis hh
4
rp $ fvars hh
"{colour,odd_pip,pip_or_face,rank,red_face,suit}"
rp $ fund hh
"{rank,suit}"
rp $ fder hh
"{odd_pip,red_face}"
fvars gg `Set.isSubsetOf` fvars hh
True
and [der tt1 `Set.intersection` der tt2 == Set.empty | tt1 <- ffll hh, tt2 <- ffll hh, tt1 /= tt2]
True
rpln $ ssplit [rank,suit] (states (ttaa (fftt hh)))
"({(rank,A),(suit,clubs)},{(odd_pip,yes),(red_face,no)})"
"({(rank,A),(suit,diamonds)},{(odd_pip,yes),(red_face,no)})"
"({(rank,A),(suit,hearts)},{(odd_pip,yes),(red_face,no)})"
"({(rank,A),(suit,spades)},{(odd_pip,yes),(red_face,no)})"
"({(rank,J),(suit,clubs)},{(odd_pip,no),(red_face,no)})"
"({(rank,J),(suit,diamonds)},{(odd_pip,no),(red_face,yes)})"
...
"({(rank,9),(suit,hearts)},{(odd_pip,yes),(red_face,no)})"
"({(rank,9),(suit,spades)},{(odd_pip,yes),(red_face,no)})"
"({(rank,10),(suit,clubs)},{(odd_pip,no),(red_face,no)})"
"({(rank,10),(suit,diamonds)},{(odd_pip,no),(red_face,no)})"
"({(rank,10),(suit,hearts)},{(odd_pip,no),(red_face,no)})"
"({(rank,10),(suit,spades)},{(odd_pip,no),(red_face,no)})"
Fud $H$ is overlapping because the underlying transforms for red_face and odd_pip share rank,
isnonoverlap hh
False
rp $ fvars (depends hh red_face) `Set.intersection` fvars (depends hh odd_pip)
"{rank}"
rp $ fund (depends hh red_face)
"{rank,suit}"
rp $ fund (depends hh odd_pip)
"{rank}"
A one functional transform $T \in \mathcal{T}_{U,\mathrm{f},1}$ is non-overlapping if it is equal to the transform of a non-overlapping fud, $T=F^{\mathrm{T}}$,
transformsIsOverlap :: Transform -> Bool
For example,
let tisnonoverlap = not . transformsIsOverlap
tisnonoverlap (fftt ff) == isnonoverlap ff
True
tisnonoverlap (fftt gg) == isnonoverlap gg
True
tisnonoverlap (fftt hh) == isnonoverlap hh
True
If the transform, $T$, is non-overlapping, then its formal is always independent, $A^{\mathrm{X}} * T = (A^{\mathrm{X}} * T)^{\mathrm{X}}$, where $A$ is any underlying histogram, $\mathrm{vars}(A) \supseteq \mathrm{und}(T)$. For example,
ind aa `tmul` fftt ff == ind (ind aa `tmul` fftt ff)
True
ind aa `tmul` fftt gg == ind (ind aa `tmul` fftt gg)
True
ind aa `tmul` fftt hh == ind (ind aa `tmul` fftt hh)
False
Substrate functional definition sets
Given a set of substrate variables $V$, the set of substrate functional definition sets $\mathcal{F}_{U,V}$ is the subset of one functional definition sets, $\mathcal{F}_{U,V} \subset \mathcal{F}_{U,1}$, that (i) have underlying variables which are subsets of the substrate, $\forall F \in \mathcal{F}_{U,V}~(\mathrm{und}(F) \subseteq V)$, and (ii) consist of partition transforms, $\forall F \in \mathcal{F}_{U,V}~\forall T \in F~\exists P \in \mathrm{B}(\mathrm{und}(T)^{\mathrm{CS}})~(T = P^{\mathrm{T}})$. In addition, partition circularities are excluded by ensuring that the partitions are unique in the fud when flattened to substrate,
systemsSetVarsSetFudSubstrate :: System -> Set.Set Variable -> Maybe (Set.Set Fud)
For example,
let ffvv uu = fromJust $ systemsSetVarsSetFudSubstrate uu (uvars uu)
Set.size $ ffvv $ sysreg 1 1
6
rpln $ Set.toList $ ffvv $ sysreg 1 1
"{}"
"{({({(1,1),({ { {} } },{ {} }),({ { {(1,1),({ { {} } },{ {} })} } },{ {(1,1),({ { {} } },{ {} })} })},1 % 1)},{ { { {(1,1),({ { {} } },{ {} })} } } }),({({(1,1),({ { {(1,1)} } },{ {(1,1)} })},1 % 1)},{ { { {(1,1)} } } }),({({({ { {} } },{ {} })},1 % 1)},{ { { {} } } })}"
"{({({(1,1),({ { {} } },{ {} }),({ { {(1,1),({ { {} } },{ {} })} } },{ {(1,1),({ { {} } },{ {} })} })},1 % 1)},{ { { {(1,1),({ { {} } },{ {} })} } } }),({({({ { {} } },{ {} })},1 % 1)},{ { { {} } } })}"
"{({({(1,1),({ { {(1,1)} } },{ {(1,1)} })},1 % 1)},{ { { {(1,1)} } } })}"
"{({({(1,1),({ { {(1,1)} } },{ {(1,1)} })},1 % 1)},{ { { {(1,1)} } } }),({({({ { {} } },{ {} })},1 % 1)},{ { { {} } } })}"
"{({({({ { {} } },{ {} })},1 % 1)},{ { { {} } } })}"
Avoiding partition circularities is computationally expensive. The infinite-layer substrate functional definition sets $\mathcal{F}_{\infty,U,V}$ is the superset of the substrate functional definition sets, $\mathcal{F}_{\infty,U,V} \supset \mathcal{F}_{U,V}$, that drop the exclusion of partition circularities. The infinite-layer substrate fud set is defined recursively, \[ \begin{eqnarray} \mathcal{F}_{\infty,U,V} &=& \{F : F \subseteq \mathrm{powinf}(U,V)(\emptyset),~\mathrm{und}(F) \subseteq V\} \end{eqnarray} \] where \[ \begin{eqnarray} \mathrm{powinf}(U,V)(F) &:=& F \cup G \cup \mathrm{powinf}(U,V)(F \cup G) : \\ &&\hspace{10em}G = \{P^{\mathrm{T}} : K \subseteq \mathrm{vars}(F) \cup V,~P \in \mathrm{B}(K^{\mathrm{CS}})\} \end{eqnarray} \] The cardinality of the infinite-layer substrate fud set is infinite, $|\mathcal{F}_{\infty,U,V}| = \infty$.
Example - a weather forecast
Some of the concepts above regarding functional definition sets can be demonstrated with the sample of some weather measurements created in States, histories and histograms,
let lluu ll = fromJust $ listsSystem [(v,Set.fromList ww) | (v,ww) <- ll]
llhh vv ev = fromJust $ listsHistory [(IdInt i, llss (zip vv ll)) | (i,ll) <- ev]
ared aa vv = setVarsHistogramsReduce vv aa
red aa ll = setVarsHistogramsReduce (Set.fromList ll) aa
ssplit ll aa = Set.toList (setVarsSetStatesSplit (Set.fromList ll) (states aa))
lltt kk ww qq = trans (unit (Set.fromList [llss (zip (kk ++ ww) ll) | ll <- qq])) (Set.fromList ww)
query qq tt aa ll = norm (qq `tmul` tt `mul` ttaa tt `mul` aa `red` ll)
ent = histogramsEntropy
cent = transformsHistogramsEntropyComponent
rent aa bb = let a = fromRational (size aa); b = fromRational (size bb) in (a+b) * ent (aa `add` bb) - a * ent aa - b * ent bb
tlent tt aa ll = setVarsTransformsHistogramsEntropyLabel (vars aa `Set.difference` (Set.fromList ll)) tt aa
tlalgn tt aa ll = algn (aa `mul` ttaa tt `ared` (der tt `Set.union` Set.fromList ll))
layer ff v = fudsSetVarsLayer ff (Set.singleton v)
isnonoverlap = not . fudsOverlap
let [pressure,cloud,wind,rain] = map VarStr ["pressure","cloud","wind","rain"]
let [low,medium,high,none,light,heavy,strong] = map ValStr ["low","medium","high","none","light","heavy","strong"]
let uu = lluu [
(pressure, [low,medium,high]),
(cloud, [none,light,heavy]),
(wind, [none,light,strong]),
(rain, [none,light,heavy])]
let vv = uvars uu
let hh = llhh [pressure,cloud,wind,rain] [
(1,[high,none,none,none]),
(2,[medium,light,none,light]),
(3,[high,none,light,none]),
(4,[low,heavy,strong,heavy]),
(5,[low,none,light,light]),
(6,[medium,none,light,light]),
(7,[low,heavy,light,heavy]),
(8,[high,none,light,none]),
(9,[medium,light,strong,heavy]),
(10,[medium,light,light,light]),
(11,[high,light,light,heavy]),
(12,[medium,none,none,none]),
(13,[medium,light,none,none]),
(14,[high,light,strong,light]),
(15,[medium,none,light,light]),
(16,[low,heavy,strong,heavy]),
(17,[low,heavy,light,heavy]),
(18,[high,none,none,none]),
(19,[low,light,none,light]),
(20,[high,none,none,none])]
let aa = hhaa hh
let vvc = unit (cart uu vv)
rp uu
"{(cloud,{heavy,light,none}),(pressure,{high,low,medium}),(rain,{heavy,light,none}),(wind,{light,none,strong})}"
rp vv
"{cloud,pressure,rain,wind}"
rpln $ aall aa
"({(cloud,heavy),(pressure,low),(rain,heavy),(wind,light)},2 % 1)"
"({(cloud,heavy),(pressure,low),(rain,heavy),(wind,strong)},2 % 1)"
"({(cloud,light),(pressure,high),(rain,heavy),(wind,light)},1 % 1)"
"({(cloud,light),(pressure,high),(rain,light),(wind,strong)},1 % 1)"
"({(cloud,light),(pressure,low),(rain,light),(wind,none)},1 % 1)"
"({(cloud,light),(pressure,medium),(rain,heavy),(wind,strong)},1 % 1)"
"({(cloud,light),(pressure,medium),(rain,light),(wind,light)},1 % 1)"
"({(cloud,light),(pressure,medium),(rain,light),(wind,none)},1 % 1)"
"({(cloud,light),(pressure,medium),(rain,none),(wind,none)},1 % 1)"
"({(cloud,none),(pressure,high),(rain,none),(wind,light)},2 % 1)"
"({(cloud,none),(pressure,high),(rain,none),(wind,none)},3 % 1)"
"({(cloud,none),(pressure,low),(rain,light),(wind,light)},1 % 1)"
"({(cloud,none),(pressure,medium),(rain,light),(wind,light)},2 % 1)"
"({(cloud,none),(pressure,medium),(rain,none),(wind,none)},1 % 1)"
size aa
20 % 1
We considered the case where we wish to predict the rain given the pressure, cloud and wind in Transforms, by creating a transform which related cloud and wind, $T_{\mathrm{cw}}$,
algn $ aa `red` [cloud,wind]
2.7673349953974995
let cloud_and_wind = VarStr "cloud_and_wind"
let ttcw = lltt [cloud,wind] [cloud_and_wind] [
[none, none, none],
[none, light, light],
[none, strong, light],
[light, none, light],
[light, light, light],
[light, strong, light],
[heavy, none, strong],
[heavy, light, strong],
[heavy, strong, strong]]
It was shown that the alignment between cloud_and_wind and rain is greater than the alignments between any of cloud, wind or pressure and rain,
algn $ aa `red` [pressure,rain]
4.27876667992199
algn $ aa `red` [cloud,rain]
6.415037968300277
algn $ aa `red` [wind,rain]
3.9301313052733455
algn $ aa `mul` ttaa ttcw `red` [cloud_and_wind,rain]
6.743705970350529
or
tlalgn ttcw aa [rain]
6.743705970350529
The relative entropy is
rent (aa `tmul` ttcw) (vvc `tmul` ttcw)
0.9819412530333693
The label entropy is
tlent ttcw aa [rain]
11.51537752694459
In the case of medium pressure, heavy cloud and light winds, the forecast for rain is heavy,
let qq1 = hhaa $ llhh [pressure,cloud,wind] [(1,[medium,heavy,light])]
rpln $ aarr $ query qq1 ttcw aa [rain]
"({(rain,heavy)},1.0)"
In the case of low pressure, but no cloud and light winds, the prediction of the model $T_{\mathrm{cw}}$ is ambiguous,
let qq2 = hhaa $ llhh [pressure,cloud,wind] [(1,[low,none,light])]
rpln $ aarr $ query qq2 ttcw aa [rain]
"({(rain,heavy)},0.16666666666666666)"
"({(rain,light)},0.5833333333333334)"
"({(rain,none)},0.25)"
Then it was found that a better predictor of the rain can be made by constructing a transform $T_{\mathrm{cp}}$ that relates cloud and pressure,
algn $ aa `red` [pressure,cloud]
4.623278490123701
let cloud_and_pressure = VarStr "cloud_and_pressure"
let ttcp = lltt [cloud,pressure] [cloud_and_pressure] [
[none, high, none],
[none, medium, light],
[none, low, light],
[light, high, light],
[light, medium, light],
[light, low, light],
[heavy, high, strong],
[heavy, medium, strong],
[heavy, low, strong]]
tlalgn ttcp aa [rain]
8.020893993593209
The relative entropy is
rent (aa `tmul` ttcp) (vvc `tmul` ttcp)
1.4736881918377236
The label entropy is
tlent ttcp aa [rain]
9.982888235155102
In the case of medium pressure, heavy cloud and light winds, the forecast for rain is still heavy,
rpln $ aarr $ query qq1 ttcp aa [rain]
"({(rain,heavy)},1.0)"
In the case of low pressure, but no cloud and light winds, the prediction of the model $T_{\mathrm{cp}}$ is also ambiguous, but the forecast of no rain is less probable,
let qq2 = hhaa $ llhh [pressure,cloud,wind] [(1,[low,none,light])]
rpln $ aarr $ query qq2 ttcp aa [rain]
"({(rain,heavy)},0.18181818181818182)"
"({(rain,light)},0.6363636363636364)"
"({(rain,none)},0.18181818181818182)"
Now consider a fud constructed from the two transforms $F = \{T_{\mathrm{cw}},T_{\mathrm{cp}}\}$,
let ff = llff [ttcw, ttcp]
rp $ fund ff
"{cloud,pressure,wind}"
rp $ fder ff
"{cloud_and_pressure,cloud_and_wind}"
rp $ der (fftt ff)
"{cloud_and_pressure,cloud_and_wind}"
The label alignment of the fud transform, $F^{\mathrm{T}}$, is
algn $ aa `mul` ttaa (fftt ff) `red` [cloud_and_wind,cloud_and_pressure,rain]
14.228011799347124
or
tlalgn (fftt ff) aa [rain]
14.228011799347124
rpln $ ssplit [cloud_and_wind,cloud_and_pressure] (aa `mul` ttaa (fftt ff) `red` [cloud_and_wind,cloud_and_pressure,rain])
"({(cloud_and_pressure,light),(cloud_and_wind,light)},{(rain,heavy)})"
"({(cloud_and_pressure,light),(cloud_and_wind,light)},{(rain,light)})"
"({(cloud_and_pressure,light),(cloud_and_wind,light)},{(rain,none)})"
"({(cloud_and_pressure,light),(cloud_and_wind,none)},{(rain,none)})"
"({(cloud_and_pressure,none),(cloud_and_wind,light)},{(rain,none)})"
"({(cloud_and_pressure,none),(cloud_and_wind,none)},{(rain,none)})"
"({(cloud_and_pressure,strong),(cloud_and_wind,strong)},{(rain,heavy)})"
So the label alignment of the model, $F$, is even greater than the sample alignment,
algn aa
11.850852273417695
The reason is that transforms $T_{\mathrm{cw}}$ and $T_{\mathrm{cp}}$ share underlying variable cloud,
rp $ und ttcw `Set.intersection` und ttcp
"{cloud}"
so the fud is overlapping
isnonoverlap ff
False
and the formal is not independent,
ind aa `tmul` fftt ff == ind (ind aa `tmul` fftt ff)
False
algn $ ind aa `tmul` fftt ff
6.984724496052557
The formal alignment is non-zero, so consider the label content alignment of the fud transform, $F^{\mathrm{T}}$, which is the label alignment minus the label formal alignment,
algn $ aa `mul` ttaa (fftt ff) `red` [cloud_and_wind,cloud_and_pressure,rain]
14.228011799347124
algn $ ind aa `mul` ttaa (fftt ff) `red` [cloud_and_wind,cloud_and_pressure,rain]
4.885137181846869
tlalgn (fftt ff) aa [rain] - tlalgn (fftt ff) (ind aa) [rain]
9.342874617500254
algn aa
11.850852273417695
So the label content alignment is now less than the sample alignment.
The relative entropy is
rent (aa `tmul` (fftt ff)) (vvc `tmul` (fftt ff))
2.018496742223732
The label entropy is
tlent (fftt ff) aa [rain]
8.018185525433372
So the fud, $F$, has (a) higher relative entropy, (b) higher label content alignment, and (c) lower label entropy than the other models.
Again, in the case of medium pressure, heavy cloud and light winds, the forecast for rain is heavy,
rpln $ aarr $ query qq1 (fftt ff) aa [rain]
"({(rain,heavy)},1.0)"
In the case of low pressure, no cloud and light winds, the forecast is also ambiguous, but the forecast of no rain is the least probable of the models so far,
rpln $ aarr $ query qq2 (fftt ff) aa [rain]
"({(rain,heavy)},0.2)"
"({(rain,light)},0.7)"
"({(rain,none)},0.1)"
This query is effective in the sample, $Q_2 \in (A\%K)^{\mathrm{FS}}$, predicting light rain,
rpln $ aarr $ norm $ qq2 `mul` aa `red` [rain]
"({(rain,light)},1.0)"
so the prediction from the sample differs from the predictions of the models. The fud transform, $F^{\mathrm{T}}$, has the closest forecast,
rpln $ aarr $ query qq2 ttcw aa [rain]
"({(rain,heavy)},0.16666666666666666)"
"({(rain,light)},0.5833333333333334)"
"({(rain,none)},0.25)"
rpln $ aarr $ query qq2 ttcp aa [rain]
"({(rain,heavy)},0.18181818181818182)"
"({(rain,light)},0.6363636363636364)"
"({(rain,none)},0.18181818181818182)"
rpln $ aarr $ query qq2 (fftt ff) aa [rain]
"({(rain,heavy)},0.2)"
"({(rain,light)},0.7)"
"({(rain,none)},0.1)"
So model $F^{\mathrm{T}}$ is most likely in this case.
In the case of high pressure, no cloud and strong winds, the prediction of the model $T_{\mathrm{cw}}$ expects rain,
let qq3 = hhaa $ llhh [pressure,cloud,wind] [(1,[high,none,strong])]
rpln $ aarr $ query qq3 ttcw aa [rain]
"({(rain,heavy)},0.16666666666666666)"
"({(rain,light)},0.5833333333333334)"
"({(rain,none)},0.25)"
but the predictions of the models $T_{\mathrm{cp}}$ and $F^{\mathrm{T}}$ are only for dry weather,
rpln $ aarr $ query qq3 ttcp aa [rain]
"({(rain,none)},1.0)"
rpln $ aarr $ query qq3 (fftt ff) aa [rain]
"({(rain,none)},1.0)"
So models $T_{\mathrm{cp}}$ and $F^{\mathrm{T}}$ are most likely in this case.
Now consider another transform $T_{\mathrm{cwp}}$ which relates the two derived variables, cloud_and_wind and cloud_and_pressure,
let cloud_wind_pressure = VarStr "cloud_wind_pressure"
let ttcwp = lltt [cloud_and_wind,cloud_and_pressure] [cloud_wind_pressure] [
[none, none, none],
[none, light, none],
[none, strong, none],
[light, none, none],
[light, light, light],
[light, strong, light],
[strong, none, none],
[strong, light, light],
[strong, strong, strong]]
If we add the new transform, $T_{\mathrm{cwp}}$, to the fud, $F$, we obtain a two layer fud $G = \{T_{\mathrm{cw}},T_{\mathrm{cp}},T_{\mathrm{cwp}}\}$,
let gg = llff [ttcw, ttcp, ttcwp]
layer gg cloud_and_wind
1
layer gg cloud_and_pressure
1
layer gg cloud_wind_pressure
2
rp $ fund gg
"{cloud,pressure,wind}"
rp $ fder gg
"{cloud_wind_pressure}"
rp $ der (fftt gg)
"{cloud_wind_pressure}"
The alignment of the fud transform, $G^{\mathrm{T}}$, is lower than for the single layer fud, $F^{\mathrm{T}}$,
algn $ aa `mul` ttaa (fftt gg) `red` [cloud_wind_pressure,rain]
9.654261917348958
or
tlalgn (fftt gg) aa [rain]
9.654261917348958
rpln $ ssplit [cloud_wind_pressure] (aa `mul` ttaa (fftt gg) `red` [cloud_wind_pressure,rain])
"({(cloud_wind_pressure,light)},{(rain,heavy)})"
"({(cloud_wind_pressure,light)},{(rain,light)})"
"({(cloud_wind_pressure,light)},{(rain,none)})"
"({(cloud_wind_pressure,none)},{(rain,none)})"
"({(cloud_wind_pressure,strong)},{(rain,heavy)})"
The fud, $G$, is non-overlapping, however,
isnonoverlap gg
True
and so the formal is independent,
ind aa `tmul` fftt gg == ind (ind aa `tmul` fftt gg)
True
algn $ ind aa `tmul` fftt gg
0.0
tlalgn (fftt gg) (ind aa) [rain]
0.0
The label content alignment of the fud transform $G^{\mathrm{T}}$ is greater than the label content alignment of the fud transform $F^{\mathrm{T}}$,
tlalgn (fftt gg) aa [rain] - tlalgn (fftt gg) (ind aa) [rain]
9.654261917348958
tlalgn (fftt ff) aa [rain] - tlalgn (fftt ff) (ind aa) [rain]
9.342874617500254
The relative entropy is
rent (aa `tmul` (fftt gg)) (vvc `tmul` (fftt gg))
0.9832138502623167
The label entropy is
tlent (fftt gg) aa [rain]
8.018185525433372
So the fud transform $G^{\mathrm{T}}$ has lower relative entropy but the same label entropy as fud transform $F^{\mathrm{T}}$. Overall, the forecasts for rain for model $G^{\mathrm{T}}$ are always the same as for model $F^{\mathrm{T}}$,
rpln $ aarr $ query qq1 (fftt gg) aa [rain]
"({(rain,heavy)},1.0)"
rpln $ aarr $ query qq2 (fftt gg) aa [rain]
"({(rain,heavy)},0.2)"
"({(rain,light)},0.7)"
"({(rain,none)},0.1)"
rpln $ aarr $ query qq3 (fftt gg) aa [rain]
"({(rain,none)},1.0)"
let kk = vv `Set.difference` Set.fromList [rain]
and [query qq (fftt gg) aa [rain] == query qq (fftt ff) aa [rain] | ss <- Set.toList (cart uu kk), let qq = unit (Set.singleton ss)]
True
To summarise the models,
[ent (aa `tmul` tt) | tt <- [ttcw, ttcp, fftt ff, fftt gg]]
[0.9502705392332347,0.9972715231823841,1.333074293476779,1.0296530140645737]
[cent tt aa | tt <- [ttcw, ttcp, fftt ff, fftt gg]]
[1.603411018796562,1.5564100348474128,1.2206072645530173,1.5240285439652228]
[rent (aa `tmul` tt) (vvc `tmul` tt) | tt <- [ttcw, ttcp, fftt ff, fftt gg]]
[0.9819412530333693,1.4736881918377236,2.018496742223732,0.9832138502623167]
[tlalgn tt aa [rain] - tlalgn tt (ind aa) [rain] | tt <- [ttcw, ttcp, fftt ff, fftt gg]]
[6.743705970350529,8.020893993593209,9.342874617500254,9.654261917348958]
[tlent tt aa [rain] | tt <- [ttcw, ttcp, fftt ff, fftt gg]]
[11.51537752694459,9.982888235155102,8.018185525433372,8.018185525433372]
The weather forecast example continues in Decompositions.