Derived set maximisers

Haskell commentary on the implementation of Tractable and Practicable Inducers/Haskell Implementation/Derived set maximisers

Sections

Limited-derived derived variables set list maximiser

Highest-layer limited-derived derived variables set list maximiser

Highest-layer limited-derived derived variables set builder

No-sum-layer highest-layer limited-derived derived variables set builder

Level limited-derived derived variables set list maximiser

Level no-sum-layer highest-layer limited-derived derived variables set builder

Limited-derived derived variables set list maximiser

Define the limited-derived derived variables set list maximiser \[ Z_{P,A,A_R,F,\mathrm{D}} = \mathrm{maximiseLister}(X_{P,A,A_R,F,\mathrm{D}},P_{P,A,A_R,F,\mathrm{D}},\mathrm{top}(\mathrm{omax}),R_{P,A,A_R,F,\mathrm{D}}) \] where (i) the optimiser function is \[ \begin{eqnarray} &&X_{P,A,A_R,F,\mathrm{D}} = \{(K,(I_{\mathrm{a}}^{ * }(A * G^{\mathrm{T}})-I_{\mathrm{a}}^{ * }(A_R * G^{\mathrm{T}}))/I_{\mathrm{cvl}}^{ * }(G)) : \\ &&\hspace{10em}K \subseteq \mathrm{vars}(F),~K \neq \emptyset,~G = \mathrm{depends}(F,K)\} \end{eqnarray} \] (ii) the neighbourhood function is (Text) \[ \begin{eqnarray} &&P_{P,A,A_R,F,\mathrm{D}}(D) = \{(J,X_{P,A,A_R,F,\mathrm{D}}(J)) : \\ &&\hspace{2em}(K,\cdot) \in D,~w \in \mathrm{vars}(F) \setminus V_A \setminus K,\\ &&\hspace{2em}J = K \cup \{w\},~|J^{\mathrm{C}}| \leq \mathrm{wmax},~\mathrm{der}(\mathrm{depends}(F,J)) = J\} \end{eqnarray} \] and (iii) the initial subset is (Text) \[ \begin{eqnarray} &&R_{P,A,A_R,F,\mathrm{D}} = \{(J,X_{P,A,A_R,F,\mathrm{D}}(J)) : \\ &&\hspace{2em}w,u \in \mathrm{vars}(F) \setminus V_A,~u \neq w,\\ &&\hspace{2em}J = \{w,u\},~|J^{\mathrm{C}}| \leq \mathrm{wmax},~\mathrm{der}(\mathrm{depends}(F,J)) = J\} \end{eqnarray} \] The optimised limited-model fuds are (Text) \[ \begin{eqnarray} &&\{\mathrm{depends}(F_{\mathrm{L}},K) : \\ &&\hspace{5em}\{F_{\mathrm{L}}\} = \mathrm{leaves}(\mathrm{tree}(Z_{P,A,A_R,\mathrm{L}})),\\ &&\hspace{5em}K \in \mathrm{maxd}(\mathrm{elements}(Z_{P,A,A_R,F_{\mathrm{L}},\mathrm{D}}))\} \subset \mathcal{F}_{\infty,U_A,V_A} \cap \mathcal{F}_{\mathrm{q}} \end{eqnarray} \] The neighbourhood function is defined in module AlignmentPracticable,

parametersSystemsSamplesShufflesFudsFunctionNeighbourhoodDerived :: 
  Integer -> System -> Histogram -> Histogram -> Fud -> Map.Map (Set.Set Variable) Double -> 
  Maybe (Map.Map (Set.Set Variable) Double)

parametersSystemsSamplesShufflesFudsFunctionNeighbourhoodDerived wmax uu aa aarr ff bb
  ...
  | ... = Just $ llmm 
      [(jj, (algn (aa `fmul` gg) - algn (aarr `fmul` gg)) / (x' ** (1/m))) |
        kk <- keys bb, w <- qqll (qq `minus` kk),  
        let jj = kk `add` w, let x = vol uu jj, x <= wmax, 
        let gg = depends ff jj, fder gg == jj,
        let x' = fromIntegral x, let m = fromIntegral (Set.size jj)]
  | otherwise = Nothing
  where
    qq = fvars ff `minus` vars aa
    ...

The initial function is defined in module AlignmentPracticable,

parametersSystemsSamplesShufflesFudsFunctionInitialDerived :: 
  Integer -> System -> Histogram -> Histogram -> Fud -> Maybe (Map.Map (Set.Set Variable) Double)

parametersSystemsSamplesShufflesFudsFunctionInitialDerived wmax uu aa aarr ff
  ...
  | ... = Just $ llmm 
      [(jj, (algn (aa `fmul` gg) - algn (aarr `fmul` gg)) / (x' ** (1/m))) |
        w <- qqll qq, u <- qqll qq, u /= w,  
        let jj = llqq [w,u], let x = vol uu jj, x <= wmax, 
        let gg = depends ff jj, fder gg == jj,
        let x' = fromIntegral x, let m = fromIntegral (Set.size jj)]
  | otherwise = Nothing
  where
    qq = fvars ff `minus` vars aa
    ...

The optimiser function is defined in module AlignmentPracticable,

parametersSystemsSamplesShufflesFudsFunctionOptimiserDerived :: 
  Integer -> Integer -> System -> Histogram -> Histogram -> Fud ->  
  Maybe [(Map.Map (Set.Set Variable) Double, Map.Map (Set.Set Variable) Double)]

parametersSystemsSamplesShufflesFudsFunctionOptimiserDerived wmax omax uu aa aarr ff
  ...
  | ... = Just $
      opt (zzcsdderneigh wmax uu aa aarr ff) (topff omax ff) (zzcsdderinit wmax uu aa aarr ff)
  | otherwise = Nothing
  where
    opt pp ii rr = let qq = ii rr in (qq, rr) : ls qq
      where
        ls yy = if isempty yy then [] else let nn = pp yy; qq = ii nn in (qq, nn) : ls qq
    zzcsdderinit wmax uu aa aarr ff =  fromJust $
      parametersSystemsSamplesShufflesFudsFunctionInitialDerived wmax uu aa aarr ff
    zzcsdderneigh wmax uu aa aarr ff bb = fromJust $
      parametersSystemsSamplesShufflesFudsFunctionNeighbourhoodDerived wmax uu aa aarr ff bb
    ...

For example,

let zzffcstupinit aa rr ff xmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionInitialTuple xmax (sys aa `uunion` fsys ff) aa rr ff

let zzffcstupneigh aa rr kk a ff xmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionNeighbourhoodTuple xmax (sys aa `uunion` fsys ff) aa rr ff (Map.singleton kk a)

let zzffcsdderinit aa rr ff wmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionInitialDerived wmax (sys aa `uunion` fsys ff) aa rr ff

let zzffcsdderneigh aa rr kk a ff wmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionNeighbourhoodDerived wmax (sys aa `uunion` fsys ff) aa rr ff (Map.singleton kk a)

let zzffcsdderopt aa rr ff wmax omax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionOptimiserDerived wmax omax (sys aa `uunion` fsys ff) aa rr ff

let zzllfuds aa aarr lmax xmax omax bmax mmax umax pmax = fst $ fromJust $ parametersSystemsSamplesShufflesListVariablesSearcherFud lmax xmax omax bmax mmax umax pmax (sys aa) aa aarr [VarInt i | i <-[10..]]

let ff = qqff $ Set.fromList $ [(trans (regtranspose [1,5] (regdiag 3 2)) $ Set.singleton (VarInt 5)),(trans (regtranspose [2,6] (regdiag 3 2)) $ Set.singleton (VarInt 6))] 

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(1,1),(5,1)},1 % 1)"
"({(1,2),(5,2)},1 % 1)"
"({(1,3),(5,3)},1 % 1)"
"({(2,1),(6,1)},1 % 1)"
"({(2,2),(6,2)},1 % 1)"
"({(2,3),(6,3)},1 % 1)"

let aa = resize 100 $ regpivot 3 2

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcstupinit aa (ind aa) ff (3^4)
"(0,(0.0,{1,5}))"
"(1,(0.0,{2,6}))"
"(2,(43.39417647346198,{1,6}))"
"(3,(43.39417647346198,{2,5}))"
"(4,(43.39417647346198,{5,6}))"

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderinit aa (ind aa) ff (3^4)
"(0,(14.464725491153994,{5,6}))"

let (a,kk) = (sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcstupinit aa (ind aa) ff (3^4)) !! 4

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcstupneigh aa (ind aa) kk a ff (3^4)
"(0,(43.39417647346198,{1,5,6}))"
"(1,(43.39417647346198,{2,5,6}))"

let (b,jj) = (sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcstupneigh aa (ind aa) kk a ff (3^4)) !! 1

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcstupneigh aa (ind aa) jj b ff (3^4)
"(0,(43.39417647346198,{1,2,5,6}))"

let (a,kk) = (sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderinit aa (ind aa) ff (3^4)) !! 0

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderneigh aa (ind aa) kk a ff (3^4)

let aa = resize 100 $ regpivot 3 3

let ff = last $ zzllfuds aa (ind aa) 1 (3^3) 3 3 3 (3^3) 1

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(1,1),(64,1)},1 % 1)"
"({(1,2),(64,2)},1 % 1)"
"({(1,3),(64,2)},1 % 1)"
"({(2,1),(65,1)},1 % 1)"
"({(2,2),(65,2)},1 % 1)"
"({(2,3),(65,2)},1 % 1)"
"({(3,1),(66,1)},1 % 1)"
"({(3,2),(66,2)},1 % 1)"
"({(3,3),(66,2)},1 % 1)"

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderinit aa (ind aa) ff (3^3)
"(0,(15.918481590327701,{64,65}))"
"(1,(15.918481590327701,{64,66}))"
"(2,(15.918481590327701,{65,66}))"

let (a,kk) = (sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderinit aa (ind aa) ff (3^3)) !! 0

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderneigh aa (ind aa) kk a ff (3^3)
"(0,(31.253377751394225,{64,65,66}))"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^3) 3
"(0,{({64,65},15.918481590327701),({64,66},15.918481590327701),({65,66},15.918481590327701)})"
"(1,{({64,65,66},31.253377751394225)})"
"(2,{})"

let ff = last $ zzllfuds aa (ind aa) 2 (3^3) 2 2 2 (3^3) 1

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(1,1),(3,1),(328,1)},1 % 1)"
"({(1,1),(3,2),(328,1)},1 % 1)"
"({(1,1),(3,3),(328,1)},1 % 1)"
"({(1,2),(3,1),(328,1)},1 % 1)"
"({(1,2),(3,2),(328,2)},1 % 1)"
"({(1,2),(3,3),(328,2)},1 % 1)"
"({(1,3),(3,1),(328,1)},1 % 1)"
"({(1,3),(3,2),(328,2)},1 % 1)"
"({(1,3),(3,3),(328,2)},1 % 1)"
"({(1,1),(439,1)},1 % 1)"
"({(1,2),(439,2)},1 % 1)"
"({(1,3),(439,2)},1 % 1)"
"({(2,1),(329,1)},1 % 1)"
"({(2,2),(329,2)},1 % 1)"
"({(2,3),(329,2)},1 % 1)"
"({(3,1),(329,1),(438,1)},1 % 1)"
"({(3,1),(329,2),(438,1)},1 % 1)"
"({(3,2),(329,1),(438,1)},1 % 1)"
"({(3,2),(329,2),(438,2)},1 % 1)"
"({(3,3),(329,1),(438,1)},1 % 1)"
"({(3,3),(329,2),(438,2)},1 % 1)"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^3) 2
"(0,{({328,329},15.918481590327701),({329,439},15.918481590327701)})"
"(1,{({328,329,439},22.1155308168681)})"
"(2,{})"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^3) 3
"(0,{({328,329},15.918481590327701),({329,439},15.918481590327701),({438,439},15.918481590327701)})"
"(1,{({328,329,439},22.1155308168681),({328,438,439},13.87386906972094)})"
"(2,{})"

Now consider predicting the rain in the weather forecast example (summarised in Functional definition sets),

let aa = hhaa hh

let ff = last $ zzllfuds (aa `red` [pressure,cloud,wind]) (ind (aa `red` [pressure,cloud,wind])) 1 (3^3) 3 3 2 (3^3) 1

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(cloud,heavy),(296,1)},1 % 1)"
"({(cloud,light),(296,2)},1 % 1)"
"({(cloud,none),(296,2)},1 % 1)"
"({(pressure,high),(wind,light),(297,1)},1 % 1)"
"({(pressure,high),(wind,none),(297,1)},1 % 1)"
"({(pressure,high),(wind,strong),(297,1)},1 % 1)"
"({(pressure,low),(wind,light),(297,2)},1 % 1)"
"({(pressure,low),(wind,none),(297,1)},1 % 1)"
"({(pressure,low),(wind,strong),(297,2)},1 % 1)"
"({(pressure,medium),(wind,light),(297,1)},1 % 1)"
"({(pressure,medium),(wind,none),(297,1)},1 % 1)"
"({(pressure,medium),(wind,strong),(297,1)},1 % 1)"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^3) 3
"(0,{({296,297},3.0601487103147385)})"
"(1,{})"

let ff = last $ zzllfuds (aa `red` [pressure,cloud,wind]) (ind (aa `red` [pressure,cloud,wind])) 1 (3^3) 3 (3*2) 3 (3^3) 2

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(cloud,heavy),(197,1)},1 % 1)"
"({(cloud,light),(197,2)},1 % 1)"
"({(cloud,none),(197,2)},1 % 1)"
"({(pressure,high),(196,1)},1 % 1)"
"({(pressure,low),(196,2)},1 % 1)"
"({(pressure,medium),(196,1)},1 % 1)"
"({(wind,light),(161,1)},1 % 1)"
"({(wind,none),(161,1)},1 % 1)"
"({(wind,strong),(161,2)},1 % 1)"
"({(wind,light),(171,1)},1 % 1)"
"({(wind,none),(171,2)},1 % 1)"
"({(wind,strong),(171,1)},1 % 1)"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^3) 3
"(0,{({161,197},0.6057834603672152),({171,197},0.5399739167361197),({196,197},2.4883225433699216)})"
"(1,{({161,171,197},0.8225606143847344),({161,196,197},2.751189151274419),({171,196,197},2.7587225646757725)})"
"(2,{({161,171,196,197},2.7398023210803686)})"
"(3,{})"

let ff = last $ zzllfuds (aa `red` [pressure,cloud,wind]) (ind (aa `red` [pressure,cloud,wind])) 1 (3^3) 3 (3*2) 2 (3^3) 2

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(cloud,heavy),(607,1)},1 % 1)"
"({(cloud,light),(607,2)},1 % 1)"
"({(cloud,none),(607,2)},1 % 1)"
"({(cloud,heavy),(614,1)},1 % 1)"
"({(cloud,light),(614,2)},1 % 1)"
"({(cloud,none),(614,3)},1 % 1)"
"({(cloud,heavy),(631,1)},1 % 1)"
"({(cloud,light),(631,1)},1 % 1)"
"({(cloud,none),(631,2)},1 % 1)"
"({(pressure,high),(wind,light),(577,1)},1 % 1)"
"({(pressure,high),(wind,none),(577,1)},1 % 1)"
"({(pressure,high),(wind,strong),(577,1)},1 % 1)"
"({(pressure,low),(wind,light),(577,2)},1 % 1)"
"({(pressure,low),(wind,none),(577,1)},1 % 1)"
"({(pressure,low),(wind,strong),(577,2)},1 % 1)"
"({(pressure,medium),(wind,light),(577,1)},1 % 1)"
"({(pressure,medium),(wind,none),(577,1)},1 % 1)"
"({(pressure,medium),(wind,strong),(577,1)},1 % 1)"
"({(pressure,high),(606,1)},1 % 1)"
"({(pressure,low),(606,2)},1 % 1)"
"({(pressure,medium),(606,1)},1 % 1)"
"({(wind,light),(615,1)},1 % 1)"
"({(wind,none),(615,2)},1 % 1)"
"({(wind,strong),(615,3)},1 % 1)"
"({(wind,light),(630,1)},1 % 1)"
"({(wind,none),(630,1)},1 % 1)"
"({(wind,strong),(630,2)},1 % 1)"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^3) 3
"(0,{({577,607},3.0601487103147385),({577,614},2.3923266698540573),({606,607},2.4883225433699216)})"
"(1,{({577,606,607},3.891346416043536),({577,606,614},3.358279087934026),({577,607,630},3.1643965626442903)})"
"(2,{({577,606,607,630},3.8511125582823595),({577,606,607,631},3.8213440591293804),({577,606,614,630},3.8286774042639347)})"
"(3,{})"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^7) 3
"(0,{({577,607},3.0601487103147385),({577,614},2.3923266698540573),({606,607},2.4883225433699216)})"
"(1,{({577,606,607},3.891346416043536),({577,606,614},3.358279087934026),({577,607,630},3.1643965626442903)})"
"(2,{({577,606,607,630},3.8511125582823595),({577,606,607,631},3.8213440591293804),({577,606,614,630},3.8286774042639347)})"
"(3,{({577,606,607,614,630},3.8388619125562546),({577,606,607,630,631},4.215798138230515),({577,606,614,630,631},3.810727551110596)})"
"(4,{({577,606,607,614,615,630},3.01163366978457),({577,606,607,614,630,631},3.7980303142856853),({577,606,607,615,630,631},3.2658375420150483)})"
"(5,{({577,606,607,614,615,630,631},3.001766092132741)})"
"(6,{})"

The weather forecast example continues in Practicable shuffle content alignment valency-density fud inducer.

Highest-layer limited-derived derived variables set list maximiser

Define the highest-layer limited-derived derived variables set list maximiser \[ Z_{P,A,A_R,F,\mathrm{D,d}} = \mathrm{maximiseLister}(X_{P,A,A_R,F,\mathrm{D}},P_{P,A,A_R,F,\mathrm{D}},\mathrm{top}(\mathrm{omax}),R_{P,A,A_R,F,\mathrm{D,d}}) \] where the initial subset is (Text) \[ \begin{eqnarray} R_{P,A,A_R,F,\mathrm{D,d}} &=& \{(J,X_{P,A,A_R,F,\mathrm{D}}(J)) : \\ &&\hspace{1em}w \in \mathrm{der}(F),~u \in \mathrm{vars}(F) \setminus V_A \setminus \mathrm{vars}(\mathrm{depends}(F,\{w\})),\\ &&\hspace{2em}J = \{w,u\},~|J^{\mathrm{C}}| \leq \mathrm{wmax}\} \end{eqnarray} \]

The optimised limited-model fuds are (Text) \[ \begin{eqnarray} &&\{\mathrm{depends}(F_{\mathrm{L}},K) : \\ &&\hspace{5em}\{F_{\mathrm{L}}\} = \mathrm{leaves}(\mathrm{tree}(Z_{P,A,A_R,\mathrm{L,d}})),\\ &&\hspace{5em}K \in \mathrm{maxd}(\mathrm{elements}(Z_{P,A,A_R,F_{\mathrm{L}},\mathrm{D,d}}))\} \subset \mathcal{F}_{\infty,U_A,V_A} \cap \mathcal{F}_{\mathrm{q}} \end{eqnarray} \]

The initial function is defined in module AlignmentPracticable,

parametersSystemsSamplesShufflesFudsFunctionInitialDerivedHighest :: 
  Integer -> System -> Histogram -> Histogram -> Fud -> Maybe (Map.Map (Set.Set Variable) Double)

parametersSystemsSamplesShufflesFudsFunctionInitialDerivedHighest wmax uu aa aarr ff
  ...
  | ... = Just $ llmm 
      [(jj, (algn (aa `fmul` gg) - algn (aarr `fmul` gg)) / (x' ** (1/m))) |
        w <- qqll (fder ff), u <- qqll (fvars ff `minus` vars aa `minus` fvars (depends ff (sgl w))), u /= w,  
        let jj = llqq [w,u], let x = vol uu jj, x <= wmax, 
        let gg = depends ff jj, fder gg == jj,
        let x' = fromIntegral x, let m = fromIntegral (Set.size jj)]
  | otherwise = Nothing
  where
    ...

The optimiser function is defined in module AlignmentPracticable,

parametersSystemsSamplesShufflesFudsFunctionOptimiserDerivedHighest :: 
  Integer -> Integer -> System -> Histogram -> Histogram -> Fud ->  
  Maybe [(Map.Map (Set.Set Variable) Double, Map.Map (Set.Set Variable) Double)]

parametersSystemsSamplesShufflesFudsFunctionOptimiserDerivedHighest wmax omax uu aa aarr ff
  ...
  | ... = Just $
      opt (zzcsdderneigh wmax uu aa aarr ff) (topff omax ff) (zzcsdderinit wmax uu aa aarr ff)
  | otherwise = Nothing
  where
    opt pp ii rr = let qq = ii rr in (qq, rr) : ls qq
      where
        ls yy = if isempty yy then [] else let nn = pp yy; qq = ii nn in (qq, nn) : ls qq
    zzcsdderinit wmax uu aa aarr ff =  fromJust $
      parametersSystemsSamplesShufflesFudsFunctionInitialDerivedHighest wmax uu aa aarr ff
    zzcsdderneigh wmax uu aa aarr ff bb = fromJust $
      parametersSystemsSamplesShufflesFudsFunctionNeighbourhoodDerived wmax uu aa aarr ff bb
    ...

For example,

let zzffcstupinit aa rr ff xmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionInitialTuple xmax (sys aa `uunion` fsys ff) aa rr ff

let zzffcsdderinit aa rr ff wmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionInitialDerived wmax (sys aa `uunion` fsys ff) aa rr ff

let zzffcsdderhighinit aa rr ff wmax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionInitialDerivedHighest wmax (sys aa `uunion` fsys ff) aa rr ff

let zzffcsdderopt aa rr ff wmax omax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionOptimiserDerived wmax omax (sys aa `uunion` fsys ff) aa rr ff

let zzffcsdderhighopt aa rr ff wmax omax = fromJust $ parametersSystemsSamplesShufflesFudsFunctionOptimiserDerivedHighest wmax omax (sys aa `uunion` fsys ff) aa rr ff

let ff = llff [cdtt [1,5] [[1,1],[2,2],[3,2]], cdtt [2,6] [[1,1],[2,2],[3,2]], cdtt [3,7] [[1,1],[2,2],[3,2]], cdtt [4,8] [[1,1],[2,2],[3,2]], cdtt [5,6,9] [[1,1,1],[1,2,2],[2,1,2],[2,2,2]], cdtt [7,8,10] [[1,1,1],[1,2,2],[2,1,2],[2,2,2]]] 

rpln $ (concat . map (aall . ttaa) . Set.toList . ffqq) $ ff
"({(1,1),(5,1)},1 % 1)"
"({(1,2),(5,2)},1 % 1)"
"({(1,3),(5,2)},1 % 1)"
"({(2,1),(6,1)},1 % 1)"
"({(2,2),(6,2)},1 % 1)"
"({(2,3),(6,2)},1 % 1)"
"({(3,1),(7,1)},1 % 1)"
"({(3,2),(7,2)},1 % 1)"
"({(3,3),(7,2)},1 % 1)"
"({(4,1),(8,1)},1 % 1)"
"({(4,2),(8,2)},1 % 1)"
"({(4,3),(8,2)},1 % 1)"
"({(5,1),(6,1),(9,1)},1 % 1)"
"({(5,1),(6,2),(9,2)},1 % 1)"
"({(5,2),(6,1),(9,2)},1 % 1)"
"({(5,2),(6,2),(9,2)},1 % 1)"
"({(7,1),(8,1),(10,1)},1 % 1)"
"({(7,1),(8,2),(10,2)},1 % 1)"
"({(7,2),(8,1),(10,2)},1 % 1)"
"({(7,2),(8,2),(10,2)},1 % 1)"

let bb = regtranspose [9,10] $ regdiag 2 2 `add` resize 63 (regcart 1 2)

let aa = resize 1000 $ apply (vars (regcart 3 4)) (vars (regcart 3 4)) (bb `Set.insert` fhis ff) (regcart 3 4)

rpln $ aall $ aa `fapply` ff
"({(9,1),(10,1)},500 % 1)"
"({(9,2),(10,2)},500 % 1)"

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcstupinit aa (ind aa) ff (3^4)
"(0,(124.58189767609292,{1,9}))"
"(1,(124.58189767609292,{2,9}))"
"(2,(124.58189767609292,{3,10}))"
"(3,(124.58189767609292,{4,10}))"
"(4,(124.6062808304041,{5,9}))"
"(5,(124.6062808304041,{6,9}))"
"(6,(124.6062808304041,{7,10}))"
"(7,(124.6062808304041,{8,10}))"
"(8,(374.48479409127594,{1,10}))"
"(9,(374.48479409127594,{2,10}))"
"(10,(374.48479409127594,{3,9}))"
"(11,(374.48479409127594,{4,9}))"
"(12,(376.9829012644941,{5,10}))"
"(13,(376.9829012644941,{6,10}))"
"(14,(376.9829012644941,{7,9}))"
"(15,(376.9829012644941,{8,9}))"
"(16,(686.4799897569001,{9,10}))"

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderinit aa (ind aa) ff (3^4)
"(0,(54.95086504139181,{5,6}))"
"(1,(54.95086504139181,{7,8}))"
"(2,(93.039559074919,{5,7}))"
"(3,(93.039559074919,{5,8}))"
"(4,(93.039559074919,{6,7}))"
"(5,(93.039559074919,{6,8}))"
"(6,(188.49145063224705,{5,10}))"
"(7,(188.49145063224705,{6,10}))"
"(8,(188.49145063224705,{7,9}))"
"(9,(188.49145063224705,{8,9}))"
"(10,(343.23999487845003,{9,10}))"

rpln $ zip [0.. ] $ sort $ map (\(a,b) -> (b,a)) $ Map.toList $ zzffcsdderhighinit aa (ind aa) ff (3^4)
"(0,(188.49145063224705,{5,10}))"
"(1,(188.49145063224705,{6,10}))"
"(2,(188.49145063224705,{7,9}))"
"(3,(188.49145063224705,{8,9}))"
"(4,(343.23999487845003,{9,10}))"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderopt aa (ind aa) ff (3^4) 10
"(0,{({5,7},93.039559074919),({5,8},93.039559074919),({5,10},188.49145063224705),({6,7},93.039559074919),({6,8},93.039559074919),({6,10},188.49145063224705),({7,8},54.95086504139181),({7,9},188.49145063224705),({8,9},188.49145063224705),({9,10},343.23999487845003)})"
"(1,{({5,6,7},243.22331215903455),({5,6,8},243.22331215903455),({5,6,10},395.6136746748716),({5,7,8},243.2233121590341),({6,7,8},243.2233121590341),({7,8,9},395.6136746748716)})"
"(2,{({5,6,7,8},448.24385179586693)})"
"(3,{})"

rpln $ zip [0.. ] $ fst $ unzip $ zzffcsdderhighopt aa (ind aa) ff (3^4) 10
"(0,{({5,10},188.49145063224705),({6,10},188.49145063224705),({7,9},188.49145063224705),({8,9},188.49145063224705),({9,10},343.23999487845003)})"
"(1,{({5,6,10},395.6136746748716),({7,8,9},395.6136746748716)})"
"(2,{})"

Highest-layer limited-derived derived variables set builder

To implement the highest-layer limited-derived derived variables set list maximiser, $Z_{P,A,A_R,F,\mathrm{D,d}}$, (Haskell) define the highest-layer limited-derived derived variables set builder $I_{P,U,\mathrm{D,d}} \in \mathrm{computers}$ such that (i) the domain is $\mathrm{domain}(I_{P,U,\mathrm{D,d}}) = \mathrm{P}(\mathcal{V}_U) \times \mathcal{F}_{U,1} \times \mathcal{A}_U \times \mathcal{A}_U$, (ii) the range is $\mathrm{range}(I_{P,U,\mathrm{D,d}}) = (\mathrm{P}(\mathcal{V}_U) \times \mathcal{A}_U \times \mathcal{A}_U) \to \mathbf{Q}$, and (iii) the application is (Text) \[ \begin{eqnarray} I_{P,U,\mathrm{D,d}}^{ * }((V,F,X,X_R)) = \mathrm{maxd}(\mathrm{buildd}(\mathrm{vars}(F) \setminus V,X,X_R,\mathrm{init}(\mathrm{der}(F)),\emptyset)) \end{eqnarray} \] where $\mathrm{init}(V) := \{(((\{w\},\emptyset,\emptyset),0),0) : w \in V\}$, $\mathrm{buildt} = ((\mathrm{P}(\mathcal{V}_U) \times \mathcal{A}_U \times \mathcal{A}_U) \times \mathbf{Q}) \times (\mathbf{Q} \times \mathbf{N} \times \mathbf{Q} \times \mathbf{N} \times \mathbf{N})$ and $\mathrm{buildd} \in \mathrm{P}(\mathcal{V}_U) \times \mathcal{A}_U \times \mathcal{A}_U \times \mathrm{buildt} \times \mathrm{buildt} \to \mathrm{buildt}$ is defined \[ \begin{eqnarray} &&\mathrm{buildd}(W,X,X_R,Q,N) = \\ &&\hspace{1em}\mathrm{if}(M \neq \emptyset, \mathrm{buildb}(W,X,X_R,M,N \cup M), \mathrm{final}(N)) : \\ &&\hspace{2em}P = \{J : (((K,\cdot,\cdot),\cdot),\cdot) \in Q,~w \in W \setminus K,~J = K \cup \{w\}\}, \\ &&\hspace{2em}M = \mathrm{top}(\mathrm{omax})(\{(((J,B,B_R),(a-b)/c), \\ &&\hspace{10em}((a-b)/c,~-l,~-b/c,~-u,~D_{\mathcal{X}}(J))) :\\ &&\hspace{4em}J \in P,~u = |J^{\mathrm{C}}|,~u \leq \mathrm{wmax},~\mathrm{der}(\mathrm{depends}(F,J))=J,\\ &&\hspace{4em}m = |J|,~l = \mathrm{sumlayer}(F,J),\\ &&\hspace{4em}B = I_{\%}^{ * }((J,X)),~B_R = I_{\%}^{ * }((J,X_R)),\\ &&\hspace{4em}a = I_{\mathrm{a}}^{ * }(B),~b = I_{\mathrm{a}}^{ * }(B_R),~c=I_{\approx\mathrm{pow}}^{ * }((u,1/m))\}) \end{eqnarray} \] where the power approxer $I_{\approx\mathrm{pow}} \in \mathrm{computers}$ is such that $I_{\approx\mathrm{pow}}^{ * }((x,y)) \approx x^y$, and the alignmenter $I_{\mathrm{a}} = \mathrm{alignmenter} \in \mathrm{computers}$ is such that $I_{\mathrm{a}}^{ * }(A) \approx \mathrm{algn}(A)$.

The derived set builder is defined in module AlignmentPracticable,

parametersSystemsBuilderDerivedVarsHighest :: 
  Integer -> Integer -> System -> Set.Set Variable -> Fud -> Histogram -> Histogram ->  
  Maybe (Map.Map (Set.Set Variable, Histogram, Histogram) Double)

parametersSystemsBuilderDerivedVarsHighest wmax omax uu vv ff xx xxrr
  ...
  | otherwise = Just $ maxfst $ buildd (fvars ff `minus` vv) (init (fder ff)) Map.empty
  where
    init vv = llmm [((sgl w, histogramEmpty, histogramEmpty),(0,0,0,0)) | w <- qqll vv]
    buildd ww qq nn = if mm /= Map.empty then buildd ww mm (nn `Map.union` mm) else (final nn) 
      where
        pp = llqq [jj | ((kk,_,_),_) <- mmll qq, w <- qqll (ww `minus` kk), let jj = kk `add` w]
        mm = top omax $ llmm [((jj, bb, bbrr), ((a-b)/c,-l,-b/c,-u)) |
          jj <- qqll pp, let u = vol uu jj, u <= wmax, fder (depends ff jj) == jj,
          let l =  sumlayer ff jj, 
          let bb = xx `red` jj, let bbrr = xxrr `red` jj,
          let u' = fromIntegral u, let m = fromIntegral (Set.size jj),
          let a = algn bb, let b = algn bbrr, let c = u' ** (1/m)]
    final nn = llmm [((kk,aa,bb),a) | ((kk,aa,bb),a) <- mmll nn, card kk > 1]
    ...

For example,

let aa = resize 1000 $ regpivot 3 4

let ff = llff [cdtt [1,15] [[1,1],[2,2],[3,3]], cdtt [2,16] [[1,1],[2,2],[3,2]], cdtt [3,17] [[1,1],[2,2],[3,3]], cdtt [4,18] [[1,1],[2,2],[3,2]]] 

let zzffcsdderhighopt aa rr ff wmax omax = fst $ unzip $ fromJust $ parametersSystemsSamplesShufflesFudsFunctionOptimiserDerivedHighest wmax omax (sys aa `uunion` fsys ff) aa rr ff

let buildffdervar aa rr ff wmax omax = let vv = vars aa; ww = vv `Set.union` fvars ff; xx = apply vv ww (fhis ff) aa; xxrr = apply vv ww (fhis ff) rr in map (\((kk,_,_),a) -> (kk,a)) $ Map.toList $ fromJust $ parametersSystemsBuilderDerivedVarsHighest wmax omax (sys aa `uunion` fsys ff) vv ff xx xxrr

rpln $ zzffcsdderhighopt aa (ind aa) ff (3^4) 3
"{({16,17},88.62278953098446),({16,18},109.64379180618926),({17,18},88.62278953098446)}"
"{({15,16,18},188.4625708697548),({15,17,18},162.07731482923498),({16,17,18},188.4625708697548)}"
"{({15,16,17,18},257.3608026114683)}"
"{}"

rpln $ buildffdervar aa (ind aa) ff (3^4) 3
"({15,16,17,18},257.3608026114683)"

No-sum-layer highest-layer limited-derived derived variables set builder

Similarly to the tuple builder above, some implementations may drop the computation of $\mathrm{sumlayer}(F,J)$, especially if the layerer is subject to the excluded-self restriction. The no-sum-layer highest-layer limited-derived derived variables set builder $I_{P,U,\mathrm{D,d,ns}} \in \mathrm{computers}$ is defined in module AlignmentPracticable,

parametersSystemsBuilderDerivedVarsHighestNoSumlayer :: 
  Integer -> Integer -> System -> Set.Set Variable -> Fud -> Histogram -> Histogram ->  
  Maybe (Map.Map (Set.Set Variable, Histogram, Histogram) Double)

parametersSystemsBuilderDerivedVarsHighestNoSumlayer wmax omax uu vv ff xx xxrr
  ...
  | otherwise = Just $ maxfst $ buildd (fvars ff `minus` vv) (init (fder ff)) Map.empty
  where
    init vv = llmm [((sgl w, histogramEmpty, histogramEmpty),(0,0,0)) | w <- qqll vv]
    buildd ww qq nn = if mm /= Map.empty then buildd ww mm (nn `Map.union` mm) else (final nn) 
      where
        pp = llqq [jj | ((kk,_,_),_) <- mmll qq, w <- qqll (ww `minus` kk), let jj = kk `add` w]
        mm = top omax $ llmm [((jj, bb, bbrr), ((a-b)/c,-b/c,-u)) |
          jj <- qqll pp, let u = vol uu jj, u <= wmax, fder (depends ff jj) == jj,
          let bb = xx `red` jj, let bbrr = xxrr `red` jj,
          let u' = fromIntegral u, let m = fromIntegral (Set.size jj),
          let a = algn bb, let b = algn bbrr, let c = u' ** (1/m)]
    final nn = llmm [((kk,aa,bb),a) | ((kk,aa,bb),a) <- mmll nn, card kk > 1]
    ...

Level limited-derived derived variables set list maximiser

Define the level limited-derived derived variables set list maximiser \[ Z_{P,A,A_R,F_{\mathrm{g}},F,\mathrm{D}} = \mathrm{maximiseLister}(X_{P,A,A_R,F,\mathrm{D}},P_{P,A,A_R,F_{\mathrm{g}},F,\mathrm{D}},\mathrm{top}(\mathrm{omax}),R_{P,A,A_R,F_{\mathrm{g}},F,\mathrm{D}}) \]

The neighbourhood function is (Text) \[ \begin{eqnarray} &&P_{P,A,A_R,F_{\mathrm{g}},F,\mathrm{D}}(D) = \{(J,X_{P,A,A_R,F,\mathrm{D}}(J)) : \\ &&\hspace{2em}(K,\cdot) \in D,~w \in \mathrm{vars}(F) \setminus V_A \setminus \mathrm{vars}(F_{\mathrm{g}}) \setminus K,\\ &&\hspace{2em}J = K \cup \{w\},~|J^{\mathrm{C}}| \leq \mathrm{wmax},~\mathrm{der}(\mathrm{depends}(F,J)) = J\} \end{eqnarray} \] The neighbourhood function is defined in module AlignmentPracticable,

parametersSystemsSamplesShufflesLevelsFudsFunctionNeighbourhoodDerived :: 
  Integer -> System -> Histogram -> Histogram -> Fud -> Fud -> Map.Map (Set.Set Variable) Double -> 
  Maybe (Map.Map (Set.Set Variable) Double)

parametersSystemsSamplesShufflesLevelsFudsFunctionNeighbourhoodDerived wmax uu aa aarr ffg ff bb
  ...
  | ... = Just $ llmm 
      [(jj, (algn (aa `fmul` gg) - algn (aarr `fmul` gg)) / (x' ** (1/m))) |
        kk <- keys bb, w <- qqll (qq `minus` kk),  
        let jj = kk `add` w, let x = vol uu jj, x <= wmax, 
        let gg = depends ff jj, fder gg == jj,
        let x' = fromIntegral x, let m = fromIntegral (Set.size jj)]
  | otherwise = Nothing
  where
    qq = fvars ff `minus` vars aa `minus` fvars ffg
    ...

The initial subset is (Text) \[ \begin{eqnarray} &&R_{P,A,A_R,F_{\mathrm{g}},F,\mathrm{D}} = \{(J,X_{P,A,A_R,F,\mathrm{D}}(J)) : \\ &&\hspace{2em}w,u \in \mathrm{vars}(F) \setminus V_A \setminus \mathrm{vars}(F_{\mathrm{g}}),~u \neq w,\\ &&\hspace{2em}J = \{w,u\},~|J^{\mathrm{C}}| \leq \mathrm{wmax},~\mathrm{der}(\mathrm{depends}(F,J)) = J\} \end{eqnarray} \] The initial function is defined in module AlignmentPracticable. This implementation is derived from the highest-layer limited-derived derived variables set list maximiser,

parametersSystemsSamplesShufflesLevelsFudsFunctionInitialDerivedHighest :: 
  Integer -> System -> Histogram -> Histogram -> Fud -> Fud -> Maybe (Map.Map (Set.Set Variable) Double)

parametersSystemsSamplesShufflesLevelsFudsFunctionInitialDerivedHighest wmax uu aa aarr ffg ff
  ...
  | ... = Just $ llmm 
      [(jj, (algn (aa `fmul` gg) - algn (aarr `fmul` gg)) / (x' ** (1/m))) |
        w <- qqll (fder ff), 
        u <- qqll (fvars ff `minus` vars aa `minus` fvars ffg `minus` fvars (depends ff (sgl w))), u /= w,  
        let jj = llqq [w,u], let x = vol uu jj, x <= wmax, 
        let gg = depends ff jj, fder gg == jj,
        let x' = fromIntegral x, let m = fromIntegral (Set.size jj)]
  | otherwise = Nothing
  where
    ...

The optimised limited-model fuds are (Text) \[ \begin{eqnarray} &&\{\mathrm{depends}(F_{\mathrm{L}},K) : \\ &&\hspace{5em}\{F_{\mathrm{L}}\} = \mathrm{leaves}(\mathrm{tree}(Z_{P,A,A_R,V_{\mathrm{g}},F_{\mathrm{g}},\mathrm{L}})),\\ &&\hspace{5em}K \in \mathrm{maxd}(\mathrm{elements}(Z_{P,A,A_R,F_{\mathrm{g}},F_{\mathrm{L}},\mathrm{D}}))\} \end{eqnarray} \] The optimiser function is defined in module AlignmentPracticable. This implementation is derived from the highest-layer limited-derived derived variables set list maximiser,

parametersSystemsSamplesShufflesLevelsFudsFunctionOptimiserDerivedHighest :: 
  Integer -> Integer -> System -> Histogram -> Histogram -> Fud ->  Fud ->  
  Maybe [(Map.Map (Set.Set Variable) Double, Map.Map (Set.Set Variable) Double)]

parametersSystemsSamplesShufflesLevelsFudsFunctionOptimiserDerivedHighest wmax omax uu aa aarr ffg ff
  ...
  | ... = Just $
      opt (zzcsdderneigh wmax uu aa aarr ff) (topff omax ff) (zzcsdderinit wmax uu aa aarr ff)
  | otherwise = Nothing
  where
    opt pp ii rr = let qq = ii rr in (qq, rr) : ls qq
      where
        ls yy = if isempty yy then [] else let nn = pp yy; qq = ii nn in (qq, nn) : ls qq
    zzcsdderinit wmax uu aa aarr ff =  fromJust $
      parametersSystemsSamplesShufflesLevelsFudsFunctionInitialDerivedHighest wmax uu aa aarr ffg ff
    zzcsdderneigh wmax uu aa aarr ff bb = fromJust $
      parametersSystemsSamplesShufflesLevelsFudsFunctionNeighbourhoodDerived wmax uu aa aarr ffg ff bb
    ...

Level no-sum-layer highest-layer limited-derived derived variables set builder

The no-sum-layer highest-layer limited-derived derived variables set builder $I_{P,U,\mathrm{D,d,ns,l}} \in \mathrm{computers}$ is defined in module AlignmentPracticable,

parametersSystemsBuilderDerivedVarsLevelHighestNoSumlayer :: 
  Integer -> Integer -> System -> Set.Set Variable -> Fud -> Fud -> Histogram -> Histogram ->  
  Maybe (Map.Map (Set.Set Variable, Histogram, Histogram) Double)

parametersSystemsBuilderDerivedVarsLevelHighestNoSumlayer wmax omax uu vv ffg ff xx xxrr
  ...
  | otherwise = Just $ maxfst $ buildd (fvars ff `minus` vv `minus` fvars ffg) (init (fder ff)) Map.empty
  where
    init vv = llmm [((sgl w, histogramEmpty, histogramEmpty),(0,0,0)) | w <- qqll vv]
    buildd ww qq nn = if mm /= Map.empty then buildd ww mm (nn `Map.union` mm) else (final nn) 
      where
        pp = llqq [jj | ((kk,_,_),_) <- mmll qq, w <- qqll (ww `minus` kk), let jj = kk `add` w]
        mm = top omax $ llmm [((jj, bb, bbrr), ((a-b)/c,-b/c,-u)) |
          jj <- qqll pp, let u = vol uu jj, u <= wmax, fder (depends ff jj) == jj,
          let bb = xx `red` jj, let bbrr = xxrr `red` jj,
          let u' = fromIntegral u, let m = fromIntegral (Set.size jj),
          let a = algn bb, let b = algn bbrr, let c = u' ** (1/m)]
    final nn = llmm [((kk,aa,bb),a) | ((kk,aa,bb),a) <- mmll nn, card kk > 1]
	...

top