From a956b9ca792357bb0feeb65eaa67761b23166af8 Mon Sep 17 00:00:00 2001 From: Evan Sheng Date: Mon, 14 Jun 2021 11:47:27 -0700 Subject: [PATCH 1/2] CA proposal --- .../proposals/expander-plugin-grpc.md | 78 ++++++++++++++++++ .../images/external-expander-plugin-grpc.jpg | Bin 0 -> 49858 bytes 2 files changed, 78 insertions(+) create mode 100644 cluster-autoscaler/proposals/expander-plugin-grpc.md create mode 100644 cluster-autoscaler/proposals/images/external-expander-plugin-grpc.jpg diff --git a/cluster-autoscaler/proposals/expander-plugin-grpc.md b/cluster-autoscaler/proposals/expander-plugin-grpc.md new file mode 100644 index 000000000000..21fb29f1745f --- /dev/null +++ b/cluster-autoscaler/proposals/expander-plugin-grpc.md @@ -0,0 +1,78 @@ +# Expander Plugin over gRPC + +Authors: +Evan Sheng (evan.sheng@airbnb.com) @evansheng + +## Context and Rationale + +Cluster Autoscaler makes node scaling decisions based on Expander strategies, to select which node group to add nodes to. +Currently, there are 5 expanders: random, most-pods, least-waste, price, and priority. These provide a variety of behaviors in which CA behaves to expand a cluster. + +However, whenever users would like to implement a customer expander they must either submit a change upstream, or fork CA and maintain their own implementation. +Iteration time would be slow, and oftentimes, users may have a case where they want to implement custom business logic into the expander that may not be relevant to the wider community, but do not want to maintain a fork. + +Namely, we (Airbnb) would like to iterate on custom expander strategies (such as a weighted random expander, custom spot market strategy, etc) out of band with CA. +To do this, we propose a solution that adds a new expander to CA, but does not break backwards compatibility. + + +## Proposal + +We will extend CA to utilize a pluggable external expander. The design for this expander plugin is heavily based off of this [proposal](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/proposals/plugable-provider-grpc.md) to CA, for a pluggable cloud provider interface. + +The solution will include a server acting as an external expander alongside CA, and communicate via gRPC with TLS. This expander will run in another pod, as a separate service, deployed independently of CA. +This is depicted below. + +![Figure](./images/external-expander-plugin-grpc.jpg) + +The gRPC server must implement the API of the [expander strategy interface](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/expander/expander.go#L50) in CA, which only has one method. + + +```go +// Strategy describes an interface for selecting the best option when scaling up +type Strategy interface { +BestOption(options []Option, nodeInfo map[string]*schedulerframework.NodeInfo) *Option +} +``` + +The gRPC service, running as an external pod, is defined below: + +```protobuf +syntax = "proto3"; + +package clusterautoscaler.expander.v1; + +import "google/protobuf"; +import "k8s.io/autoscaler/cluster-autoscaler/cloudprovider/generated.proto"; +import "k8s.io/api/core/v1/generated.proto"; + +option go_package = "v1"; + +service Expander { + rpc BestOption (BestOptionRequest) + returns (BestOptionResponse) {} +} +``` +The messages used in these calls are also defined: +```protobuf +message BestOptionRequest { + repeated Option options = 1; + map nodeInfo = 2; +} + +message BestOptionResponse { + Option option = 1; +} + +message Option { + k8s.io.autoscaler.cluster-autoscaler.cloudprovider.NodeGroup nodeGroup = 1; + int32 nodeCount = 2; + string debug = 3; + repeated k8s.io.api.core.v1.Pod pod = 4; +} +``` + +To communicate with the external gRPC server, CA needs new flags to expose details about the server. + +We’ll add a new option to the expander flag: `--expander=externalgrpc`, and inntroduce a new flag `--expander-plugin-url=https://external-grpc-url/server` to reach the gRPC server. + +Additionally, we’ll need to use TLS for secure communication. An additional flag `--external-expander-cert=~/path/to/cert` will be exposed to specify the path to the certificate authority bundle used to validate the TLS cert used by the server. diff --git a/cluster-autoscaler/proposals/images/external-expander-plugin-grpc.jpg b/cluster-autoscaler/proposals/images/external-expander-plugin-grpc.jpg new file mode 100644 index 0000000000000000000000000000000000000000..28cc8f706fd7646d19cdf75deb07930138422b43 GIT binary patch literal 49858 zcmeFZ2_RJc{y#pVP$4^6Mp4L`ELo?`mXIxFn<9j4A%ihf%9;p8sZ2egDKIij(pZDkW-Vgl;eH?V?qMo51 zh=BnFG64R8=s1v%F3imZ1Tr=TodAJAEFdO^eIQ2Q69Z@u@QdsB&%gx?Qhz%h=>}r@ zbq)xm1N;vHF}`8=+nI0n{N>v{MQ<4Y@|nSB=R`V=^~f!Me;*A61usAO8#lcjo#map zJr!U#d=yT~D=L7rATXaBP9DzwVvf$PZeH3)S1K?^#oTUcAGKC7Ry6iG?|jS6Aoz~6 zMX-sbQ?Q4V`pu&d9WgDK2F%mP)7k%q7|ipwm!Aeq`{=L1HGt1MhZT;B{W`_pL;L7e z!~BBY@7Drlh?9lEXHO9$!%@&4 zP6kF!26{aR3|L@$8Gda$cGS+su!oUp?>=T0*8K;76N(Rk_AoFq?qOowyO#;D&KN?0 z??FtQd%2FBJiCwE><05uUmm6V3F$22=L%|h&AadtryTEuvF_*N7Z5yr?6{ggL8UbL{Zx?+9xnvK&-YW~{ty}PHkum8uuAZ}uEYI=6GJmJ6-$vB$RQ-2s@QhTONJz24$f3IPUxQBwalI1g|SYhroUwhPmKUo+vXrYDL zZO#x}9%eifv#yIcOb6i_dy!l5XzM}EgXHT-I}6jNAay(LdrkYx)?1wbX09kd44r>K-tcmiorXe^X+dJs(qo!pq8FZiMvjJCV4 z*Gr^Ck}PN~**_pf?}BzYcYU`p9ni(0m?JbEPpEe@G8{Z$NoFQK?U15_B1!;e$I(Gu zyd)1gC{oCZ4w})4CwX|2q*kb$&+cc2Q7K>njm6?B@mDOWU=%D?ozh)7?_~94(p6*8KE-cxYCtDA#f^okw z&y4!a^b^wU(&`_{ocjk1se-zb5$7pVgTKuJ@;Qgdu_T>l_76nsk8J7BUHZj&p>@S0vScIlH@^Qii~nyte4VQ#9Iz}osnt3l3LgR4gcPK3kdVHaB$*36 zDhqQTtB<;S`yDHJ$vAwmKsJr*@xgHMBa9l-x1I0JU_2gpY5AjTJ4n(o@uWcJB}O{v z=1@auBz4cS$7ZnNUHR@fR7+(g;;^zauV{AqgL;hu&t$2Lm~qWLF;Z{-%)YPigmj=7 zNXc++FG3sUN~BPW5t1XYBx{l(-n&-KL9`2QI-$U3J3K$Ci7BjlQK|W1#7(J9uDCS9 z+nhnRFSex`eosc;_ErDlDSs7G2${Pch5rb*xyOHiU^#ghbWEPt-)`~AS+@{r@1Rn_|`s(*ooBfH8r&dk8@i;Eyg-mvNNK1MEP zJ9*|i8l7fN`z7ltI4V}BqsE`LE4|r}WbQn?fh?3(U5r<8QIUQ5&o91nJbMg&&Xqx) z2tvXkz!Hqy5n<#Xfp2#azW%}RDzWU~<;4>Hi*GU=HzslB5Wyp%8z)Nny;e*cOrF9 zqVGdRE15H3YS#FON4TZCtVLi>Q}mGi;KJtNywCeQUMm>E9MqLdx9%EN8f3@%u|5{| zyw0Mc926Cqg%5Jv_-Xh28?T1ta&oo_H@`-) zED>_GvUZF@dm8kJhNN2Y@=m4|C(t`-9|1&xtXj}P&ST^g)|l-GQd*IY*gSu^>Am7A z3u9lAh7y#Y?4>hD?8P0knZ?HAHc$XW3(`S}D6%{qG+4ZeN1&sa4x$CNK8z|AT9+MrFkTmJrxe5~l=o5c(Z4INYj zd5F2mF)&7~BbZ{*TqoeCNx2gpT%^pp=o^{~s=)}1YC7`}S8tc(7Z1}S{o_X;qi)En z_6=`{m%Yq3o{t|x-N%$})_BK>9wSKJlZ?EEheT7PvmZN|cfp;qtK)-2pAHoS^LZyM zRqKr$)npQ59!i6#V%#Alnav1_DNZ4hA5zE-E5!Pe7AE*3{6{pQz5arW>nKh)e(Ns% zK;GAF-<-LMOnfnor?C>UDZDb19Wh$YWTm;mnFO4@%y?-Gh2MLE&CjT+ zGYdm&OXs2ELMK94%^|tXhjN;VRN)veS#L6Ucu@uipw8nSTbqJlGM6rr7#ULiUfsl}jqBx>Z_3NEx~%dSP|bnLT1K(L<$N`T|j6dxQMQ z#&i%9mUIrWkA~IROY%=U+*>KuSsMWN*LJ6A+nN>d_a_cV&8D(=HtiYzWOzg`-0)J# za9`28X_-?eo=%kLY-YsBr01Eq)Wt|;UH@XH%X3wp$4r;+KsfL_45!Ou_P;s^*9R8T zNjm7M526!7G_C|3op%+f)X_)F@z^bE90!G`iDp(yJ_9T=RRWZjx?{#DltVbC9gcet znmUO}9bpb{d(3o$^xpxu1O1tu`F<)zhZ) zIL<>O#|q4w#|}^&K2g^Iv*eA#FdbCqoHyB5C0y)~gMF@P4pBsozoUan`Inr@;lipOG;SSZJ_%TT4Xv0%C2xpnb|r&(&+=n8OR|IXkRz?dn@2=npKX9fJM8IK zSs}?rVokYfGe_)ioyVb~7mrun5*U&#wJw3h=*bibI9nd6(QInD5 zA5kD03rzETSuX-K;E{HW5csf|Woh~51FYMJxlP|uH*Ukj>JcOISB>1CGX;^)N=yYC zw{s=bJe6cYJ(}=jAInqEY(R}%Z zF}4;p_B{D4C$p~r8n9!@NH{A=Zi0i2bfRU*mM|Mdxt8@h(6~!WB-6&=CWqHu7ew5xF_TeijH5?4ESsH5q0ac@;L4EJ9?R^aoj8N;% z1{2-1?JTNz9Z2S|={%Nf=y^P>PI@+m^}>gv&-E-nkH0^i04+gXCo1Z^rp6-2vuM18 zz*=ZLrdVx>Ve&u(UM*Zl$~~AbQf1>|@@Dd&_dwB1(Pvxb`(6%HBbT}aDzl#({16AL zgy!OziL0cugtdtan0+RXC*m1NShCgvU%A?&^`N4cFGtj&PNrVA@H=~t1%4>=pY;)# z-I^glYF6SQMUsmZ1gD9n2>9Mfg#)c`5j`>l>7oNh=d(y3X3xcZ|LUC9sA?CQ_YmV| z>DiDKdCSGQmMlrMzz1T%v8Ylcb|q%)paMdfqF0FDmy$bTJb{T)vU4kUNYcq>{_*(X zmHt8JqLji*SvJn$2}f@Aomss;lc`?fYX0g<)4H)PkM%jzoEBgiS^bYo7dYv^E%MCw z=;PF6|)C%y9>ahlY4p0&it7j%n8WoCFHa) zxQK(A_fOrE#e_kH zbf3&a2Ypo}ARu!&6u=1TX|+KQ&5|R3LZcuBrj#TOe&m1!DVq+;NiKGHXF>5!sC|xP zZ|awC?=l2Q-i)yfGrH-;F`H07 ztj)0-+$Y7Z)=1o>44e>*n>ZK7Y!3SwAr{6i{q^CIGC9#PxoFK4Z^2>pujXePwe?uu zW<0UqFy&=encLrH*1J)VhNw?qgbaf~~`H){u1VmtphQZ$lVtl!oG)^Q*YK+PYju<1Ina5CiyJ>@U zbdVzQ!;h6|9l~(+*mFAQu$n^Y45YD|oep{krR`UvgLL(ArL;pDKx|LNANgx?LVq_o zNXaM6H}HIk=9Xz?;IGRcw=QkP6VMMZIDSA;JqBWSW+Wi6`H&q5XSUgc@sEIIrAG(N zz26BK)G=EDk-Yr)rH=}0%-eZ!G>a9HU)N{eAtRh99^)&!5U;-rfn<*Zlv-M@6Xq$o z>DQ&Vey53~`3Ks8P%8_D!sAVgiN|_V!~&o+_qqSD1~azgap`_~qoJ;OV_ShKz!3 za_};;Fx-ZeL?|48L5Ob{@ZX}Gb(%2cB_WnQPYl``88$_3&$~m&e*QdT; zaH>h>3w3@wuF9aDE9R+4x7PlmX%jE*E4N7TwEa$WkkmN!4RY}%3=n?fKoxQo{|Gp8 z6i9N=s#bqM2pykZTvSi_+V}F29hZ zzjg_SB1yA8c}B{?kMCh^qKMh#YfRE2%1w7!jt(~qUn$*9(h1W#8JGM)a#Zsdoay~O z(%9|`ewT5KcGhsie-PleZXO^|A}1nqYv&01}WSNsbSX*vzcyD z622I|#(D{@2UmRh@gZQc&3G>8^dg&`oT;moyVShQSBx{qmcdFRntU9?j$B`u+`o~q ztL+p4*!Gu7N!jxaJZ4a>PzHMUzhonc=)%Pb7fvB-^$=VcPLeoGz3|O(SF%N2m2v2> zMhxlvL^LC9zY6|oLIRKh8=K$x{XhKO10RWtS1N-%i!GA^hk1#|oF4n$QoMXsq6TgyWt*r0Ql^x`J(kIVpt3 z8K6MTA{L1M4Evm_oy2sHVuXu1NYd7gn%)ol$4YL3+5F3NKtEIK`^@LrK7OTxSP%Rx=|?)I zSjT-E+|nCxwhnkWyp^|qE8J4p`W0;$RRUg;AxIKksBiP>AXg~!ReP8?Zu!N8!ohis z^}+gNn{TP%Sr4fB%XK4iA{T7@3=Rm&-$>MZT<`I0np{KhEi53v+y6O25X;WG6(n+Y z_-w~JhBn?=DD$$H0)h)>vrN=abt?+l7wA`6$YLSeaQ|4ey5!L+zO1jAu=bytfb(JW zI&Nhr6y*1*Vj1v(gcs$>z)^k81JyeLq4v6e^yNj3n2TmTmZd>QGe#yVFr8a!G}Z|} zG7v*1n$eCU=b^d%!z%a{3OAN!Ott|uVmtw`8qKK=)f4DWJgO!#opeRh&iD!)^aAy- zyHhWA+$qh~e?GLNLc?uaPjD;(PLu?;VT@==lcs~_o%5`h*8YLUwUDN%HRGD-po7~J z)#8Bt3Yf($ry;k%W~YQT`- z%lTa^W{LU-#451kPfaiK9GX=7rI_@3@oWo4m;R&;&N35FNH!D_1(hBtu zmh<^=2|_%Ue2b>?Q%A0gBIJ>NZ9c+S=yG}g(&%!_vi8|Lv+wC|LW<;5 z+}@13IUK-0vK_p&{j`5utM1yyWv{%-AD*wZI<-oz!><~0LSNi1pnBzaX2Psyt#`~V zyTywCL8Nz=cd2=4X?-(u>XLxhTeA41_n_V6#5BOh;V=@>9}b_2>Ri6kK!xA-pf@UvmUuVV&ef5vN>{!4TE?Najbatz@Psy|IKe6DqbV)4FD3n2 z_e;MtNzS{mCN!;>7>LnWrgqkN4)salFAd2{S6}Zk+>PzT}mIJB@dW>6A zxfOz^+lMD5MT>j7&y9NufPxQNd~iO#sljDsOGNjW)Vvhg;63H? z29+ZTGM)xr^&ae}K1D`jwo~ReHhJNiKy)7{KuSXNVHPtfK&-z9V~zfjyM9R$e`F59 zA4dlT<tT48oiO}Ls}5^$l9?{=wCHmH^+A7T@83PZ1TF2@kc4Bz0FfV0*7#c}F(WogK6h79r2CtfB6|h^v z2B34#VqK|8bWnQF#h1R_;_8!q7*+2ph>?q|b!@TD#)rm{y&#C3@)hMExj@k5_oIc^4#aQ3(6)jxVTQ4>6a^{o zjKfLN{kN_1x(yTGz3VKpo+@LG7 z;Jb24w`>tfIY|CBlY{5=*9B6m z_1`Sc-hUO_ju4D!<|_DmGgQi^1_iS&+Ts|O@nUi%r=UXHw zbg~h&oyfcZ-OEjslIX1*n^BFy64avMn&Wv#k%wR%my5s4-fr5qYFNw|mAShREs=EF zqe)y=00>a1tmbC|S{Vq(hQWXG$7j1^M5;w??eu)Qj3z==)hw3td{`*Hme0OrIXtw_ zQVxBr3%90&IF1A72OzsbNxpPYxWh9bm}`l|e#MNxhOR>q!bqHT7IM`}$NC4<28-;d z56$@}t`bIn3iV*4gCLy5EM5YXqVZQy&`DZq`#3l0&^qKm6S5L`y4*UuTB3**-A#QQB^Mw0zlc|z zQXx&w1^%03QoG|m`OfYZbw~j60Yoq8G<2eSLnm(;IRe=hIi}!8`$-3N{(Kw0H_kxx zHPQ~LK^e3y+&sJ=AG69A<8t%b!GmQ_ytXW3Uw`BoFX;@@ju#PP3x}w|$i#MPTs6=9 z_c<_+jvPz@8%Pkz^Zek^r)n`cbk_*cG4a)JwQs@|P6zPl2hMqQ zUo_2%e+=2h=*KH8odl3}ljySJ@xDjSi%$fT=c}7`dsI5yeO2!u4zd)o}S}za_)-iruF5#Z+BH! zEBQH8TFwyKP86A)i$1~ke2;-CGh(#*U~i>G#vGDso$Wksq$GWO0mJ9cMS6>LY0XlJ zOJ{$rlGStPUV44uzH3+XqPsn$r6q261%Y4p<(R)6o|fyVQp0S<&)gcNgGM1Q&?g9} zCP>&Apu0<%>rMIGY`9^L?L(J)`?jZ#ddPST+zcDNb`~P*XLK-!8m+pkC=97rS0-lWbn z+jw#pb!DbJzTsl^q&zIP_A0Zcz7gJ!c#p(QzTj3`zEa~YaEqqcy*`947#Vise=^$S ztsZqz{MmS*Zs^BsuhpPQ$ruH(O>$v>>3=0W9OA+;gsw#v%{F?A!jE>XM;G14sk!-A zY|X}94QLe#X??9avpm+?%!x(1DQgw-naZ0m#S|~mB&hFoKrmZe8>+$HsG;0;F8E!N zU%p$v;^?`bp*P&Tyxt{`l!iaiXW=-`e(2EEd$39Wt{?BT(9e zc6hIX!{ED-g^?6_RVvE1DVU2i!pinWWnDTaOq|8KWj&);G+v+a=*CvAb@Vpb;5^uJgTkcD(3WU2{Gx=WDoUzJ* ztkCKDO~x`q=pY!)rP>ZsqQ5wqX^I20QO*<6!f_}b*!RZ9P1DW*5tC7&*~4M`?y-#Y zMkT~G|G4*Av2KcBKN?TosX{zLDhR@Z&l0L5LndtnZhk?X3nQwlu;8K^1xJU$CqYUI zqnCXMnWG;S&&KKA6pLe#3DY1z50TJB20Z_RK~n)l zBR>!Ts_&YfEbYFm7O`TmRhc_-CE!i%`8edHnu+H=5FS2u_$#8m=&ANKlPfnNN%`})`Dt)P%jmKpjIQlAfAyP{9cU` zNPPy0L2keBu$p&> zF+ZN)R-!W8U;ql#+_v6J(44ToUwsI!(_1Y?(d`lM_GddhvYhWVZ9eaK;0Y(MeVYE{ z?X!lSHEiF`^#_GZL*N<_p!w{N?fiVu9y&n5rJGBIz)au#p@;c`DhI;rU zrz$H+EpQDVc-c@(2SMEg#n6mvZeATf2v;$=H>?~=P658S4&0jh>7yelo(%n(L-MOF zPySkjiBz@r8w@J>rj)p_$+xPn(uVcva1+jT<3=od7rja)7vi#OO&_54u0N+};*oW! zknoRU0&!zT1rd$P9ivCCTbZ{}F6qrIH&#Xx+o&0EeXK7q!sSV-QIcfleB)Md~Nv0k_8{a z=xOKT#M1-v`x7|t2;C+(0+_f4sU*w%hE0!GsFPc%q*?3^)TK{!2wjD1c2%0J|q<4hc zYy*(zu9TSo^IRPe@~F_1b5~%$(BUH+!0P!pujkyEgv!3hJ|FCad1vBb+O$O~%h(Ze z1!3NV8P}oAk6cOE(4vEYvH@Zv^*zOvtO*mU*>1uv@$2>lAcbLx{n(O8`(9i9?i*^V zrPjxTysiu`g+1_|{_s(v%u7*^Ej|KVv|Q?z`C|mp9VmJbzrMX(R4y8?5MB(FG@6!` z!yK$)jLM3Jn8E^oT<|@Oowl-TZWYuX-y~xy$wvQ;At^x2B`k*4xPAR|;%wtZ#~$VP zDkSH)#*J~~qccI{F*LXEvIrqPk06%vpuJ_fqY@{Y+eYWX0M7Tnqg;J7{&Dm;TfR{! z-?MT}qm<9adj5uvg(1CBgIPTV2KmWLQctJgZ$x?=&aP~_cGtdo>G$ehTS8y8 zI$nQilYEqNuA;02-|<-PqRr31a;&|5t)H_aG1!SARMtA>P?M*|wPmo|t(!?-Au{6x z2Tz=$2baCH+eDq>M2Jb9Uh3#4@2HgkD$2-7>h|TsjTI=S8wXy? zw{-cpuD#dZcpg1d8X|sM9+xULeeGu4EvK80l&&}CFEk-~$BKzSQTPP^{c3KQPAfLi zou=`~@=Iw?r4^8FyZ%#}`tdX>X*0Mxsw5>$iw(bm9hIbJE#||SCyZT$W>IIuWFC9n>NiPgN&AVY^_D4)GEX{C z8gsf&n!9cFt&>1$9&L2&GbB>$4vHf{LGa9W@`&6NS+2Nkq*5kNOZxhnT`l(zR;2&c zJN{(LHIU=W5}ohAW>6zbJ%5s7eEhfD=S*M*7jUML2pNuuQyznsZ0`=xK}$asbUt}Yb?{A*k*D53g))E1gl6ttii~=Fs zmy`^-zDjge+$N*tpRnF8wf9rpJY4fTGw$1E=c&MH>pohMftw7 zf3FE`sd$`uCvV`MH_)8PFa;k1@=at;Kz6O`I|f{9b)*Oj`v}5Ut9_$ck)bi#}Po*FKwA9(m}}n&pf*YOM{ztOXV+qLx#c1ya}egF;YN%&o(2y|KI8o zQWVVvVkci&w~gU1oyP4n-+EC%sYf12Vo?W8rqa;IU*2B4%3#vE;T z%f$r3otGxf6eUt28G`5@D?-*QL_-d1x#1%(cZ>C|BN>Knr5X}=^RlCokp~wJKcn<3 z+7BAbedH&$`K0h=FF{gZjs1_a@+qFedgN~vrR#?x%)-KDy!N=fuOl z32}YiD404e{zBm4ReYEpb^C_UxQ0B#SgHHx2&N~Nr#;_rcZLLC9unT`Z2x5tE6ls7 z-9a`6vZaGOA=W+ax$}s9aAwjG@>R-yd?RWC$_wWM;!y2KPWVxLKen~%gr@}%-ix-H z4R2dFCMKs%w!AdyNnUc6yX_^<*{P85%Bfv~`T%nq@=yaBQ7u9d>H?Gasqrq4mS4wm>0Jy<$lC7bo;2$Ev#V4>qq$I zXJi4yh>qrWtrJ>KxICBk73|o-N+?d~y^5z1mY8avpPlEHaMoM&k#ojk%-E$o8mqO+`ehY(pJi2i((rnH>vKUY z&T5pEy&tqWl;*K?(=Fr7r?S0z3}1x<=2*ifh%a`2r+nG#B|$V}IeL6_$;gB~M1yNW0C{X3daoOUP&@LV4hPC^$4{OBM#@PuBv zp!od~Q&!mHS570D=a=Gxw;u-ZuC6QrKH{;LMBqVuP6?%h-p5Dh&L8OkYGqmw33Zs| z&VMb1eB=Mx6tcK0v$tf4M~n8!TOWK!wqA;um7H$AEgG3EaGK(KrIZ8NN(begVb6H^ zC4>g0x~u)04`c?Z=3GB;NP|6lt>!(~&zsjj9Jo0tetPmj`^j1!1~0$^-uHilcRT#V z(4oSV773N4NQChN2pk8u{4%s=3s=E&o$naAR&2_X%-pYIv$h1i8VFfNQQ504Nf4lL zgpXAG6-@A~7F}iMpgg2p`;JjV-lFP9iF|JG7O$p$LAoLjA-XYp(n(IZX#c0>kLn#n z`^Am#DnN|ONC;Gl`u@(S#8)eHnA zP{wntthD@*Zr|Y7qOxRx61{%ausVT!?h~JLT=#LV_N@SUj`1`Imx^{USOk8dqI0}= zDFCw{DmfrutZmoojT^Zz&o7i8Ad@qDr3@aVc!5`J>wY5ij zDJ-seV6_PS>BS9&rj;yPS^k=7ydpzK?RE!+D}@+Mdhljm5wH+PB62@Ez=Rqd4kcYH zuk1lD7fWlIx=kJ13Y)lF+?{ZYp&IKX5XI;DmhA%u@53+>us3Mi)KO5#;0Fp zKDn9rHd{%^n=(yG3i~%I2JwXGiGJ6|f=kegn2C--wHh79PAyTmW>C=h3a^bvoAZQb zki2s4+|uo3&(!0O89W~jFJHF)u&((9j}bxuaSYCCO$w-jnp`dIHp+^3eW0cL(RLUX z24-?w9n$W7{L%69lW3;{2fw%RNY5@g7BE}w(dU+&?`*a&(>{66KtP09I3cY}tMdw% z;cEX(^Yt^%;y%s-*G&bQ96|wZX}^uf+Mg$0rxsIm39gM`)+0SyS|vK-Bu(7P9`{kn zFWp{mieI+&!1WUCY$T2yC0pK2l1&^)HZb~z(r63^qY2Cu?v%;6DuSOO;aErPjh-R} zHc~5g^jNn^=KFbbL+^J#m278?jO7IjSfs zyc-bUG~(cx-$nTOLviW}iXUi3Cz|Jc1Y&Q>pKT<)2E>#`&BWY-@+^$;A>>Jqu))Zv zw~@7z`o*2qL!KV9q;qR#6$$eBVyQw;`4i?ZLGH|Zwn`f>KEy26CYoO><+5d(dL>e& zq@)qznD9t_Z<=1+LqVq}nIb3S?r3Z@GQ0mayUlThMTH4WB!ma9PROh;9lWI0ms(x% z0ebe`*MPvvd7IEQ)SkTMblFlh3tisSpd|G(HH%k!d89XiR}|qmn!wS)X)oWyA!YdX z;&dU+mm+?(qbi?6UJIG6-HbDkH@Dvq0$iFd z&5Dcv*niu_pvTjZLu~vLvo>*7a5HmIk(dbKD8i7ADb!;k?>cZ2B(YS#i>Pbe{%wIK zW!(}f7F|UiQUSSZ?}r+)g@m%c0M#rh5t{0+>gV*j8LY&A0`?=`WQ+p{$Rq2$?++HQ z^XrHbI>P-S2kKJ&&26cvB9g}T+NomSXciG>$`as_C%nOXAObQQGNds&V)!1L zmHQjEMpU7T-))Fk(=QhadSub5p66K2w*?jWNE#nQGVayAi9X*pgD6bCyZ7q&p|j#S znyoqW`YkMsvBG|7Px%}XE{@EOP3U|-RE=;5FvnLzIxf_tYNjzn2Z|MgJJdVcVt^o-mCSBc8u|ZZCDvL>4HvsbRP zOm4x~()k{`J^qLG6F7H&X4DNa(j$VXK<%u1!^Fr9p1VIq`SERtSjjQ44P_EA6Q^Qe z@w66bJ!Yq9pfB;eeveOT$?;6u9Ht0ENdevmKx@%S|ILC}NhN`a$v}#Di+l=b($?>R zY;Y1(LJ3e;!1b*Vw!4WLq8a>CJ$8jaj~#CMzxLSe`htf4vBz$=x}EAjQO&cyuCkU* z;TETvm6COk1IqYpT56umTw!N_I}(2>5r7*HB3vRIjQ>cH{yGPluyX4cgk-_FX+j%1 zg^w_i9M)SXuP$TRQSYZ80-cVriaOrhqb6^n#I7$X7N*2nU%@KOn9wg4d&NY_?HZWZ zyuZhpr_6MPqYX*=poAI)r_wU-Ew;*HJptKXQ*_)9w7>X4;H}ajYmgqC@d%yYE-DR;&&?iB=aq3c;phQ;diVUyNwI3t};x|By+Q`a)7#UUn zDhcb|I5aOa-7%zmpYW3CCYeo2!jrzS7R?HZGmQsNvAqpmYw)B*&cS8J zdbL$_fck|nSS+IBw?atNuR=&RIAHEDtpbIR$X|t!mwqpVlv%$7-YJCicrN4kA!~To zy23xxiuG-$6>B6J&b4v;T{-Ka^f@f}HBi77e~Cf`W|NHQ#PDEl0)+8jC5_;3x!oZp zkJ;k;p1B0%Uh`VKys+IH_k7jpAV1${!>K0!YrHP8iEsF3Foz+yX$ao3V_Q&%7f65b z<^r*8GiVmjZE1_z(oX}V!3l*$Q;qzCDe7-HfaANI4pdrY)Aj?V|9Qze4CQDslIAWm z6J~~RWA7C)eUo#6+v`V5LnL$S6(0_}n+N^haJ&v~Z&~X~G z^#(pAdx2{}%zsP&4u`nT7xB^8tBco8Vb`0NAssvn>*zlcAqAAAJO$GC7Ld7{Bx}GK zz5JN@{`s4>SG-{K zF<^vU;VAyh0G8I~2(|%jLusNGKr$E*we%z$l*Va;0oq!5kk}UFcn)e^2`&Q^<;Vat zpUJ4cm2poeWp{!?34q|%Qj3soKrq02Qmq~!6;l*T6R6NR@LlWV#~=1E55uzW4^K;%{elEIQmYc$>ZLxNpd~!q$HuPngVbfC?NSdO#yNM1q!%LS_HT zYx#VrG6ZHK@K_RWOstSkAc6A34W+(Z;?_UvX2=2fE;Qu>=)Qq2Lo>lc?92S{_wdaT70*@5RQJpmz0Be{U*tkAo#?yE!E?w4k5+GqEO(R%PAn#WOZD~fciyeQf*X`YGfOC+IkYC|c09_U{|Ju0lxfwaqF0TZH zptp|f{`QBDaT%2_i9ibKX&hxI9N~eV1s-IzV*a-#l4LkwP4G~!0&kU>jei5#{k;(X zyOqC439?od=$&e6^u>4AMKS|vyZR5^oe#D+p}K#GKJSI>M4uPcOi5SLcsgkN^JK=M zHT^F(;P&|Qy|kM&`A*6;@b?YC@>Z=M2+4~gi z`M|mDqg1*5a8lLV1Ur)zmQxb9oRH=aJOCU6nVM6*G>PO5FgM&0vU%z%gdsK&3A|EB zv9g{I$nQGi7oMC<41n@+qa417K42_HM9c?QqcYtr7J@4YHiI^G-5tu`Wqq5t_FNly z9=`ue>q#+&HrVOyjT_hMUZon$wBpyRVv^!Mk}e@@Rgf+jzSKejfS&A;v2@T@fC;dE z-4JcabwSb&I-PGsQp)p(8c?O++D8VJvZ#Fn)iOXJ+!AHjomVLo`ffByuK45TXXT`} zki8$>#Y!F3Our=Z3#!ZlxL>PF;SAM|j93SQNLUMkeP2P&Eo&X&(vr{idhMGR!-R6` z-`W0HibcA9iMTo)6C@t+1T-*FMlF%q8XLWP!qTA&>Wid_WRU@brqgu-N)zD_q=Q!H z0W-4}Fd1A5fcJsMM%7E6n;?(5-hlw#=n345&U3U|bW(%Y&0zEiR$$29#gm57%3RrHrF_%BXo*7PBOWrXV_ zpy(j^93*ZKRTxO!yR)=_W5DF(b9Q-Jt2a z4$s`N5)`AZzZai0%L}19;({#Tqh&Sda}R({1?E7}3YF}~aP6Zon+X{?w|>hK>1_K; z6USnvwq8W1o)fq7$^U-nzF|9abc6m9q*D7CcpB=vb{$XzZnUC>k#;)%3uK`^QJz?w1e980)qFoQ!2MOlzc#d?6?L}6cP{> zdy!kLD9L2g_gE&ly~gDpzkFgz-o zZQI_QIz&NVq$o+E_@^s(QA~Yz!~~@|2f!m;^A&kSJGz`VVWtu1*)PdpnM$NQ6vXD( zdAlqP@5z*>Y&h|lA9pf-nrHq%U5WRkTTA~}2t}+$iu=8w>j_C5>2>AdyB=%&bcsE- zPgf4eLT*-LT{d6;L{~x^0c>nl)BqX>DXcqGU1Ah$Y-?QI0bONzAJ$M%od2G>cUdjj zAIpr{4(}5y^@)V+tlOtA%02`f-aYljMR<=CZM=Mh|+c9~pgfT=N zi~x&;mkdv6k_AD2p*gD!q)m#mjOYS~%AoIWYz<%oNn`3)o zvjxmMx3*P1_(+zA(lVL5x+!)9yM3m29k6Icy{}ghgZNIT5DB9q7xZfdE!5SswX@|n z9uENfN`_t}jNEoYbi;fR*k9i>$UVV>4S<9}q53JSsO~)?PqJ(6tlWfH1SgHE-!LN`abG#Xt@|iWM z5X2s%+|McRdRo| z?lkSi@IV|gpY)+x&`?cvdIxrAg&Nu61m9$sJLv}VVvj+Bu8&}IWsTInAM_Z@`pch- zGk$K_N&5i=qj%A}FR^TO;<^y)%cXoJ}sGx^em}_WmbT24pS31cJ>Dx{}65 z)`AtWy*YRWDUf*+-dN-D`R#QXjm~HOF`khY1@*G}0@**-K4cVVM|yz;xR->Bt&mIQ z+e}o5`3m8po#?cWmXP9G(8ZO68ZQa>FJzSy*lQAmEYdSQLqiG~Hcqwc6t{TKavr?k zU=KgY^xpNOTufn7={<<8YAq5&#P#Q_+LIA6xyVE5&gMEAR-1({l`nw0X(OSHO^YKX zC6&k4*&83xUJ!g4eD_DMF2rc^l8s|4_xL4OZJf^$Dth5qa_?Bv%QwQ_tvsC_k3R+* zyw8T-C7komZypYX6-MBJOABn5aCm_?YI_-3EWXgsTZ4V6y8CS)e)EBkT|=^YGJ6YS zf6eW}!+w)S^OTDTxL{~bg)`L;^$`JG*|gT~^x0!TM5?J4cgf6p$MnBbGkUDIhqJTe zY@w{akna85o-`@H^^Y=FZ)z*r9R1vob5x{mhNM()HhTBm>7I41cO2v11Vu_^@eng<7%y{ENSIt% zFp3G=7(hTJU0v0$W}bFg&3J;Sf8rR_Z zB32zK_)+&5Sb=6a(HTlHTn^&b?~tS!KE34B5UTa??7hL4eeD<8k*{>p4!${F|M4#DtjbNAbGykKM3$RRVX9&x7>(WS*4;iSUKBag$9 zJz8Q{o~D)(n6vi~AhCRu%jo6xnj)bJaq8R07c8h zms|5dN1l)VZC5foM*2uzdOVxX#I2+EB$b5*b_knfH;W{5>wkQFZ)*9glo_4)yyk24^QQP6hl*=>D zXn(%x&gbjE0~)xYSs>fAna4>S&ubYR?$a7wh8IlDlE;QIzHIziG$?V<6yd1lDz`_e z&oHqJGUl5cJ2KW*L9K$5j5Wk+G0L}TAJ$g#R) zZCcyaq=##G+#8=ed!2Qqk=;08y{W+~gK1>;qd>}?tGq{YW{@oKkrZ@+B`>}A?+$3j zcbwAv7c^5cZlwEW#g$B?8o4B<(1^c=4|mVSyMOc4F6Z(D$q-T5Q}cp&z0-z;0u85p z)REbM>P=iZRNdTz`0mWwd-{=9q|E5%4dXr2nxgNFG>w#8^&JxXR35v^ppM99b&pT~#peMO+6y0`NZn$;~mhNsm8^^SqUs%^=3yTYvrdXRF`|u`LncUa(rQAtg z0%Eu?+JLsO0DF^0%b8b!Qf_31fbH1?9Sfi zN@-?k$}w|N^j_)kl266y#RQ)FeLLhMBUOiN`>Ry@!_{xWi)3aSy+(Q~Mfzwuq?$^B zkX57OMrMjJ@^;C$YuXJNji>xiS&W@q5SnXvQUFO)kkmQaJ%#q$jN@K4A3$z?*CRan zlb;$&w}t)P8#7ypYpIjD)peXnNV0-_CSmUbuPaJ9oZ}^=Llr4Veyu^#SfbUSVcxXY zYprd1ADH)cuVOZ_?zt7pFRc8=PB01EZ?_+d|zWg-=2*a>o!#0^2E zwvD!+OOuN;eSv}f_c>|Vx6W0G#)w|6@zmWIIREPCUe+zW~a`%%47G3XU?W~Tp+Zvt`WL$;9$DEuzoNVlq0`M z-a-rj(+^H}{{rSE%7b1tyr`|N3-Cx800wQQe@>B;#HG#AEyC2togyU}XtyzW}4+dx~ zroaspqEtU&H^$3Eh%n~U!qhDFi%&Hj#}ngU5#$*GF-b+><+IF zw7~f*oC{mb2-j#n_-W6#uhUk*Lu<4I5DngJ1`R)-2^vEx4ds_&!6?(x2wdgZL$Hy zFTG`|p9r0e8guWL8QYsr#(}^?h&T9RiW?#dnt#!8vG&C@CeR&rD@k|hiDAEy=1PiM zJLb4^;K^duAMCYz+5%z0=#hKfDLdYcv_zmpD+}BqNR93aqoDN0^cm&1nq^+LdHttd zyH=g|yGwRohd zr%bNH330tV^(qAQc>h(1=i3!T@9{f)eZ&6iRax|UAsOJ`9l8eFkG?XuD2 z%c8j3;2xd{@d>PZ7e42nR*bv-F|p;w66=%O0~HNTGq&sH=x7*3Qyr8F-BB2hxTHkh z`yh`p)M(vG(~^9--}3WXqGMkn-wTWFMr|3d*mjvc=+Zr{=`5Ld3^gg-Fg8z>aDl~l zF=4Yak6cIl8HTQfvKfC87>C+E>NtM)Qr;oYb(sp94}jdRATS6vlsKLoQS*`r-8Fd0 z$x0~G_bRbnC0eN+spE5P;LK`kOxZ-Jn_H?la{a?F->s>s;jm$n9`J5>NV*XyLBF?S zrzkFH>d=dR3eSdQFuwO?^sMv%1tGMZSYoNqL z>kn5|wY|NI;gf*v)2ZR7AF}1p^{_EZqmFxIX=R&RjR7Y2mNV)>U(KYG8-R3O#Fm45Nz4duDlQariB4(3w zrco;71p+#j&UxSFvycKM;!s!84XlFiF~>gXglEkPGLO{l7tWL}j>AMF%hRM)H7ia0 zI_^veMI=3J=K+j5E4b69eJ{Z}k#Nt|l&-tkuY=CCpG$~o>W=?7>F+Sm z9+cy?nQdno`sR4q3S~H8SuG^(*k^9P&-w(_5r#+I9axf?@lC$@9P)l5ca3rH%n;rP zWaY`Q+xXdEb7Pb$&U~(w=FIoLpsdG}A*G6)_0@v&y9No7#@)CS`aw$eXpmR+g+4?m z$P=Q}(rkzz*V~CJU?sc!xJK9O7kw7>Q~kOMhfP_^EPu3naL%r;wSMAlBjRdU-eZ-5 z<}}`qvHrFl8?Z_GbQ-vFum!l;FsdyrfD#lqmPWR&N-^)D>5jTX*I}evL&oLwea)Aa z43|Rwsbb*Jx)WyRA!nmcQL5tr8gEO zPSr1=i3gJ!m(<^tuWAlVU8{o*g1N+Gzxl?35$ud>RY*XhruJ?t0{klHRL0@E+CNU4 zg=zb#VHs${g9=aBJwg#^H2(fqhzoW@rg9g6(S5hRPn(&}QcJZ<2qmvop($<)==H4J zIA|~rr%K(m2D$e>Qv)EpfR=XdhPyy}WL95n##FLk`xt8woU}uubDE7cLcDEx-sSGw z2dBM^_P)!tyyn(AsD!)my28-g?%d_u?4amGgpmFfa$#U@WQhR7!X87Adq!n?jL(v{ z-^(UVBBD@-gtm3fiiRH$a5ovmiLF6eU0 z>R9n#V)O3NMw!J4;#sp{fvm%D(%=#F1o~_ff z4Ys%}F!f}7?I4atVGAL^lFVnD$=!7UeLZpBev4GCw!9>h#O3LYuz{{FDL#?@x*P{5 zOeOE$(}|BWIJ^)Jz&7Asbxt~QMbzqLOPo@p1O-Rc@#)s=s!f{v@yda1M(^1oWL>N8 zf0$v|keYaP^1WljYZp@Pi=K;nYq0m@-}`0_3^2><@SVzeZD~SA4e^NW z9*0$<`*X&&XI9CCztvudFePZJk@w-3QCh@hqE1Vi*TIhG6dI|2P zkJxQv&dP6qhr;(O9+^kmi;$DFi<2&OAyspVvANng7mYbz7V`O|*IoPczS4QH_ED5F zkA+}v?^qUq-^M6qfu!dPtIs}F^6Pq$QhU=&mG`{Z-u2Khs-Zqh3e(%*?;KLK=Y#zv zzv&aOOn$3Fwm11`VK*5aWkWdTcrL~!%0b;qGgy}Puln`gf3zJY>u?D##w2YGafOr= zAz4S=``UphgQm(1f(hr)vfaWj(K(3wOn0(rWk+1x?cz#kpAZ`hF*H#&FGUHv6x#2M#kb# zz|XM}jxL1ZB|KwQ&Cjv!F0IEy^gal9dgvniuzqYc<3e9mFtfl?`*KxY%&N!c<4R3>FAk>eMM@*O#}~pFd;IwEl>(ij56wxN z-I3DLlikujj(U59@-tcPDYl!(y>W+PlHHpaJ|e-+2QI}m;DM-OR>6k(j5>m&Df{BS zr4Q}3-F$=oTuA}Wz@EW{t%Mo|5_9uryDqp8Xt3ZOz+DnrXfngwL7=t#+um9!OcA4U z(|Z6-3sN?Rl0@ff2~}8_&FA#X!D}LD2i6kTv8329%5QbCSc+Z}%#jLvT)^BoT;Me~ zvDqVDmt4uA7bw0@X{sI4nn$16I4bX2 z)qj}^FD&XGpwCH-MRBHFJEV4j^R%k1P!|uxD+Py9*Cr{~Aoi<^kf~PfMd2i9q?Gpu zcW+mvF=OjxT-!3W=b4Shf{dMW_&sVvykbJrgWV9Otc`Y%e@PM{q^!-QjbeD{U3>~k zeV~oVBgA)s)f)96wQf=Nuw@to|F z208H&UN1pfUm^)cWBUpjiwMU#?aJo2j3KVYTAz7SnKXAi{6Ttg(y@|4TJz)v|PNgfOv>4f#R|3H0#Z1O&OX9Npi=SIT?O)WmCks zglE^G!HHX>(@!AV!X0=_!gkjNk4ICrMoPefVL^IRw1_y6%>d2gn$jUlg}vJ>tO#cJ z`Jgkv5w(wye2{IY%2Cv14qCX|z_T)Q^r!a2*(l#ZY%ILU7mto!pq_K8WS|5RS$+Bu zf|My@M3l6T^<(Mf$4byWh#vLn?X(;DnYq{ddJUBq=I5rO^$%G*OP-VETY-n6Nro88 zx#P)r_t;A2(39?wsZg?Jj(;xyf}D|zNxNcTcI@b}*4HJK#UWP|i)v( zovjdW&FfGR5!P1^`=H^Ks6X-Mdd;I)k8`0%cd(URx~W48&S_V5nR;EtE0|Mn5mlO1 zrzX@Mn};%>af69zOk*eEW(n@3hojB8eb|_EQgH?`rumYQ!P+@%_}s&5u5RP6!dJ#a z22=WUw8l?|UbCy(-MgVSg1v=vvzB4=vpr2|^&zUs?C~=-PHaD$d?GOB3vS;44?B;& zpPST=eBMq&cKO>{ZZtKeHSf$7rP`&v>Mae!?{*u+JQU%Nsyi-J#w%D%NkWg0yuuO&{e8*O{6mgKSrSm=S^4cCzW7?ySg?ehZYxC{jD#MXm171 zC^X);3vZhrxG!+Y(uu2|7*p@;*<{Z_=LEk-9qdQykg&DxQ8!X{3X0Sm=X$D<;?#$U z+}<1?k^bezS(khBLu{g!MVHB^UWp|${a}}BGLeobIw)E5>H~vzOT#g4?CQyWY*1N7 z&-lWPLyI-9j$g+UfRX+c7IvxzT5F^bwg!TS$Y*pj)EYM zouGUf^Yczm?dz{A;a;t;hj8HfD_-$GjfJdenl~WaG!jmki9lXnYD(#?8Qg z<(rhPU!{e2qfT9YC*BxvZv30S5{q5brL+StXZ!H&2kr)PLwCxy{GPOLE{$i)`nH7+ zKa^zfx*J^Fy0P^qhep+bb`v;7qpAc03LxFS_!MscQkYqll1#!clT%}z7kUNHDP3bR zgCrV?LxS`fmlW5B0qDcqC8_8hF$}HTZC$1*O!looFdJv~9ewRDLYgnQnm1G*yIY&{ zZfw(hPu~Pv+{f)cm-BM8OCR;r_ua{B$w>M>&V!2oexCY9w6)$S(K^Wf8KQ9`9`Fv2 zSE9f+vTCvd-Q3Cm+=3^;HS|7gXu(V5QcV730E$%GLPdgPZtY7EXj0A}hOf{w9?v%q*?-_ViV~8!`!q;_J z{0bZGu2AF>D0~!NXQO=iFMdURcQg=_LHq#v%f!oIgoI-72##fZ7ASd@$(@KE4U{P7 zFH+^^D!K4R%ggT0_FfP8WI73{rpX-d0U7ob?V zk+|(m?+kojg{A*BxzwG8OmE9|C0$hf1}z2kce99^cVFTo5yI8^iC;i6ftGQEVQXQz z-|Vc5)LCs;S0+*aYkcC6qfd51MqJN5MP2Scp9V6`hFV>BnjKQ;c-#UiR9~iz$;^Jr z&=RL4BmHO4qX}f?fF)y4mcNVQOVvf5AtCA>YMesv_RFsh7`mBtao9=+MzuBmluztn zuUnA|(Uw1LbtGJ9hi!>?8$l+G-Yc&qMu7G{ueAnaR$bR&e_*eY*sTh^^r?2~G;)5n zu8GY~S8;6@R485{&6Jf>G4NmM8^=MoIns%OUn1w&oV7~w7cMx20nU&`U;vVKo>;MSf_Op`x-7^ImRpQdt+|p z06==?#b{ig$LOq6IUAsT4x}#t{T>v;?M1n3oq#2QW$lgVk6884JO=Db6z6p6fmdDX zHhIom0gnk$KNcQTD-ZcHc}mLR(1|J)PrbP>RWoRkv2m0!X=j5wTc01gMo2z}pnORk z_sLC{rZ26NR(80ZOMhR-(KNTXROMY_Dg;idLc_edpHwh|=%b{t`Y*%GOPNYbiOM@% z?x|r!Nr7WLW8Fi+a9PdW^p8MX!n-B15AcDrB9D>~RkabPOJBDcrE`^Rq`yrcxs|Ff zeBNU2ysN2Ons>0nfk);Ee*CLil$ljCu%cpm!)m?6xd{M}AK+WO$=KaG)wZZi+ZXd7r&=*7Yz3!JdQ=Q7I%+P)S9nM_* zsq_i7F-~Ij2MT7)orUJMiPaRhJn2Ae5%9yDC2Piv#suP&KfEo!cZ6sjVRzMP$l`9) z(CN@vF{@eAx?z&ZXXGJ3c{wyiT#Lx_>4E}jYc(;YmPteXhW1+=4JZ&T=9`2%Rz=r9wIA^hgb~8Z_8iWz(FdNk*;K1je9JX z%R2_n?1T~~X%j*UT@SgVx~_yzjHg&G9FF^uVMymhu=J-ahA$~psx?U_l%}0Kt$CoU z@S$}~^A2;Y(OsgBv*jW4xIXpxT{;uSQv`#VI~>AkOfIS?!>SWLLwGlo92pl&h{Ll~ z@`6pL=C&cEDXHnW1`Z#`f00Cg)3p&?wj?#xJ~#JrJT4gXz>U9{id^CpMYZJ zIqlDztOOBJ+uNQec|_E`(;AEh>N523+hXo9KH49f8kidNnQ;^caJ2$LlqMIw4JAsx zbLpD2daq}&8c=%^ie#^$CHFi9$1)ZN3N7l%&s_@ z^C7Bpd7W*JHohWhAsH<2_vc6Q10WRR)k!nTgJ(!J3Y1u#%$=%8K9l5m`m_q)g?rOk zHWwekN@I2&_6k`%z%0hOmUT`xDG@T&5xfH}98L5#4jY1yhSHwTq#RD-kpJB0*d}># zB6XKQj{i>mQH|C*%9lz-(S%xGobs9xgmZ0*whu!$mKht~j6y?k%jLC-r2R^tEN?Ah&7yNmpEww8#-P{Kh_z@?!&Zc8JdfVDmbqJJ!eokpkws9N~q59&4` z*ke43r9ZdoL=D5g%el6^m=Q265-Mw+Y5&`m9Fd@A`d>-lE)AMdvvDFHs%rzSP?@O| z_$o8S{TaPiNPe?G-3ASh^oc{!yG>iYskc}io?fiV?3+$(*%tCcvK+xhR{nAoKM#?G z??Te2%Dv(-^qs=wA34N!k%o!uyXu;XvqvkVA$2?aE=rGD?A6Q9xvno9Fb6n#kyujv zD9@HO9hUQe*kl>Sc|p6@iVTybfP{dDsGfzRR&!uOx2+y4RMO?&wI zf2<+V^lCUW9$3Jvh+5%!90$^lWy8TroxbB8BT;PjDajCsJ

KP;>}qapXgTr3Fcr z+TjG;f_K@z@4inTfAN$US0^h1-l2-SQjbwuP zKbwIx{HbEF^+`jfm$yI)ZG1CSG~D=R-$cz11_MZ4N`jlwMYCuT zvBf%Q7V|!UIT~jtpu@tpMJOO83rX8Ssy5`4{g+{xck9XDv}nCXI^ z$9`yi{T1>SEut&27cu|3(6o5#4o83%4#E@C?ncP0S@a$0hfy;Z6@I$A0iR6Gb^XW3 zIZj+@-0V_{zCoZ*)8hEQ-Rt>kg8qeHTg~<>#G9@Y_wC)P7M@=rK5@?;|MWsVvpa5A zRfLAezP~jt0o}amN{HM|j!Y-7be6+*PdmMg zvM;<);&x~-(NY+F;;J+7DLjb|jWwZ|X9_`gEjZDe#mS=2wJ?Q3-t z?KmXjjUe5ogs3Eb9J~lyDSm6!Ec)JIyz&-_UvbrMMuJeuNaG}BzK2I}(YKQZO1RrA z+J|PW;+CS^i}=$h=et_(e%)cipzEhTOd(=Me9$gdjF7Zuc@hVA0zUyDmpVs98 zg83D4c=IK?SC{e-2-l+W5gz7v+^_lYai^=%+k_&eNrbY_DEHk9ebF~lqO>O)&$O{O=h_P$(IV>@ox7sWCj1QQ zb~{7`Vqx_NiWh7SSoqq1i|*xru8W4yy>P@Pbg?l|QTRKyjrr@|Zy5i7-4EvOn4#Uw z9Z2)*{&`%0C|5FM!)_0lishu!Qng)clI1Zo4N+x2O6Ezj@=TO{05|&-LU0g*V5h_h00HsQ4A~e9a@|)FP^X_e_79Yc9uaU0dAC z>*)QJ{C-5r?0$?EkqEQ%IokRmV*AzIJLj0L1UqdhBCM+0 z@A~4WsQY0<@pc}uGg*-3KtPD@XLHX-$0S**uA*%Hv{r9XvZ+E=1k%E4dBi!gOsM2G z>uYd!$=v4^vc1oUSIA6UtqNV;`yMe@#2+rYi}P3RJ)<4eO==%QR1L@RoxUVYIXU_c zR(l*dCVTOk@~5FX$_(SF`SN7So!gBAJL^nlQHbcOj;AK=cP>R_nwko%)I+( z2Acb?J1pVPY^OTkc3**9RrK4s)_;CW`~`tOvfZEE#g@ja`R>S@x7Ky-16M^(Sl%Z= z1dTq$*?WyV+hQ$TPE_e)G}3Au`$o@8lOVc z@d6O=z8mo-YvmTd2J>nH2uZZ`mX9PyYRda19+9?foipsWJSFcD=r7NbJ9m)rK(mj$ zsE3TIJ>$YvX8>hfh28=N5kKIb)g}7IP8i0ajbA}u8uC;(XAhQN95yJGT5Rc$6TiX>zHGzH`Yo)jIIrsWh`r(VHNKbM~BvySqT5 zwte2*q~Z1Rnf+`vuWz$4T?}DDi%-XTVUsnOT>xNyL_+y9>L$MI;w#c)rx;6_GMFH} z4dmS+;P&XB)qxJ@S4xZkJv89i(Eh2gcs;)W6c)GKA5%j+s@?c+$khAKiUiTF1Dc?M z#k(KWkEy5%5=3AbO3@T?>?>X8bzBn=2nw%I8?!lxhEiILi#liBnK#Vd&7K;S2%ml+ zlH~15U_5GpX65{*55xqZpb+HD8YXK7w+$t9aX`(^#UgW!I1+9AUJXd2##=7c09wXmnW?_ zAwE=&ZwU+S?FBCa!b01&goSQB2dLGDLNvY@y(KJkq7n!T9Rb2ZQb12gU~&GZYDnE$ zG-(dICH2E569J?*eis>?z>E2fO?3n6yX}|37`zH)W=uI}keI zjBqm1RWFa|%tjxiCM;%#!LPu2eBWDz%+PmGWJhfH=6G!!UX$j+Dxw=aSsX`9Z~OB$ z2bt|mmyLXOHgxB$-HTE5m-?cKY>Q$EKH>skY0BHzHWCT5B-gzxR<}DmL zfw%?#H7Wbg(5U}|d;&}hCrLRpW2DVtekb6{7J=dKxEK-hung1z`tOA@R^+f_$an!X zsU`2bbnyLmzt@8P92)P2e0?=25W{A#xc?#oWVi28Acp=xf5`I6{@1;8|!0cj80 zxAb>djllX%aGo9Tqr}Q9ecN>}53Zv`7&~N!k=6qK!y(m+9@Rp}KFvYhmt2jXRGPtGO}7*O)(R)vtd?;Nuvbe^yE4 z_o|=7N&35`VJ|@u7pD}69p?aPQds%2^1F^r=pUB>lKkH%1GE8DBSNC6+xjO#ZS=F~ zBf-B%HWGAF;vFB3p!D-7@qoWH&i<6fnZ%YT#egF?;5Bb*bDY2TUurr2qVxDauIc!n zB_#iAa?(rz;@N1G-#_(zrPfQ&O=<24cnqC)rS`v3fbW0jS;J$7I5%@N`DsXsKWWX95|-vT#7=e#&C-4{ zIDK}Yf8dL93TN|EiR>9JmWJso9*OZDg+9%wo1vkpXvWa>cx~w^dnpn5{hd+kn87-W z%eRl6sQ+wq<3ifbSKgD$s;}<9xbO5jUq7(Xc0`794TpJ$CZ?c7K*SGy3%;HX3KGH} zQK(-bI;~`o(6d7bWL-=q;!dd7t$l@5VkoVX%DByCbX`Tp?JM_|G^0FjD89srECDG2 zKCiKbO@7@PG>ykk3lI((z=@ffnqMKUg9{~{b@V*Pq((tbUFn~-1H^0&^0zq*cVf_<@9E*VFm0igM-j?n#e zf8E_+_<7P`ei?l9Wg_}MysOFG{R3@l$iFk?Fu&?CiDkxLTJyKgrdI*4@)-I*FnOSJ zYUDXhhfT0nMm#;xTSNE{AGG6?pDIk}z*op|L=TU1W|LTL&v>B~4HJjE;=TiUiV{R( zSjI(LaSZWx0U|lS7MtTXgU#vPT&sO+k&1!_9eS$;Ee!TWrE!t^)=4*0s9bmCgWt}Wv8Evk5c z`J!2xe-Gz|h|rXy)KWQ+S17uqnNJ8xM6D(Zc}GGTJ3I)3Jwo3GB?n9w=Fj04^D>#2%Bg*M<0JU0kNowW|7x2}7t+J?3i!GD=!e)+;r zD;{H>XV#Who=74%wL-UsNeX1Vl()CN8A6hljL0foGN+gIw-Jnq$E%floQxZ zeF<<#|FUVvHjJdllo>OzpV1xF+Z4Ct;$DwJTE}-P=AAf47zS1dt#U432%$@m z&LoW>xQ4&<6=&xo{DS7jiqX(7$32fef1ERXrcLqV(`^hB4E1U`%Nuv(#|}l=#`Q;C z510yzNbedZL8^Qehv;iRiK{3(? zMHu4Gl6+zE;D5Jm*1&&*%jbbl&)_>C{w#XiLMgyiwhv!_7A?N3k_eDj?w~EDPHF1a zuKEwRE6fTsf%^p>?)x54`a7UrKmRSLx3^J~H6Y1En{z_w{_g>Fw(j}2BB|of-VH^B zy0U=><*L2#k#s;F-Jhx?$w z0{m5q-Y+_>G+?NP+y~jlDsV>__3#kDliO{YQ488@xc*}j`?ME z=}0i$PHj!sKXUzntf&znY8Sx!k0ly^Dl6)*arr&t^KXsIw@~b}8h9UNjr+9c~JOk{OZ009cX#`5QCDk6)xaF!iet}195USm|r&HO0w#YCiJnE(N zN*te!X9HT*|R$Vc54EU6x_bPaDvA@-n212Z^Uk9qdZ zy6QU@?J;oPTX!wRxB@D_y{GjjDzXXKuYOe_KwgKDpr~3icR~X2=dewMR1M%(Mjsum$VE5r$ z{7!R!t7GP@G(xFznM*?X--bcK4dLE|j`RO+gTLD|t1pGBHXoR-W6XbkyYjaIsrY`I zztcC=3z*CFCQyWwGym<~{NIM-(r*p^cc#f^j_xb;Uz6d#Z!*9%&{aQJpz)!C?s{r0q)~=8F)2F+tsYSgFg48I!t`JO|b|DP&@P5BF*p z1>rMlBtnGM!I}E~0pBLK|Dkx;B(#6oau;F<6^iNxq+V@uc3cTou&1ZjX zZzrEs>YrS1#(yoBzxK|*ohyG$hJTt2uM^I@iC*^VFi%7^>}LR$gD|C&jaP(-!>6kp zSX4CTeiSQoUxaxepaVy$N0ZKNUr!oC;C)1%A|kA^LI?<1{A~l|B?F}XNF7rDZ+H4y$D?8Wd^};D%qW*9eum>osEFG7p;(u^c tdTfP!Jup*!dCDMD-r@7kCnATF8xa5-|B}fTC~)q|JLkHx>4C4k{|B=*gG2xT literal 0 HcmV?d00001 From 66af6d1339f86e87a37d5f505109b59c729de198 Mon Sep 17 00:00:00 2001 From: Evan Sheng Date: Mon, 14 Jun 2021 11:50:08 -0700 Subject: [PATCH 2/2] add error fallback line --- cluster-autoscaler/proposals/expander-plugin-grpc.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cluster-autoscaler/proposals/expander-plugin-grpc.md b/cluster-autoscaler/proposals/expander-plugin-grpc.md index 21fb29f1745f..2e593c84ced7 100644 --- a/cluster-autoscaler/proposals/expander-plugin-grpc.md +++ b/cluster-autoscaler/proposals/expander-plugin-grpc.md @@ -71,6 +71,8 @@ message Option { } ``` +If errors arise with the gRPC server, the client will handle these by falling back to the random expander strategy, similarly to the fallbacks of the current expanders. + To communicate with the external gRPC server, CA needs new flags to expose details about the server. We’ll add a new option to the expander flag: `--expander=externalgrpc`, and inntroduce a new flag `--expander-plugin-url=https://external-grpc-url/server` to reach the gRPC server.