You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Redo the model function for floor to handle inputs like 1E200.
The current model of floor calculates it by using a very non-intrusive (with the actual primitive being tested) approach by using strings to calculate the result. But it fails to evaluate inputs like 1E200 and fails.
The text was updated successfully, but these errors were encountered:
Logging this here for the future, since the PR has been closed
Proposed model in the closed PR was:
⍝ model of floor using exponents⍝ has issues with hdbl and hfl valuesmodelFloorNew←{
fmt←{⎕FR≡1287:¯33⍕⍵⋄¯16⍕⍵}⍵
(me)←'E'(≠⊆⊢)fmten←⍎ediff←-(⍵<0)∧('.'∊⍕⍵)
en<0:diff+0m↑⍨←3+(⍵<0)+endiff+⍎m,'E',e
}
It does the same thing as the current model but it uses the scientific form the number and removes the decimal places. But it is still not precise.
Redo the model function for floor to handle inputs like
1E200
.The current model of floor calculates it by using a very non-intrusive (with the actual primitive being tested) approach by using strings to calculate the result. But it fails to evaluate inputs like
1E200
and fails.The text was updated successfully, but these errors were encountered: