Commits on Nov 4, 2022

1. Core: Multiple input/param gradient modification …

   - change the core Hook to support the modification of multiple inputs
     and params
   - for this, now each input and parameter that requires a gradient will
     be hooked, and a backward, which is aware of which the current 'sink'
     is, will be called for each
   - use View instead of custom Identity to produce a .grad_fn
   
   Note:
   - this may be a breaking change for custom hooks based on the old
     implementation
   
   TODO:
   - finish implementation:
       - parameters have no grad_fn, and we cannot simply overwrite them
         with a view; hooking directly with tensor hooks is problematic
         when the parameters are used in different functions
       - there may be potentially a better approach than calling the
         backward function once per 'sink', although the current
         implementation may allow for better modularity
       - multiple outputs are still not supported, it may be worth to think
         how to do it, however, it may also be better to do this at a later
         stage
   - implement tests
     - new tests for the new functionality: multiple inputs and params in
       hooks
     - fix old tests that assume the use of Identity and are not sink-aware
   - add documentation

   

   chr5tphr committed Nov 4, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for 8f11583
   • Browse repository at this point

   Copy the full SHA

   8f11583 View commit details

   Browse the repository at this point in the history