Shorthand conditional evaluation doesn't work with parameter[(kwargs)] syntax

[DillonJ]

For example, in constraint_connection_flow_capacity.jl, we only want to include the investment variable if candidate_connections is defined so we conditionally include the term as follows

*
             ( (candidate_connections(connection=conn) != nothing)  ? 
                + expr_sum(
                    connections_invested_available[conn, s, t1]
                    for
                    (conn, s, t1) in
                    connections_invested_available_indices(m; connection=conn, stochastic_scenario=s, t=t_in_t(m; t_short=t));
                    init=0,
                ) : 1)

The above works - however, it fails to behave correctly if we include the square brackets in the call to candidate_connections() as follows: candidate_connections[(connection=conn)]

Any thoughts @manuelma ?

[manuelma]

Thanks @DillonJ. yeah, I'm pretty sure we haven't implemented the proper methods for that syntax to work with our Call type. Can you provide the MethodError that comes out of this?

[manuelma]

candidate_connections sounds funny for a parameter name, would you consider is_candidate_connection()?

[DillonJ]

Can you provide the MethodError that comes out of this?

I get a not method defined for * nothing, int64 (or whatever). I.e. the parameter value call results in nothing but for some reason candidate_connections(connection=conn) != nothing is false but candidate_connections[(connection=conn)] != nothing is true.

candidate_connections sounds funny for a parameter name, would you consider is_candidate_connection()?

The variable is used for both regular connections and ptdf based connections and in the latter case it can only be zero one, but for regular connections, there is no reason candidate_connections can't be greater than one. For example, if you want to make the capacity of an interconnection a variable, but in steps of 500 MW.

Edit: and actually, in the connection investment tests, we test for candidate_connections=4

[DillonJ]

Here is the example I'm working on

The code:

 *             
            (            
                (connection_capacity[(connection=conn, node=n, direction=d, stochastic_scenario=s, analysis_time=t0, t=t)] == nothing) 
                    ? typemax(Int)
                    : connection_capacity[(connection=conn, node=n, direction=d, stochastic_scenario=s, analysis_time=t0, t=t)]
            ) 

The error:

 unable to evaluate expression:
        0.0 + 0.0 - {connection_capacity(connection=connection_ab, node=node_a, direction=from_node, stochastic_scenario=Object[parent], analysis_time=2000-01-01T00:00, t=2000-01-01T00:00~>2000-01-01T01:00) = nothing} * 1.0 + 0.0
  MethodError: no method matching *(::Nothing, ::Float64)

[DillonJ]

On a side note, I see that in the tests, it tells you exactly where the nothing is - so I guess that can be very useful in debugging constraints! Didn't realise that until now!

[manuelma]

for some reason candidate_connections(connection=conn) != nothing is false but candidate_connections[(connection=conn)] != nothing is true.

remember the brackets return a Call object, not the result of the Call. You need to call realize() on that object to get the actual result.

Edit: so, candidate_connections[(connection=conn)] returns a Call which compared to nothing evaluates to false. Even if the result of realizeing that Call is indeed nothing.

[manuelma]

But I believe the problem here is we haven't implemented ? with Call so it gets correctly interpreted by JuMP.build_constraint()

[manuelma]

What about this

connection_capacity[(connection=conn, node=n, direction=d, stochastic_scenario=s, analysis_time=t0, t=t, _default="something")]

that would return "something" if the parameter is not defined, instead of just nothing?

[DillonJ]

What about this

connection_capacity[(connection=conn, node=n, direction=d, stochastic_scenario=s, analysis_time=t0, t=t, _default="something")]

I really like that a lot - it would be incredibly useful!

[manuelma]

I'm not sure it's too great, _default sounds a bit confusing... maybe something like (..., _nothing="something")) to make it more unambiguous?

[manuelma]

And I'm not sure that it solves the issue...?

[DillonJ]

Actually - you're right - what we want to do in some cases is conditionally include a term based on the value of a parameter - the investment variable for example

[manuelma]

The problem is, the value of the parameter is not known at the moment of generating the constraint. It's something that might change as the model rolls. So I suggest we split the indices into two, one set that needs the first constraint form, and a second set that needs the second constraint form, and write the two constraints independently. (While reusing as much code as possible)

[DillonJ]

What about the case of method parameters though - they should not be time dependent

[manuelma]

Then I guess you use the call without the brackets that actually works... right?

[DillonJ]

Ah, ok - cool

[manuelma]

Another option is to preprocess a parameter that returns 0 or 1 so we can multiply it with the other expression to get what we want. That's something that JuMP would understand I believe, and would be compatible with the automated model update.

[DillonJ]

But sometimes multiplying is not enough - in cases where the variable won't exist at all, we want to omit the term - but I think we can live with the restriction that we can't have terms that are conditional on something that might be time varying

[manuelma]

Ah, right. Another option then is to use get() for that particular case. get() works for variable dicts and it looks like we already use it in constraint_node_injection to access node_state variables that might not exist. What do you think?

I did try some ideas this morning on getting the time varying conditional but hit some walls. There might be ways around it though, maybe I can think of something.