Support multiple volumes in a discretization collection

[inducer]

This isn't finished yet. Missing:

• Finish eager cross-rank inter-volume communication
• Finish lazy cross-rank inter-volume communication
• Document *inter_volume_trace_pairs
• Test inter-volume communication (i.e. complete the test stub)
• Test MPI inter-volume communication
• Needs Allow distributed partition boundaries to be not just rank boundaries meshmode#307
• Needs Add flat_size_and_dtype arraycontext#150
• Point requirements.txt back at meshmode main

cc @majosm

Closes #104.

[inducer]

@majosm I think this is headed towards being not totally broken. Could you take a look to see how things look to you and look into the remaining issues above?

[inducer]

The changes discussed in inducer/meshmode#308 (comment) are now in. Ready to take a look, @majosm. (assuming CI is happy 🤞)

[majosm]

@inducer Any particular reason to be using

def func(..., volume_dd=None):
    if volume_dd is None:
        volume_dd = DD_VOLUME_ALL
    ...

vs.

def func(..., volume_dd=DD_VOLUME_ALL)?

?

I don't remember the exact guidelines for this. (I guess mutable objects should use the first form... not sure what else.) Does it make a difference here?

[inducer]

I agree that mutable objects should use the first form. The other aspect that I run into is that if there's multiple layers of functions calling each other, and only the "innermost" function looks at the argument, then I find it cleaner to use None instead of redundantly specifying the default. If you're not in either of those cases, I'm not sure I have a preference.

[majosm]

Superseded by #280 and #285.