Reduce decoder latency by 2 frames

[geohot]

I'm not sure this is a target of this project, but this reduces my latency on a 20fps stream from 110ms down to 10ms.

I tried adding flags to the abstraction layers, but they are so deep! I understand you may not always want these, so I added environment variable.

[rarzumanyan]

Hi @geohot,
Thank you for the PR!

this reduces my latency on a 20fps stream from 110ms down to 10ms.

Yes, it reduces latency indeed.
Honestly I didn't conduct how does it influence throughput, hence never implemented this flag support.
Any chance you can test this? It shall be relatively straightforward to add this flag to PyNvDecoder ctor arguments.

[geohot]

There's two frame speedups, there's bLowLatency and setEndOfPicture which each remove a frame of lag. setEndOfPicture is free (but I'm not sure it's okay in all cases, we are putting in packets that are exactly one frame with no b-frames), bLowLatency cuts throughput by 40%.

I tried adding it to the constructor but it's like three levels of abstraction deep...

[geohot]

I also added NV_NUM_DECODE_SURFACES, it allocates 20 by default. You can save a lot of GPU ram if you only allocate 6, which should be fine according to the HEVC spec and tested on our test videos. (though idk about very large frames or b-frames)