The LFH throttles returning memory to the back-end heap. When an application is in a cycle in which it allocates lots of memory, then frees the memory, and then repeats the process, the LFH intentionally does not return all the committed memory. This occurs because the allocation pattern suggests that the application will have to allocate the memory again shortly. Therefore, the LFH will cache a part of what the application freed to speed up allocations. As the application continues the pattern, the LFH will continue to cache memory, and an increase in private bytes will occur.
The main point is that the LFH (and the heap generally) cannot guarantee that when a block of memory is freed, that block of memory will also be decommitted. HeapCompact can be called to indicate to the heap that it might be a good time to flush out its caches. This may avoid an increase in the private bytes. However, the heap manager still may ignore the request.