To adjust video quality, go to Start > Xbox app > Settings > Game DVR > Video encoding.
Video quality controls both frames per second (fps) and bit rate (mbps, or megabits per second). Choose among Standard (30 fps, lower bit rate), High (30 fps, higher bit rate), or, on Windows 10 Anniversary Edition, Very high (60 fps, higher bit rate).
Game clips are recorded at the video resolution specified in the game, up to 1080p. For games played at a higher resolution, game clips will be scaled down to the selected level. Higher quality and resolution generally requires the PC to work harder, which may decrease performance. To determine the best settings for your system, try them out.