You'll get a notification when an alarm or timer goes off on your PC, even if the app is closed or your PC is locked. Be sure to keep the volume high enough to hear your alarm or timer.
If your PC goes to sleep, alarms and timers might not work. Only newer laptops and tablets with a feature called InstantGo can wake up from sleep to sound an alarm or timer. And even with InstantGo, the device might not wake up if it isn't plugged in.
When you select + to set up a new alarm or timer, if you see a warning that "notifications will only show if the PC is awake," then you don't have InstantGo.