Without a better description of "high pitch noise" it could be anything.
Where is the noise coming from? (Please don't respond "the computer.")
Do you hear it from the speakers?

Not the speakers.

Is it coming from the fan vents in the power supply?

Yes. It seems to be loudest from the fan vents of the PSU. I didn't
open the case to check though. I will do it tonight.

Does it sound mechanical in nature - like a metal on metal
noise - or is it a high pitched squeal that changes pitch or volume
before it disappears?

No.. it's like those old TV high pitch but not so high and loud.
definitely not mechanical. It does not disappear on its own. It
start when I boot up ... when I press the on button on the PC front.
Sometimes, it does not come. But when it does (which is more often
than not), I have to press the on button or the reset button or Ctrl-
Alt-Del to restart the computer. Sometimes, it goes away with 1
reset. Sometimes, after 2 or 3. But it does go away.

If the first time, I boot up and the noise does not come, it will not
come during the whole session when the PC is boot up.

But sometimes, when I first boot up, there is no noise.

Thanks.

Rick