It's mostly stuff you'd learn in Physics 101. If you can get your
hands on a basic physics book, it'll be in the section with all the
statics problems (trusses, pulleys, etc.). I'm sure a good mechanical
engineering text on statics and strengths or the like would have some
good info. I looked briefly for a good online reference but didn't see
one. Perhaps someone else reading this group knows one?

Here's a really brief out-of-my-head description.

When an object is at rest, the sum of all forces on it must be zero.
Some forces want to push or pull an object in one direction, such as
gravity pulling down on an apple hanging from a tree. Given that the
apple is stationary, there must be an equal and opposite force pulling
up on the apple from the stem. Other forces are rotational, such as
the torque experienced by your vanity. If it's not falling off the
wall, then the screws that fix it to the wall must be applying an equal
and opposite torque.

The calculation of torque is pretty simple; it's merely force (usually
expressed in lbs or newtons) times perpendicular distance from the
point about which the rotation occurs.

T = F*D_perpendicular

When I say perpendicular distance, think of it this way. Draw a line
along the direction of the force you're applying (straight up and down
in the case of the gravitational force on your vanity) passing through
the point at which you are applying the force. Then draw another line
perpendicular to that one which passes through the point about which
the rotation occurs. Depending on the direction of the force, this
second line will not necessarily pass through the point where it is
being applied. The perpendicular distance is the distance from your
original line (but not necessarily the point where the force is
applied) to the center of rotation along the perpendicular line. This
becomes important if the force you're applying is directed toward or
away from the pivot-point, rather than tangential to it. Imagine
trying to spin a merry-go-round by pushing straight in toward the
middle. It wouldn't happen. You'd be applying a lot of force several
feet from the center, but the perpendicular distance (and hence the
torque) would be zero.

In the case of your glass-top vanity, the downward force would not be
directed at a single point, but would be spread out evenly from the
wall out to the outer edge. You could find the total torque using
integral calculus, but in this case we can do it more simply. If we
ignore the weight of the sink for simplicity's sake, we can say that
the 100 lbs of glass is spread out evenly over the span from the wall
to a distance 2 feet away (the outer edge of the glass). This is
effectively the same as if the full 100 lbs of force was applied in the
middle of the glass, at a distance of 1 foot from the wall. Granted,
the weight of the sink might not be spread symetrically, but we can
probably ignore that and just say that the two together are applying
~120 lbs of force at a perpendicular distance of about 1 foot, giving
us 120 ft-lbs of torque. If a 200 lb person sat on the edge of the
glass, they would be applying 200 lbs of downward force at a distance
of 2 feet from the pivot point, and hence another 800 ft-lbs of torque.

Meanwhile, assuming your screws hold and the vanity doesn't fall down,
the screws must be applying 120 ft-lbs of force in the opposite
direction (or 920 ft-lbs if the 200 lb guy is still sitting on the
edge). If the screw is 1/2 ft from the pivot point, it must be
applying 240 lbs of force in order to come out to 120 ft-lbs of torque
(or a whopping 1840 lbs of force with the person sitting on the edge).
Of course there will be multiple screws spread out over multiple studs,
so no one screw will have to survive that much force.

Given your three-screw-per-stud design where the top screw is, say, 7
1/2 " from the pivot point at the bottom of the backer board, you'd
probably want to use screws that can support around 300 lbs of tensile
force. I don't think that's a really big deal; you won't have to use
3/4" diameter lag bolts or anything like that.

You can always do a mockup out of cheap lumber, screw it to a wall out
in your garage or somewhere, and sit on it or jump up and down on it or
whatever to convince yourself that it'll hold.

Josh