If I shoot a brad or staple in it now, it would be noticeable, right?

Yes, for a little while. Apply glue and re-nail or staple the
woodwork back in place. Any small nail/staple hole can be filled with
wax from an appropriate colored crayon.... just "pencil"/press it in
the hole, rub the "finish" (wax) repair to create heat, softening the
wax and sealing the ding. You can touch-up other small furniture
dings, like from vacuum cleaners, using this quick fix, also.

Sonny