Because that would also 'short' the wanted input audio input signal to 0v... along with the unwanted noise component.This is also why R2/R3 are usually large values - the two values in parallel (usually) effectively dominate or define the following amplifier stage
input impedance.
Input biasing arrangements like that (for single-supply stages like that, whether transistor input, or opamps) always need three resistors, but there are two ways you can arrange it:
1 - as you diagram shows, or
2 - two resistors to split the rail, cap at midpoint to reference (0v); resistor from
this midpoint junction, to the input device input pin, and the input signal fed-in at its junction with that input device (i.e. opamp input pin or transistor base). There's little to choose between them - except as part of a whole design. (1 is probably the better choice overall, imo)
if you want an example of 1 implemented as a diy mod, it's exactly what
my suggested mod 7 for Naim CD3.5 and CD5 players does - adding the equivalent of R2a and C11 in your diagram, which is then shared by two pairs of R2/R3, one for each channel, which directly biases the following opamp in each channel.
(it's a heck of a lot cheaper & more effective than adding a hicap...)