Button - Red / Blue - Finished in Chrome

The handle on our Linden Single-handled Pull-Out Kitchen Faucet (Venetian Bronze) has become lose, and I'd like to tighten it with an Allen wrench. How should I remove the rubber cap without damaging the cap or scratching the bronze finish? Should I use a small screwdriver to pry it out? Or is there another tool I should use? Will I be able to purchase the rubber hot/cold cap in case I damage it? Thank you. Jim Taber